# **Calculet Runtime API Documentation** #

## **Table of Contents** ##

- [Installation](./README.md)
- [Quick Start](#quick-start)
- [API Reference](#api-reference)

---

## **Quick Start** ##

```cpp
#include "calrt.hpp"

#include <memory>
#include <thread>

void inference(calrt::VirtualDevice *vDev, calrt::CalbinModel *model, calrt::CalrtInputBuf* inputBuffer, calrt::CalrtOutputBuf* outputBuffer)
{
    calrt::infer(vDev, model, inputBuffer, outputBuffer);
    outputBuffer->wait();
}

void threadFunc(calrt::VirtualDevice *vDev, calrt::CalbinModel &model)
{
    std::unique_ptr<calrt::CalrtInputBuf> pIbuf = calrt::createInputBuf(model);
    std::unique_ptr<calrt::CalrtOutputBuf> pObuf = calrt::createOutputBuf(model);
    inference(vDev, &model, pIbuf.get(), pObuf.get());
}

int main()
{
    auto pCalbin = calrt::Calbin::CreateParser("...");

    auto vdevice = calrt::VirtualDevice::CreateVDevice();
    calrt::configure(vdevice.get(), pCalbin.get());
    
    calrt::CalbinModel models = pCalbin->GetAllModels();
    calrt::CalbinModel modelA = models[0];
    std::unique_ptr<std::thread> threads[2];
    for(int i = 0; i < 2; i++)
    {
        threads[i] = std::make_unique<std::thread>(&threadFunc, vdevice.get(), std::ref(modelA));
    }

    for(int i = 0; i < 2; i++)
    {
        if(threads[i]->joinable())
        {
            threads[i]->join();
        }
    }

    return 0;
}
```

[Back to top](#table-of-contents)

---

## **API Reference** ##

1. [Modules](#modules)
2. [Data Structures](#data-structures)
3. [Data Types](#data-types)

---

## **Modules** ##

- [Configuration](./doc/module.md#configuration)
- [Device](./doc/module.md#device)
  - [Scan Device](./doc/module.md#scan-device)
  - [Create Device](./doc/module.md#create-device)
  - [Trigger Device Execution](./doc/module.md#trigger-device-execution)
  - [Wait Device Execution Finish](./doc/module.md#wait-device-execution-finish)
  - [Reset Device](./doc/module.md#reset-device)
  - [Update Firmware](./doc/module.md#update-firmware)
  - [Get Device Firmware information](./doc/module.md#get-firmware-information)
  - [Get Device Information](./doc/module.md#get-device-information)
  - [Get Device Execution Status](./doc/module.md#get-device-execution-status)
- [Buffer](./doc/module.md#buffer)
  - [Create Buffer](./doc/module.md#create-buffer)
  - [Get Buffer Information](./doc/module.md#get-buffer-information)
  - [Get and Modify Tensors Information](./doc/module.md#tensors-information)
  - [Get Ping-Pong Memory Information](./doc/module.md#ping-pong-memory)
  - [Get LLM Information](./doc/module.md#get-llm-info)
  - [Get and Modify Current Hyperparameter](./doc/module.md#get-and-modify-current-hyperparameter)
- [Inference](./doc/module.md#inference)
  - Infer Model with Device
- [Calbin (STILL UNDER DEVELOPMENT)](./doc/module.md#calbin)
  - [Create Calbin](./doc/module.md#create-calbin)
  - [Get Calbin Information](./doc/module.md#get-calbin)
  - [Get Model Information](./doc/module.md#get-model)
- [Memory Management](./doc/module.md#memory-management)
  - [Register Device Memory](./doc/module.md#register-device-memory)
  - [Allocate Device Memory](./doc/module.md#allocate-device-memory)
  - [Copy from Host Memory to Device Memory](./doc/module.md#copy-from-host-memory-to-device-memory)
  - [Copy from Device Memory to Host Memory](./doc/module.md#copy-from-device-memory-to-host-memory)
  - [Write Device Register](./doc/module.md#write-device-register)
  - [Read Device Register](./doc/module.md#read-device-register)
  - [Free Device Memory](./doc/module.md#free-device-memory)
- [Error Handling](./doc/module.md#error-handling)
  - [CALRT_CHECK](./doc/module.md#calrt-check)
  - [REPORT_CALRT_ERROR](./doc/module.md#report-runtime-error)
  - [REPORT_CALRT_ERROR_IF](./doc/module.md#report-runtime-error-if)

[Back to Top](#table-of-contents)

---

## **Data Structures** ##

- [struct](./doc/struct.md)
  - [CalrtCalbin](./doc/struct.md#calrtcalbin)
  - [CalbinModel](./doc/struct.md#calbinmodel)
  - [CalbinSection_s](./doc/struct.md#calbinsection_s)
  - [CalbinLLM_s](./doc/struct.md#calbinllm_s)
  - [KV_Cache_s](./doc/struct.md#kv_cache_s)
  - [CalbinTensorInfo_s](./doc/struct.md#calbintensorinfo_s)
  - [CalrtDevBuf_s](./doc/struct.md#calrtdevbuf_s)
  - [CalrtBufferInfo_s](./doc/struct.md#calrtbufferinfo_s)
  - [TaskInfo_s](./doc/struct.md#taskinfo_s)
  - [CalrtDeviceId_s](./doc/struct.md#calrtdeviceid_s)
  - [CalrtDeviceInfo_s](./doc/struct.md#calrtdeviceinfo_s)
  - [ModelHyperParameters_s](./doc/struct.md#modelhyperparameters_s)
- [class](./doc/class.md)

[Back to Top](#table-of-contents)

---

## **Data Types** ##

- [enum](./doc/enum.md)
  - [CalDevAccType_e](./doc/enum.md#caldevacctype_e)
  - [CalbinModelType_e](./doc/enum.md#calbinmodeltype_e)
  - [CalbinModelArch_e](./doc/enum.md#calbinmodelarch_e)
  - [ModelChipMode_e](./doc/enum.md#modelchipmode_e)
  - [CalbinSectionPlace_e](./doc/enum.md#calbinsectionplace_e)
  - [CalbinSectionType_e](./doc/enum.md#calbinsectiontype_e)
  - [PrimitiveType](./doc/enum.md#primitivetype)
  - [CalrtBufferDirection_e](./doc/enum.md#calrtbufferdirection_e)
  - [TaskType_e](./doc/enum.md#tasktype_e)
  - [CalJobAlloStatus_e](./doc/enum.md#caljoballostatus_e)
  - [CalrtError_e](./doc/enum.md#calrterror_e)
  - [CalrtDeviceType_e](./doc/enum.md#calrtdevicetype_e)
  - [CalrtOutputBufStatus_e](./doc/enum.md#calrtoutputbufstatus_e)

[Back to Top](#table-of-contents)

---