155 lines
5.3 KiB
Markdown
155 lines
5.3 KiB
Markdown
# **Calculet Runtime API Documentation** #
|
|
|
|
## **Table of Contents** ##
|
|
|
|
- [Installation](./README.md)
|
|
- [Quick Start](#quick-start)
|
|
- [API Reference](#api-reference)
|
|
|
|
---
|
|
|
|
## **Quick Start** ##
|
|
|
|
```cpp
|
|
#include "calrt.hpp"
|
|
|
|
#include <memory>
|
|
#include <thread>
|
|
|
|
void inference(calrt::VirtualDevice *vDev, calrt::CalbinModel *model, calrt::CalrtInputBuf* inputBuffer, calrt::CalrtOutputBuf* outputBuffer)
|
|
{
|
|
calrt::infer(vDev, model, inputBuffer, outputBuffer);
|
|
outputBuffer->wait();
|
|
}
|
|
|
|
void threadFunc(calrt::VirtualDevice *vDev, calrt::CalbinModel &model)
|
|
{
|
|
std::unique_ptr<calrt::CalrtInputBuf> pIbuf = calrt::createInputBuf(model);
|
|
std::unique_ptr<calrt::CalrtOutputBuf> pObuf = calrt::createOutputBuf(model);
|
|
inference(vDev, &model, pIbuf.get(), pObuf.get());
|
|
}
|
|
|
|
int main()
|
|
{
|
|
auto pCalbin = calrt::Calbin::CreateParser("...");
|
|
|
|
auto vdevice = calrt::VirtualDevice::CreateVDevice();
|
|
calrt::configure(vdevice.get(), pCalbin.get());
|
|
|
|
calrt::CalbinModel models = pCalbin->GetAllModels();
|
|
calrt::CalbinModel modelA = models[0];
|
|
std::unique_ptr<std::thread> threads[2];
|
|
for(int i = 0; i < 2; i++)
|
|
{
|
|
threads[i] = std::make_unique<std::thread>(&threadFunc, vdevice.get(), std::ref(modelA));
|
|
}
|
|
|
|
for(int i = 0; i < 2; i++)
|
|
{
|
|
if(threads[i]->joinable())
|
|
{
|
|
threads[i]->join();
|
|
}
|
|
}
|
|
|
|
return 0;
|
|
}
|
|
```
|
|
|
|
[Back to top](#table-of-contents)
|
|
|
|
---
|
|
|
|
## **API Reference** ##
|
|
|
|
1. [Modules](#modules)
|
|
2. [Data Structures](#data-structures)
|
|
3. [Data Types](#data-types)
|
|
|
|
---
|
|
|
|
## **Modules** ##
|
|
|
|
- [Configuration](./doc/module.md#configuration)
|
|
- [Device](./doc/module.md#device)
|
|
- [Scan Device](./doc/module.md#scan-device)
|
|
- [Create Device](./doc/module.md#create-device)
|
|
- [Trigger Device Execution](./doc/module.md#trigger-device-execution)
|
|
- [Wait Device Execution Finish](./doc/module.md#wait-device-execution-finish)
|
|
- [Reset Device](./doc/module.md#reset-device)
|
|
- [Update Firmware](./doc/module.md#update-firmware)
|
|
- [Get Device Firmware information](./doc/module.md#get-firmware-information)
|
|
- [Get Device Information](./doc/module.md#get-device-information)
|
|
- [Get Device Execution Status](./doc/module.md#get-device-execution-status)
|
|
- [Buffer](./doc/module.md#buffer)
|
|
- [Create Buffer](./doc/module.md#create-buffer)
|
|
- [Get Buffer Information](./doc/module.md#get-buffer-information)
|
|
- [Get and Modify Tensors Information](./doc/module.md#tensors-information)
|
|
- [Get Ping-Pong Memory Information](./doc/module.md#ping-pong-memory)
|
|
- [Get LLM Information](./doc/module.md#get-llm-info)
|
|
- [Get and Modify Current Hyperparameter](./doc/module.md#get-and-modify-current-hyperparameter)
|
|
- [Inference](./doc/module.md#inference)
|
|
- Infer Model with Device
|
|
- [Calbin (STILL UNDER DEVELOPMENT)](./doc/module.md#calbin)
|
|
- [Create Calbin](./doc/module.md#create-calbin)
|
|
- [Get Calbin Information](./doc/module.md#get-calbin)
|
|
- [Get Model Information](./doc/module.md#get-model)
|
|
- [Memory Management](./doc/module.md#memory-management)
|
|
- [Register Device Memory](./doc/module.md#register-device-memory)
|
|
- [Allocate Device Memory](./doc/module.md#allocate-device-memory)
|
|
- [Copy from Host Memory to Device Memory](./doc/module.md#copy-from-host-memory-to-device-memory)
|
|
- [Copy from Device Memory to Host Memory](./doc/module.md#copy-from-device-memory-to-host-memory)
|
|
- [Write Device Register](./doc/module.md#write-device-register)
|
|
- [Read Device Register](./doc/module.md#read-device-register)
|
|
- [Free Device Memory](./doc/module.md#free-device-memory)
|
|
- [Error Handling](./doc/module.md#error-handling)
|
|
- [CALRT_CHECK](./doc/module.md#calrt-check)
|
|
- [REPORT_CALRT_ERROR](./doc/module.md#report-runtime-error)
|
|
- [REPORT_CALRT_ERROR_IF](./doc/module.md#report-runtime-error-if)
|
|
|
|
[Back to Top](#table-of-contents)
|
|
|
|
---
|
|
|
|
## **Data Structures** ##
|
|
|
|
- [struct](./doc/struct.md)
|
|
- [CalrtCalbin](./doc/struct.md#calrtcalbin)
|
|
- [CalbinModel](./doc/struct.md#calbinmodel)
|
|
- [CalbinSection_s](./doc/struct.md#calbinsection_s)
|
|
- [CalbinLLM_s](./doc/struct.md#calbinllm_s)
|
|
- [KV_Cache_s](./doc/struct.md#kv_cache_s)
|
|
- [CalbinTensorInfo_s](./doc/struct.md#calbintensorinfo_s)
|
|
- [CalrtDevBuf_s](./doc/struct.md#calrtdevbuf_s)
|
|
- [CalrtBufferInfo_s](./doc/struct.md#calrtbufferinfo_s)
|
|
- [TaskInfo_s](./doc/struct.md#taskinfo_s)
|
|
- [CalrtDeviceId_s](./doc/struct.md#calrtdeviceid_s)
|
|
- [CalrtDeviceInfo_s](./doc/struct.md#calrtdeviceinfo_s)
|
|
- [ModelHyperParameters_s](./doc/struct.md#modelhyperparameters_s)
|
|
- [class](./doc/class.md)
|
|
|
|
[Back to Top](#table-of-contents)
|
|
|
|
---
|
|
|
|
## **Data Types** ##
|
|
|
|
- [enum](./doc/enum.md)
|
|
- [CalDevAccType_e](./doc/enum.md#caldevacctype_e)
|
|
- [CalbinModelType_e](./doc/enum.md#calbinmodeltype_e)
|
|
- [CalbinModelArch_e](./doc/enum.md#calbinmodelarch_e)
|
|
- [ModelChipMode_e](./doc/enum.md#modelchipmode_e)
|
|
- [CalbinSectionPlace_e](./doc/enum.md#calbinsectionplace_e)
|
|
- [CalbinSectionType_e](./doc/enum.md#calbinsectiontype_e)
|
|
- [PrimitiveType](./doc/enum.md#primitivetype)
|
|
- [CalrtBufferDirection_e](./doc/enum.md#calrtbufferdirection_e)
|
|
- [TaskType_e](./doc/enum.md#tasktype_e)
|
|
- [CalJobAlloStatus_e](./doc/enum.md#caljoballostatus_e)
|
|
- [CalrtError_e](./doc/enum.md#calrterror_e)
|
|
- [CalrtDeviceType_e](./doc/enum.md#calrtdevicetype_e)
|
|
- [CalrtOutputBufStatus_e](./doc/enum.md#calrtoutputbufstatus_e)
|
|
|
|
[Back to Top](#table-of-contents)
|
|
|
|
---
|