Files
calculet-npu-research-archive/history/topics/05-memory-kv-cache.tsv
T

14 KiB

137e65977fb05e671b1acfb4a11e96d2d6a3a5839cde1d40fc252240b2ccdab0785d53e3643f841932026-05-29T15:45:23+08:00patrickyangfix atomic memory order issues
26f95f35af17c11e7addfc22a6938dc4517d5be6451184d659f00224d1886d88e7e9970d4f3e9c03f2026-05-13T11:30:19+08:00patrickyangfix refresh bug. failed to refresh memory. failed to obtain shm lock
358f7d948f15f0f10989e3ff82ff5aaf029ef1b1983c434001d8dcec264abd87c1d5abf5ee6195699 d2f11fc9077e6a84662eb4aa554b36939858ee872026-04-17T09:41:52+08:00Yunzhe JiaMerge branch 'dev' of http://192.168.10.26:1000/taobo/calrt into dev
4e6c697b4cc5e99a902332322272d75e0d9a4ac16f2aef1942611069afac2b50604d48512ecb9ad602026-04-15T09:48:59+08:00patrickyangadd assert for past kv seq len less than cur seq len
5616337c570262b3ae1c90ed5c473b426e86484b4b5edb72053bf7a54691f0457628b520399e547c52026-04-13T11:27:06+08:00patrickyangrefactor deploy ccu data. improve memory utility. provent write device restricted memory
67d76e34e5388905fa35b20c810b4fdd4f8b343ab325dd2d9e4f1c9ddbcfc5314b266fb9f1f22abee2026-04-07T16:15:33+08:00patrickyangsupport to read write device memory for virtual device
7325dd2d9e4f1c9ddbcfc5314b266fb9f1f22abee75d32605166604a1915837a575676266dd415a582026-04-07T16:12:39+08:00patrickyangupt read and write remote chip memory
875d32605166604a1915837a575676266dd415a58043c157b5cc391585b28371736678555c56a673a2026-04-07T16:11:53+08:00patrickyangprovent user accidently write os memory space
9ce489be9dd92aa2fd97248159446466ac1b982e7258a335889c612e91f3bb0f594190c2060b8cad92026-04-02T16:06:07+08:00patrickyangupt cli function about read and write memory. support different deviceevice and chips
10c8942310d4b1072505766c3779a3f988f7f983a21ca5844b795cb7407b2fc9df2021a03ffc9272572026-04-02T15:59:33+08:00patrickyangsupport read remote chip memory. support ring buffer to deploy weights in 512MB mem usage
11a08691bcfd18262118d1f4ab9c8027884f5b43be22c7d2fab85df22acaf90cb184dc5385290dc9032026-03-31T15:50:31+08:00patrickyangenable init mem allocator by product number
129b946dfdbbe843728b7fc6cecc340144a08fa7e73d553b0918d7c0afb22fa837d50b54216ea5e3d02026-03-10T16:34:12+08:00Yunzhe Jiafix kv issue
13e4bb637a5d7fb3b7d80b8be4cff3bde3745fa30680b5bec7c0bfcb4ba4c5145b15b811f41a67d8962026-02-10T19:28:43+08:00wantaoboupt pastkv bug
1480b5bec7c0bfcb4ba4c5145b15b811f41a67d8968cfa512ffbb271f1ab952649b7a0692abe75b21f2026-02-10T17:47:43+08:00wantaoboupt pastkv model runner
15d42879ffdfc48c015dd386fb5dae89bfe6d3f012e9f95f4f06bf0742eda6649a322f327c2dfa5b382026-02-06T15:40:11+08:00patrickyangimplement past kv model runner
168eb0217d081caad6ea550075909048beee90778c9f91de70b8852666a12061a9042e2618e67ab98f2026-02-06T15:37:25+08:00patrickyangimplement past kv model runner
17e31e719a66354bb00eebaba21fe56836059253ba08f3a6159c3807465db5f5c74918e8ce85269a3b2026-02-05T16:52:58+08:00patrickyangupdate deploy function. now automatically refresh device memory each time when deploy calbin
1808f3a6159c3807465db5f5c74918e8ce85269a3bf52b519897fc32cace5fc878170ae22821975d5b2026-02-05T14:43:00+08:00patrickyangremove device address with offset
19a298898a064a9f854d021597827ec4d84d09d3e478e5765d213aa9a095604efac528a63c9f4eb8072026-01-04T14:01:17+08:00patrickyangfix bug causing segmentation fault due to mis-manage buffer life time
201f96edfc0a223fa4c79816edcf28ca8f934ab246df1842115d1cde562a9f12a7b3ca41d34bcf1b0c2025-12-29T16:37:47+08:00wantaobo[model runner] update hparam to cur_seq_len and past_kv_cur_seq_len
21ab5a3bd62c0931d5f382f7fba16e115961e9c7962ef07a46f910fbc81798bd7da2ef965f8c92669b2025-12-18T15:17:04+08:00patrickyangoutput buffer now support more status check for ccu failed to exec
22a677677e8660a454c46691effe181374d7a393a0d752424487a03856b3ba765ffd7d5550261d39b42025-12-15T15:04:58+08:00patrickyangsupport print ccu prog addr with virtual addr and physical addr
23d752424487a03856b3ba765ffd7d5550261d39b4fe0bcf650a3f3782bc51b44a0e60fc47f2f3c2d12025-12-15T15:03:47+08:00patrickyangsupport read ccu program start address
24e919b71433dbfbab0b83c8ad4233d1519fb59188240646b084e02e6b90ea82073198a2b13e1bc68d2025-12-08T16:41:06+08:00patrickyangupdate buffer to init hyper-param
250389ad48db464669155ff423b9eb5efbb189c8a85015afd73c96d28c875d7250c85c79676afadb702025-12-04T10:52:25+08:00patrickyangfix allocator bug for no corrctly free memory
2616d8256b407fe02450864c6f08a47e028e8d8969068288e034eb11d6056d8fa68b2cc9e54f63818a2025-11-27T18:01:07+08:00patrickyangallocate ccu1 align 1kb size
279f551dd8a40c3855e65e8cea6812515655d67c5990747c80fe69a6cffe5ddeca4a70b6538f3e15942025-11-13T16:21:36+08:00wantaoboupt demo for tensor address check for both dram and sram
28321da3e562974471bf50af7a5650f1dd60f8e50bf62907dab76e22d9c6c91ace5f26fac8abcbb76c2025-11-05T18:00:50+08:00patrickyangupdate python buffer interface
29d148f07d1c4d86199ea88265e7f31575dbb2ab7abe6a6f8abdb02ff7b22377147ade0cb13b87c2a52025-11-04T14:42:48+08:00patrickyangsupport memory write by ptr
30ccf054ff5147b5291d5f9b697e74a36173e8c660b6490722cf48f1de5fce4d11308badfd3f673c872025-10-28T14:58:02+08:00patrickyangprovide get model config info and kv cache info interface.
313dd38465558fe30592b4d80bff700cc853c4411c48724412c5f229c0264b32f02aa0f747c6339b7e2025-10-28T14:19:57+08:00ylsunadd calbin buffer test
32b8430b8c13022f8448ca049823f5bd8d0864919deca251cb15521e2df9d3c15c584902fe7a3e43552025-10-15T17:24:13+08:00patrickyangstop tracking files under __pycache__
338dc5c264532840a87e865389b60e1b46a82643cff90b5e2a6d0928e1f533b6d814c6a7e388e70fcc2025-10-15T09:59:49+08:00patrickyangupdate io buffer memory check
34f8f3b91034ce45c174b825191b188b3511e6dad567d7cb8dbfb9a7fd265982086a8012433ed53ea22025-10-13T16:22:53+08:00patrickyangupdate i/o buffer check
35c6937e3caa1e7447c8162acd3172d6a017a4f5c00fd75f0b76ed5c482db7e7355da457f94a7818872025-09-30T17:52:14+08:00patrickyangupdate tensor and buffer
360fd75f0b76ed5c482db7e7355da457f94a7818870e36a3ec8c26ccd7f4dd956a9f230591b61a594b2025-09-30T15:53:12+08:00patrickyangupdate buffer
375fc8080bdff8b5360046be3e6fecaa321378b2e43d908daffbe91915f238cce601e207e27ab656252025-09-17T15:32:10+08:00patrickyangupdate memory reservation
3866e9254a9a90042ac8a6eb9529a083fab21880bc18c9cb15118f87968f1fb47e1cbd9738289005e72025-09-13T16:20:06+08:00patrickyangimplement dump device memory, etc info into bin file.
3956da9c353adf990a8f00dc4630a33a668d81e28af508615c536e768722ec9940945f344e46d7dd7d2025-08-25T17:28:27+08:00patrickyangupdate configure to improve memory utiliation under single chip
40c993bbcb42f564276f47372c2d076a7ab31b1f403b3365ead89280728d3418d9bc1a447b74f8cb2e2025-08-19T15:40:08+08:00patrickyangupdate output buffer status
411e32fe04b9ed06993cf6b238a45a13303e4791e3046cafe33cf5f0f31d5748a3b8bf71ac0e865db12025-08-19T12:46:22+08:00patrickyangupdate allocator. disable NBU_CHECK
423d3f907487b08913f38db89dca1054bd3544c9ddfe1cb970af003441d7b26ed3ddcc2ce101e32d7a2025-08-11T15:45:58+08:00patrickyangupdate kv cache specs
43466c8f2784675906b51f97c4295748460f70834b6f7e8ebfd73c7b1eb218eab561bdc63d184dc3722025-08-08T16:44:24+08:00wantaoboupt kv_spec support
442767422ee6be145954ad33e910c2f06b1eca9916f916ab8bb95edca9962602e8397cddcf39b7222f2025-08-08T15:29:22+08:00patrickyangupdate configure function. add more check for kv-update
452a8983673735ad5610670f87643971123af43aef049d755ed8e8205ae7c3df0381672eda832657c42025-07-31T16:16:23+08:00patrickyangfix error causing reset all memory information including OS. Now when reset device memory, only OS mem info is reserved
46d41c5794e5771758a9b54f5efc0b5a774e6f35f6fae87906d397c98ec5505db70a5334bd93272fbb2025-07-24T09:01:47+08:00patrickyangfix buffer construction issues
475984865633c837dd8df76f0c5aebb2237e0d0afd6c41e148980c7d047cb76dce7da7dfd31d26c0362025-07-16T17:36:01+08:00patrickyangfix potential issues trigering by parsing kv info
48a59a56dc38555f02280599f459d68df6a51e5d5ec91bb35ea325310467f9468d67c61e16fc5905652025-07-11T18:44:38+08:00patrickyangupdate calrt input buffer. provide kv cache interface
49d4407cb779f738d1fe7e18b08187c6bc03587894056cb015cd60a45a31b9b9279aa009e72424bcd82025-07-10T18:05:01+08:00patrickyangenable to parse and print kv cache info. [temp]
50a5e2ef41dc68ebff2a3eaf41443eaf3eddd21eda32266fb7d2389cbbbf53be6ae9bb31cf318a0b732025-06-12T17:48:15+08:00patrickyangsupport allocate sram memory. support deploy ccu in sram. support reporting sram memory usage
515751843b9ac30f91b68b99eb030669c3e5e27dea170c4e10f294d1fa4c7f073c008418b762c887082025-06-12T10:51:46+08:00patrickyangupdate set MHCR for flush cache in non-os env pld
524db69c7daabab11fb97c7cf1ac519ae02363a0fb3bc24613cd34276136ee8b2893b3ec626f74872c2025-06-05T16:05:11+08:00patrickyang[refactor RT] remove unused header file. re-organize device class and vdevice class. Uniform interface of create calculet device. Display static of device memory usage. lighten header files.
535e1fe1ad5ba6cebf076002590e31322b3fae0ea0e934463f0408e2a238ce88d423cbc0bffadb0d1f2025-06-04T16:23:29+08:00patrickyangfix potential issues causing elf ddr address wrrorng
547c9ef26bcb1c36b9f8ce987ee39118d668b663a1596fd8b1d7043bf7a9abc4ade36ae86e9722ae332025-05-30T11:11:56+08:00patrickyangrefactor name style. free allocated memory from inferlanucher
55a362ef95425d09dcad988f1e6e93534e4e354d8bc7cbed02bb2ba8ab4d9465b66ed1ddebbb8563442025-04-25T19:33:45+08:00wantaoboupt demo for memory reserved info
56523cbc0385db6ee32a6eafff643638c3cef0a25eb363e285c0502846b769a09cfcfe3fa56e9ed8782025-04-16T17:32:44+08:00ylsunfix calrt memory copy to/from device tensor size bug
57cf943662338348b240d900c477016adee1d824df273a72072d567f57442edc12ef2b1f4e228dd3c12025-04-16T14:45:07+08:00patrickyangfix duplicated vaddress convert
58bc6c7b7180999296b8ac7b3ffe1b0780c09145f492a46765b5bc32394e3473df56fcf45d6c6ab1122025-04-15T17:01:20+08:00patrickyangupdate boundary check for i/o buffer. fix calback function error, leading to bad function call
5992a46765b5bc32394e3473df56fcf45d6c6ab112ed8bc3cd2b0e65452b236d5480217d106ebbce292025-04-15T12:05:36+08:00patrickyangadd reserved mem info recorder. Add thread safe for multi-threads allocate mem. improve log system during parsing calbin and deploy. Implement callback function in Job class to release sync unit
60ee3b24212fc1a184183341ccb1ccab4917fa878be56664b044ab808dea2194f731e8afa700438a392025-04-11T15:41:43+08:00patrickyangadd offset for input buffer and output buffer
61eda04f64a76defe9f5b1e15f04e623cd7ca0cbceaa45692d886823173d547ed60dd1f4d8fb4fd86d2025-03-31T11:26:51+08:00patrickyangadd shutdown to job engine destructor. remove get allocator function. add constrain during parsing calbin
62ddb137f6337b5705d4e4b49e4bf0e3e09395ce6db9b0b417e0f3853050551f55017a91c5186a18ae2025-03-28T17:12:08+08:00patrickyangadd switch for memory monitor
6378ed2f51fe0f8fdfd6e40cc4039bb809c7cd84c92f0e0d67ff7a4960c5210324e2cb7e55ba95176a2025-03-28T17:08:06+08:00wantaobocalrt gen infer launcher, udpate Jump addr size bug
64e2342f3d8e660395d8e66686f4bd25a760e89b1d671e780c47bd9f43bb7a57af627739431b03d2d52025-03-27T17:26:15+08:00patrickyangadd constraint for i/o buffer
6578e3170622a41040c1697c5fe4507282687654833f6c6352cfffa5954c1ea6bf407eeeeda6180bce ea7595ba1b1f1dac427392c6249df8fb49e10ef72025-03-26T14:59:03+08:00patrickyangimplement parse elf function as template to parse either 32-bit ELF or 64-bit ELF. Enable stream for loguru. Improve fmt_custom for error reporting. Constrain i/o buffer ping-pong address
663f6c6352cfffa5954c1ea6bf407eeeeda6180bcecccd106b6f883926393bfc9d9fda6f9b4e0fdd702025-03-26T14:58:06+08:00patrickyangimplement parse elf function as template to parse either 32-bit ELF or 64-bit ELF. Enable stream for loguru. Improve fmt_custom for error reporting. Constrain i/o buffer ping-pong address
67cccd106b6f883926393bfc9d9fda6f9b4e0fdd702ab8eaaecebeb4f1e175b93e8f3f07d41f451eab2025-03-26T10:14:20+08:00patrickyangupdate allocated addres align to 1MB
682ab8eaaecebeb4f1e175b93e8f3f07d41f451eab5ae0649f977e4ff80c677e9a9c1f35c447237e632025-03-26T09:33:55+08:00patrickyangadd memory check for i/o buffer
69be68ee70c4729e0e145789ee3372259815658117adcacad8eff12ffc9ce1b9e3fc10443a118a5f53 99ce200bc4bc220c308126a8258fe81ee2046bc92025-03-24T19:29:55+08:00patrickyangupdate allocator with mem table
70adcacad8eff12ffc9ce1b9e3fc10443a118a5f53b011b5d4e71eaf0ffb55aab0025d13327699ddd32025-03-24T19:28:36+08:00patrickyangupdate allocator function
7199ce200bc4bc220c308126a8258fe81ee2046bc9b011b5d4e71eaf0ffb55aab0025d13327699ddd32025-03-24T19:25:09+08:00patrickyangupdate allocate mem table
7283337a6eab233a3c7a99beea7646724145251957c77e0b84359c58671a29e96cf2a95d7d5e4d2b7a2025-03-24T17:56:36+08:00patrickyangupdate calrt_allocator
73c77e0b84359c58671a29e96cf2a95d7d5e4d2b7a1c6a3c94dde7f6fc475daf8f81d609aa027fc9fa2025-03-24T17:28:02+08:00patrickyangupdate interval map for register allocated memory info
741c6a3c94dde7f6fc475daf8f81d609aa027fc9fa9d02f6a5442f1ddd9dcd66298adb20552bfe6f6e2025-03-24T17:27:30+08:00patrickyangupdate calrt allocator
75df817cd7e4495c95afb71276063dbd80349f98c8212e409009b43dff8149d1ff64cf510beca00b0d2025-03-21T17:52:31+08:00patrickyangbug-free
76212e409009b43dff8149d1ff64cf510beca00b0d95f48576f70e3b8072aac0c722e5ba3cde576edc2025-03-21T17:47:08+08:00patrickyangadd memory state cmd
77bff6fc00d9010fbe70af1ae4565bdb81e6d4b086a73d34d8856f499649fa1e9ffd75a226fea5d71d2025-03-21T17:31:43+08:00patrickyangfix ccu0 place addr
785f81d2d20ec60bfadeb123d57cb8ff159e4dc4dd50ffd30c3c878f522f10ae5338c0453dec12e2792025-03-13T15:31:01+08:00wantaoboudpate emu device to add bar2 base addr
7966cbb77609120008ee4ac252d66cffba6b887b52e609d9efc620abf28402d3ad46be83ee7f0aa4382025-03-10T11:49:22+08:00patrickyangencryp compiler::allocator to calrt_allocator
8047c825dba1fff93a8da32c51885f4268b8c26be1862de6768c409de5ea58b6c1f813b76b04b5a0442025-03-06T17:12:44+08:00patrickyangupdate deploy so, and allocator
81ed2470f069381d474ae5c83c55075be40c1362339d5271feffba66cf420479dcdffbff7cdd59e9d02025-03-05T14:37:27+08:00patrickyang[Mem allocator]introduce device memory allocator.
8209be6d18a5adc9ef015b63ef0280feeae21544a26fe49ba5582f46a3e545078026abce3aed4807b52025-03-04T17:51:44+08:00wantaobochange name stream to buffer
83a182db9d0891476462344813c1e09c62fe24ef4bfbf6377a2c66a5e73bec071f05b4a1edfeb663cb2025-03-04T11:26:10+08:00patrickyangupdate from uint64_t* to uint8_t for host address
84ce9efecf56d26417faf2d6d3de6bf5cb9bd27d5e9af274bc7bac7b57738675876a1414a560b30f772025-01-23T17:51:36+08:00patrickyangintroduce allocator to device class; discard never-used files