[Feat] Merge 0.0.1 back into main #48
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adapt vllm v0.9.2 in uc_connector
[Doc] Outline of the document
* remove impl test and add uc connector test * remove contiguous * add judge before synchronize
[Fix] Add asynchronize copy non blocking argument."
[Feature] Add DRAM Connector for uc_connector
[Feature] ucm_nfs_store.py
[doc] add readme and license
[Feature] Add Dockerfiles
[Feature]Nfsstore
[doc] change docs outline
[Feature] Add Cmake build command in setup.py
[fixbug] fix issue#25 issue#31 and issue#33
…7 #29 #42) (#41) * [Fix] Fix the example and make it runable without reconfiguration, close #29 * [Docs] Add performance data in documents, close #37 * [Fix] Fix [0.0.1-release] vllm server crashed during performance test #42
* [Feat] Move kv_block_size to config * [Fix] Fix the python hash seed: must be random or integer in range[0:4294967295]
* nfs store modify ucm_nfs_store * Add NFS Connector Doc And Performance * Modify param kv_cache_size * Modify pictures * modify docs * Add 8K DRAM Conn performance exceed limit --------- Co-authored-by: Celina <hewenxun@yeah.net>
…alid#45 #46 (#47) * [doc] Add export of device type in installation;[Fix] fix version invalid #45 #46 * [Style] Remove version file
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the 1st release candidate of v0.0.1 for unified cache management. Please follow the official doc to get started.
Highlights
Based on vLLM v0.9.2 and vLLM-Ascend v0.9.2rc1, enable prefix caching
Provided dram connector based on class UcmKVStoreBase, tested with Qwen model. #18
Provided nfs store connector based on class UcmKVStoreBase, tested with Qwen model. #44
Docs
All documents can be indexed through the README on the main page.
[Doc] Add README, #24
[Doc] Add installcations, #17
[Doc] Add clarification for dram connector, #44
[Doc] Add clarification for nfs store connector, 91cc283
Core
Fill in the code framework to make it runnable, 94356b5
Add dram connector, #18
Add nfs store connector, #44