Codebase for "Sim-LLM: Optimizing LLM Inference at the Edge through Inter-Task KV Reuse"
Evaluation code will be released soon.
-
Notifications
You must be signed in to change notification settings - Fork 0
CGCL-codes/SimLLM
About
Code base for "Sim-LLM: Efficient Inference Optimization Based on Task Similarity for Large Language Models in Edge"
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published