Skip to content

CacheRoute-v0.1.1

Choose a tag to compare

View all tags
@BJTU-ANT BJTU-ANT released this 31 Jan 02:41
v0.1.1
dfd2634
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The basic CacheRoute architecture.
Finish the KDN_server.
Finish the interface between the scheduler and KDN servers. The scheduler can snapshot knowledge from the KDN server at lifespan and update dynamically.
Finish the interface between the scheduler and the proxy. The scheduler can maintain a dynamic proxy pool in the control plane.
Finish the interface between the proxy and the instance. The proxy can maintain a dynamic instance pool in the control plane.

The knowledge-oriented routing of the scheduler strategy TBD.
The scheduler maintains the status of tasks TBD.
Proxy parallel queue strategy TBD.
Scheduler/proxy resource updater TBD.
The KDN UI for easy use, TBD.
The instance resource collector above vLLM TBD.

Assets 2
Loading