Add a v1 environment entry without changing the existing homepage - #151
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2a5ac400cd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| cat > data/quickstart/cacheroute_v1.txt <<'EOF' | ||
| CacheRoute is a knowledge-oriented LLM scheduling framework. It reduces repeated prefill computation by storing and reusing KVCache blocks for frequently requested external knowledge. The Scheduler selects a KDN server and a Proxy, while the Proxy chooses between text recomputation and KVCache reuse. | ||
| EOF |
There was a problem hiding this comment.
Use a cacheable sample for hit validation
With the documented 256-token chunk size, this 42-word sample is shorter than one reusable chunk; QueueManager._effective_knowledge_len() rounds its knowledge length down to zero. Because the Proxy is then started with the iws strategy and its 100 ms decision margin, the example request is routed through text recomputation rather than KVCache injection, so the later LMCache metric check cannot validate the advertised v1 cache path. Use a knowledge sample containing multiple full chunks or run the functional check with a forced KVCache strategy and verify the applied injection mode.
AGENTS.md reference: AGENTS.md:L236-L245
Useful? React with 👍 / 👎.
| A successful build must report `dumped_keys > 0`. Inspect the generated metadata: | ||
|
|
||
| ```bash | ||
| cat kdn_server/KV_database/<kid>/run_meta.json | ||
| ``` | ||
|
|
||
| Expected fields include `"runtime_profile": "v1"`, `"key_formats": ["v1"]`, and a non-zero `dumped_keys` value. |
There was a problem hiding this comment.
Refresh Scheduler knowledge after the KV build
When the KDN starts with an empty database, its registration-triggered Scheduler refresh occurs before this CLI build; completing buildkv_file does not trigger another refresh, and the automatic refresh interval defaults to 30 seconds. If the remaining services are started and the request is sent within that interval, the Scheduler still has no knowledge entry, so RAG: true produces an empty Knowledge_List and bypasses the KDN path. Explicitly call /admin/refresh_knowledge after the build and verify that the new kid is visible and kv_ready before continuing.
AGENTS.md reference: AGENTS.md:L227-L232
Useful? React with 👍 / 👎.
Purpose
Add a visible entry for the modern CacheRoute environment while preserving the existing root README content and layout.
The homepage is not restructured, shortened, or rewritten. The existing legacy deployment walkthrough, screenshots, requirements, API examples, roadmap, and documentation table remain in place.
Root README changes
Only two small additions are made:
v1 Setupto the existing top navigation.docs/quickstart_v1.mdfor the complete modern end-to-end path;env/docker/cu130/README.mdfor the CUDA 13 image and environment setup.The notice identifies the modern stack as:
It also warns users not to mix the existing legacy YAML/offloading commands with the LMCache MP startup interface.
No previous homepage text, commands, sections, images, or tables are intentionally removed or reorganized.
Complete v1 guide
Add
docs/quickstart_v1.md, covering the single-machine path:The guide supports both newly built and already-compatible containers. Existing containers only need the current code,
CACHEROUTE_RUNTIME_PROFILE=v1, and the checked-in LMCache/vLLM startup scripts.Scope
Documentation only. No Dockerfile, dependency, Scheduler, Proxy, Instance, KDN, routing, storage, or runtime behavior is changed.
Diff check
Compared with
main, the root README contains only the v1 navigation/notice additions. The prior broad homepage rewrite was reverted. The only non-content diff artifact is the file-end newline marker.