Skip to content

Add a v1 environment entry without changing the existing homepage - #151

Merged
rickisba merged 3 commits into
mainfrom
docs/v1-homepage-quickstart
Jul 27, 2026
Merged

Add a v1 environment entry without changing the existing homepage#151
rickisba merged 3 commits into
mainfrom
docs/v1-homepage-quickstart

Conversation

@rickisba

@rickisba rickisba commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Purpose

Add a visible entry for the modern CacheRoute environment while preserving the existing root README content and layout.

The homepage is not restructured, shortened, or rewritten. The existing legacy deployment walkthrough, screenshots, requirements, API examples, roadmap, and documentation table remain in place.

Root README changes

Only two small additions are made:

  1. Add v1 Setup to the existing top navigation.
  2. Add one short notice linking to:
    • docs/quickstart_v1.md for the complete modern end-to-end path;
    • env/docker/cu130/README.md for the CUDA 13 image and environment setup.

The notice identifies the modern stack as:

CUDA 13 / PyTorch 2.11 / vLLM 0.25.1 / LMCache 0.5.2

It also warns users not to mix the existing legacy YAML/offloading commands with the LMCache MP startup interface.

No previous homepage text, commands, sections, images, or tables are intentionally removed or reorganized.

Complete v1 guide

Add docs/quickstart_v1.md, covering the single-machine path:

Redis RESP L2
  -> LMCache MP
  -> vLLM + LMCacheMPConnector
  -> Scheduler
  -> KDN Server
  -> knowledge registration and KV build
  -> Proxy
  -> Instance
  -> Client/request
  -> LMCache hit and remote-read metric validation

The guide supports both newly built and already-compatible containers. Existing containers only need the current code, CACHEROUTE_RUNTIME_PROFILE=v1, and the checked-in LMCache/vLLM startup scripts.

Scope

Documentation only. No Dockerfile, dependency, Scheduler, Proxy, Instance, KDN, routing, storage, or runtime behavior is changed.

Diff check

Compared with main, the root README contains only the v1 navigation/notice additions. The prior broad homepage rewrite was reverted. The only non-content diff artifact is the file-end newline marker.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2a5ac400cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/quickstart_v1.md
Comment on lines +170 to +172
cat > data/quickstart/cacheroute_v1.txt <<'EOF'
CacheRoute is a knowledge-oriented LLM scheduling framework. It reduces repeated prefill computation by storing and reusing KVCache blocks for frequently requested external knowledge. The Scheduler selects a KDN server and a Proxy, while the Proxy chooses between text recomputation and KVCache reuse.
EOF

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use a cacheable sample for hit validation

With the documented 256-token chunk size, this 42-word sample is shorter than one reusable chunk; QueueManager._effective_knowledge_len() rounds its knowledge length down to zero. Because the Proxy is then started with the iws strategy and its 100 ms decision margin, the example request is routed through text recomputation rather than KVCache injection, so the later LMCache metric check cannot validate the advertised v1 cache path. Use a knowledge sample containing multiple full chunks or run the functional check with a forced KVCache strategy and verify the applied injection mode.

AGENTS.md reference: AGENTS.md:L236-L245

Useful? React with 👍 / 👎.

Comment thread docs/quickstart_v1.md
Comment on lines +188 to +194
A successful build must report `dumped_keys > 0`. Inspect the generated metadata:

```bash
cat kdn_server/KV_database/<kid>/run_meta.json
```

Expected fields include `"runtime_profile": "v1"`, `"key_formats": ["v1"]`, and a non-zero `dumped_keys` value.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refresh Scheduler knowledge after the KV build

When the KDN starts with an empty database, its registration-triggered Scheduler refresh occurs before this CLI build; completing buildkv_file does not trigger another refresh, and the automatic refresh interval defaults to 30 seconds. If the remaining services are started and the request is sent within that interval, the Scheduler still has no knowledge entry, so RAG: true produces an empty Knowledge_List and bypasses the KDN path. Explicitly call /admin/refresh_knowledge after the build and verify that the new kid is visible and kv_ready before continuing.

AGENTS.md reference: AGENTS.md:L227-L232

Useful? React with 👍 / 👎.

@rickisba rickisba changed the title Document the complete v1 deployment path from the repository homepage Add a v1 environment entry without changing the existing homepage Jul 27, 2026
@rickisba
rickisba merged commit 7a63858 into main Jul 27, 2026
@rickisba
rickisba deleted the docs/v1-homepage-quickstart branch July 27, 2026 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant