Repository navigation
v0.6.0: bug hunting on vLLM offloading and LMCache
Three targets, each run with its own code unmodified. vLLM's two CPU
offloading connectors: clean, planted faults caught, storage failures
recovered. LMCache: two real, open bugs found independently and shrunk to
minimal reproductions (both already reported upstream, so no duplicates
filed); Phantom's A/B tests show both open fixes work. SGLang, llm-d and
the NIXL fault classes were not covered (they need GPUs).
- vLLM's
SimpleCPUOffloadConnectoron the CPU backend: its manager runs
unmodified, only the CUDA copy engine is replaced. Sixvllm-fuzz
profiles (cpu-offload*); 0 violations so far, 3,753 blocks stored and
317 loaded back and verified. - Offload copy engine with separate load/store queues (like vLLM's two CUDA
streams); plantedstore-reported-earlycaught; offloading in CI. phantomkv vllm-diff: run a seed twice, show the first step where the
runs differ. Found that the harness stopped stepping while vLLM still had
pending connector work; fixed.- vLLM's
OffloadingConnector(target 2) through its pluggable
OffloadingSpec, no vLLM code patched:offload2*profiles, planted
store-reported-earlycaught, and storage failures (load-fails,
store-fails) recovered correctly by vLLM; in CI. - Partial-chunk evidence, revisit-heavy chunk profiles and
scripts/offload_campaign.sh: 50 seeds per run, 8 profiles clean, planted
fault caught 24/50, storage failures recovered. README Phase 5 section. - LMCache (target 3):
scripts/lmcache_smoke.py, vLLM + LMCache's
LMCacheMPConnector+ a CPU-only LMCache server; 928 tokens stored and
loaded back through LMCache, verified by Phantom. - LMCache under
vllm-fuzz(lmcache*profiles, one server per campaign,
restart fault, planted bug in LMCache's scatter path caught 10/10).
Lead: after a fast server restart vLLM never re-registers with LMCache. lmcache-restart-settleprofile. The restart lead is a known LMCache bug
(issue #4642), found independently; an A/B test of the open fix
(PR #6106) shows caching recovers after every restart.- LMCache lock and object counts after every run (from
/status),
lmcache-only-noabort, and three scripts: a one-abort probe, a lock-leak
shrinker (ddmin with a custom failure test) and a replayer for the saved
minimal cases. The Day 22 replay differences are a read-lock leak on
aborted waiting requests (known upstream, issue #5656); A/B of the open
fix (PR #6540): 0 leaked locks, all seeds identical on replay.