Skip to content

Releases

v0.7.0: Phantom complete

Choose a tag to compare

@sameer-sde sameer-sde released this 18 Nov 10:23
23b62ca

All six phases of the roadmap are done.

  • Paper (PDF and LaTeX source in docs/paper/): the method, the
    evaluation with planted and unplanted results kept separate, the two
    LMCache case studies, limitations and related work.
  • Blog post (docs/blog/phantom.md).
  • PyPI page now uses a short description written for PyPI
    (docs/pypi.md); the README's Mermaid diagrams showed as code there.
  • README: a "Results at a glance" table at the top; the opening line now
    names only the engines actually tested (vLLM and LMCache).
  • CI and the publish workflow moved to current action versions (Node 24),
    which removes GitHub's Node 20 deprecation warnings.

v0.6.1: first PyPI release

Choose a tag to compare

@sameer-sde sameer-sde released this 17 Nov 14:51
7ae12fc
  • On PyPI: pip install phantom-kv, then phantomkv fuzz .... No
    dependencies.
  • Published from GitHub Actions with PyPI Trusted Publishing (no stored
    token) when a GitHub release is created; the workflow checks that the
    version matches the tag, runs the tests and twine check first.
  • Package metadata (keywords, classifiers, project links); README links are
    absolute so they work on the PyPI page.

v0.6.0: bug hunting on vLLM offloading and LMCache

Choose a tag to compare

@sameer-sde sameer-sde released this 17 Nov 14:15
ac44665

Three targets, each run with its own code unmodified. vLLM's two CPU
offloading connectors: clean, planted faults caught, storage failures
recovered. LMCache: two real, open bugs found independently and shrunk to
minimal reproductions (both already reported upstream, so no duplicates
filed); Phantom's A/B tests show both open fixes work. SGLang, llm-d and
the NIXL fault classes were not covered (they need GPUs).

  • vLLM's SimpleCPUOffloadConnector on the CPU backend: its manager runs
    unmodified, only the CUDA copy engine is replaced. Six vllm-fuzz
    profiles (cpu-offload*); 0 violations so far, 3,753 blocks stored and
    317 loaded back and verified.
  • Offload copy engine with separate load/store queues (like vLLM's two CUDA
    streams); planted store-reported-early caught; offloading in CI.
  • phantomkv vllm-diff: run a seed twice, show the first step where the
    runs differ. Found that the harness stopped stepping while vLLM still had
    pending connector work; fixed.
  • vLLM's OffloadingConnector (target 2) through its pluggable
    OffloadingSpec, no vLLM code patched: offload2* profiles, planted
    store-reported-early caught, and storage failures (load-fails,
    store-fails) recovered correctly by vLLM; in CI.
  • Partial-chunk evidence, revisit-heavy chunk profiles and
    scripts/offload_campaign.sh: 50 seeds per run, 8 profiles clean, planted
    fault caught 24/50, storage failures recovered. README Phase 5 section.
  • LMCache (target 3): scripts/lmcache_smoke.py, vLLM + LMCache's
    LMCacheMPConnector + a CPU-only LMCache server; 928 tokens stored and
    loaded back through LMCache, verified by Phantom.
  • LMCache under vllm-fuzz (lmcache* profiles, one server per campaign,
    restart fault, planted bug in LMCache's scatter path caught 10/10).
    Lead: after a fast server restart vLLM never re-registers with LMCache.
  • lmcache-restart-settle profile. The restart lead is a known LMCache bug
    (issue #4642), found independently; an A/B test of the open fix
    (PR #6106) shows caching recovers after every restart.
  • LMCache lock and object counts after every run (from /status),
    lmcache-only-noabort, and three scripts: a one-abort probe, a lock-leak
    shrinker (ddmin with a custom failure test) and a replayer for the saved
    minimal cases. The Day 22 replay differences are a read-lock leak on
    aborted waiting requests (known upstream, issue #5656); A/B of the open
    fix (PR #6540): 0 leaked locks, all seeds identical on replay.

v0.5.0: two-engine prefill/decode with transfer faults

Choose a tag to compare

@sameer-sde sameer-sde released this 05 Nov 09:44
39c932c
  • Two real vLLM engines (prefill + decode) exchanging KV through vLLM's
    KV-connector API; Phantom verifies every transferred slot on the decode side.
  • PhantomFaultConnector: planted transfer faults swap (KV of the wrong
    request) and silent-drop (load reported done, nothing written). Both caught.
  • PhantomAsyncFaultConnector: asynchronous loads that land steps later,
    one-sided like RDMA.
  • abort-free-race plant (the SGLang Oct-2026 bug class): an aborted request's
    blocks are freed while its KV is still in flight. Unmodified vLLM stays clean
    (4 late landings handled); with the plant, 54 misplaced-block violations and
    5 corrupted requests.
  • Seeded PD fuzzing (phantomkv pd-fuzz, pd-shrink): each seed picks the
    load delay, transfers, abort timings and fillers. Unmodified vLLM: 0/20
    failing with 35 aborts while KV was in flight. abort-free-race: 14/20
    caught; a failing seed shrinks from 24 events to 2.
  • Fourth transfer fault, duplicate: a stale copy of a KV write lands after
    the request's blocks may have been reused.
  • scripts/pd_campaign.sh, 200 seeds per mode: unmodified vLLM 0/200;
    abort-free-race 115, swap 199, silent-drop 199, duplicate 184
    caught. The seed swap and silent-drop missed read no transferred KV at all.
  • All two-engine runs are part of the real-vllm CI job.

v0.4.0: profiles: chunked prefill, preemption, block sizes

Choose a tag to compare

@sameer-sde sameer-sde released this 05 Nov 08:15
  • vLLM configuration profiles: baseline, chunked-prefill, preemption,
    block64, with engine counters proving each stressed path was exercised.
  • Second planted bug class, missing-touch (refcount). Found that vLLM's own
    assertions catch bugs that make its bookkeeping inconsistent; Phantom is
    needed for bugs that leave it consistent but wrong.
  • In-process engine for exact step ordering and replay; honest verdicts
    (PASS / CAUGHT / VLLM-ASSERT / MISSED / CRASH) in vllm_campaign.sh.
  • Engine errors recorded with file, line and source.

v0.3.0: Phantom inside real vLLM, with CI

Choose a tag to compare

@sameer-sde sameer-sde released this 05 Nov 08:15
  • PhantomForCausalLM: a weightless model registered through
    vllm.general_plugins that writes provenance tags into vLLM's real KV cache
    through its block tables and slot mappings, and verifies every read.
  • Float32-exact tag encoding.
  • Planted stale-prefix-cache bug in vLLM's block pool, caught on 15/20 seeds.
  • Seeded step-by-step harness (phantomkv vllm-fuzz) and shrinking of
    real-vLLM failures (phantomkv vllm-shrink).
  • real-vllm CI job running Phantom against a pinned vLLM CPU wheel.

v0.2.0: shrinking failing runs to minimal repros

Choose a tag to compare

@sameer-sde sameer-sde released this 05 Nov 08:14
  • Delta-debugging minimiser (ddmin): a failing seed becomes a minimal,
    1-minimal event list that still fails.
  • phantomkv shrink command.

v0.1.0: oracle and deterministic simulator

Choose a tag to compare

@sameer-sde sameer-sde released this 05 Nov 08:14
  • Provenance tags and prefix chain hashes; an exact oracle with four
    violation kinds (unwritten-read, misplaced-block, chain-break,
    slot-mismatch).
  • Toy paged-KV engine with prefix cache, preemption, PD transfer and aborts.
  • Deterministic simulator driven by a single seed; fuzz and replay commands.
  • CI running unit tests and a 1,000-seed clean fuzz on every push.