Skip to content

v0.5.0: two-engine prefill/decode with transfer faults

Choose a tag to compare

@sameer-sde sameer-sde released this 05 Nov 09:44
· 71 commits to main since this release
39c932c
  • Two real vLLM engines (prefill + decode) exchanging KV through vLLM's
    KV-connector API; Phantom verifies every transferred slot on the decode side.
  • PhantomFaultConnector: planted transfer faults swap (KV of the wrong
    request) and silent-drop (load reported done, nothing written). Both caught.
  • PhantomAsyncFaultConnector: asynchronous loads that land steps later,
    one-sided like RDMA.
  • abort-free-race plant (the SGLang Oct-2026 bug class): an aborted request's
    blocks are freed while its KV is still in flight. Unmodified vLLM stays clean
    (4 late landings handled); with the plant, 54 misplaced-block violations and
    5 corrupted requests.
  • Seeded PD fuzzing (phantomkv pd-fuzz, pd-shrink): each seed picks the
    load delay, transfers, abort timings and fillers. Unmodified vLLM: 0/20
    failing with 35 aborts while KV was in flight. abort-free-race: 14/20
    caught; a failing seed shrinks from 24 events to 2.
  • Fourth transfer fault, duplicate: a stale copy of a KV write lands after
    the request's blocks may have been reused.
  • scripts/pd_campaign.sh, 200 seeds per mode: unmodified vLLM 0/200;
    abort-free-race 115, swap 199, silent-drop 199, duplicate 184
    caught. The seed swap and silent-drop missed read no transferred KV at all.
  • All two-engine runs are part of the real-vllm CI job.