Skip to content

gridbook 0.1.0

Choose a tag to compare

@RobTand RobTand released this 28 Jul 19:28

First release in which pip install gridbook produces a working plugin.

pip install gridbook   # into an environment that already has vLLM + torch

Why this is the first usable release

Before this, the CUDA sources lived outside the Python package while cuda_ext resolved them repo-root-relative, so any non-editable install found no kernels and fell silently to the correctness-only Triton path — one stderr warning as the only signal. The sources now ship inside the package (gridbook/csrc) as package-data. Reported from the field in #1.

Serving fixes (#1)

  • Dense CB weights are resident once, not twice. process_weights_after_loading built a padded copy and left the original parameter live beside it. On the published Qwen3.6-27B artifact (21.38 GiB on disk): model weights 35.86 → 20.82 GiB, KV cache 57.09 → 72.04 GiB, KV capacity 1.40M → 1.76M tokens. Invisible on a 128 GB box, fatal on a 32 GB card.
  • Fused-module scheme lookup is namespace-robust. Probing a single namespace could miss mapper-transformed target keys and drop a layer to BF16. Now one shared resolver across all lookup sites.

Thanks to @josephduerr for both reports.

Verified before release

Wheel installed non-editably from PyPI into a clean container, kernels JIT-built from dist-packages/gridbook/csrc, entry point registered, and the 27B artifact served from that install with output byte-identical to a source checkout. GPU battery 363 passed / 0 failed.

Known limits

tp=1 only. Blackwell sm_120/sm_121 for the native path; other NVIDIA GPUs fall back to correct-but-slow kernels. See the compatibility table in the README.