TensorTorrent 0.3.4

0.3.4

• Runtime: CUDA/ROCm copy vs compute streams so parameter H2D can overlap GEMM (prefetch_distance); Transfer records a CUDA event, Compute waits on the compute stream, Release/collect still synchronize before dropping storage.
• Runtime: overflow host→device copies page-lock the transferred tensor (cached) even when the full model exceeds the pinned-host pool.
• Runtime: recycle in-flight H2D device buffers (shape/dtype pool) instead of allocating a new dest tensor every Transfer.
• Compile: partition FX graphs on repeating block ids (layers.N / blocks.N / h.N) so a transformer layer is one region even when max_region_nodes would slice it.
• Compile: beyond-VRAM bakeoff measures static GPU-prefix + CPU-overflow (Accelerate device_map=auto analog) against streamed GPU and fused CPU.
• Compile: fit-in-VRAM auto skips torch.export and runs the original module on CUDA when weights fit the hoist budget.
• Runtime: export-free eager GPU DirectPlan captures a CUDA graph after warmup (not used with Inductor).
• Benchmarks: optional --suite generate (not in all) times static padded greedy decode vs HF/Accelerate KV generate(); hetero two_gpu measures compile + concurrent GEMM when two CUDA devices exist.
• Known: native Transfer can fail at crossover 1.00×/1.10× VRAM and 2/4 GiB budget arms (those cases used transfer_evict on 0.3.2).

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md

TensorTorrent 0.3.2

0.3.2

• Production hardening: CompiledModule owns CapacityLedger leases (no serve-side / ContextVar dual ownership); cancel generations; serve timeouts cancel per-request tokens only; hoist OOM demotes for the current schedule generation without permanently disabling hoist; shared live-VRAM hoist clamp; fail-closed zero device/disk budgets.
• Runtime perf: DirectPlan keeps ambient intra-op threads; skip redundant .to / on-device input Transfer work; skip host param republish when device-hoisted.
• Serve cancel-token schedule fallback reuses single-region DirectPlan device parameter copies (same cache seed as DataflowDirectPlan).
• Host capacity: live-available RAM budgets no longer double-count resident model bytes; explicit ram_budget_bytes still reserves base state.
• Bakeoff: streaming timing that falls back to planner prediction is labeled measured=False with explicit provenance in notes/metadata.
• CI: build-only Dockerfile.cuda gate on PRs/main (no GPU runtime); longer UV HTTP timeout for large CUDA torch wheels.
• Docs: runtime / budgets / serve / training / FAQ aligned with the above.
• CI: remove temporary helper scripts that broke ruff format on main.

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md

TensorTorrent 0.3.1

0.3.1

• Planner/runtime: hoist resident parameters only when state fits
  ACCELERATOR_REGION_STATE_FRACTION (0.70) of the same effective VRAM as region
  budgets (min(allocatable, vram_budget_bytes)), so near-VRAM fits stream via
  Transfer/Evict instead of full residency OOMing on workspace (fixes
  non-monotonic 0.75× crossover).
• Benchmarks: refuse freeze_published from a dirty worktree unless
  --allow-dirty; enrich environment provenance; Qwen CPU eager uses multiple
  timed samples; host abort peak factor 2.5× weights; GPU-eager fit probe labeled
  as feasibility (not timed); crossover records execution_strategy.
• Cleanup: public suite is the Makefile/run_everything entrypoint; drop alias
  JSON names from public writes; package/docs aligned for clean remasure.
• Public evidence remasured from clean commit fb503e5 (git_dirty=false) under
  benchmarks/published/2026-08-09/ (Qwen + crossover remasured for release hygiene).

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md

TensorTorrent 0.3.0

0.3.0

• Public capacity launch suite: python -m benchmarks.public (fit / DeepMLP /
  HF transformer / budget / crossover / hetero) with RAM-safe subprocess
  isolation and frozen evidence under benchmarks/published/.
• Beyond-VRAM correctness: skip fusion when parameters exceed accelerator
  region budget; strip export CPU device asserts; disable buffer reuse when
  the schedule uses Transfer/Evict/Load/Prefetch.
• Benchmark environment records git_dirty alongside commit SHA.
• Docs: MEASURED capacity tables for DeepMLP 1.5× VRAM and Qwen3-8B fixed-shape
  logits forward (seq=16); honest Accelerate / CPU / eager baselines.

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md

TensorTorrent 0.2.8

0.2.8

• Planner top-K finalists: same-subset alternatives reach DES; non-streaming ranks
  steady-state schedules so cold-start H2D cannot overturn GPU.
• Two-stage DES winner selection (min-raw + isclose tie-break); fail closed when
  all variants infeasible; deterministic beam select with subset-pool retry.
• Beam parallelism gated on problem config / workers; finalists expose
  analytic_rank / finalist_rank.
• Native-extension gating for planner tests; build/dependency script cleanup.

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md

TensorTorrent 0.2.7

0.2.7

• Native placement planner (tt-planner): Rayon-parallel subset beam search with
  GIL released; analytic finalists ranked by batch DES before compile.
• Shared transfer cost model between planner and DES (estimate_transfer,
  contention / host-staged policy).
• Config defaults: planner_parallel_subsets=True, planner_workers=0 (auto),
  planner_des_candidates for DES shortlist size.
• Python planner search reduced to a thin native wrapper; specialize compiles
  only the DES-selected winner.

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md

TensorTorrent 0.2.6

0.2.6

• Shared capacity accounting: CapacityLedger leases host/device/disk bytes per
  in-flight forward; serve admit and CompiledModule.forward fail closed under
  oversubscription; concurrency clamps to what budgets allow.
• Direct path auto-disabled for streaming, activation spill, and training-capable
  compiles (schedule semantics preserved even if TT_DIRECT_PATH=1).
• Docs and validation copy reflect the supported single-host product surface.
• Prune: remove always-error C backend ABI stubs, dead config knobs
  (activation_overflow_policy, reduced_precision), orphan tools/dev_check.sh,
  and unused package re-export facades.

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md

TensorTorrent 0.2.5

0.2.5

• Beyond-VRAM: Prefetch is pack I/O only in DES; NUMA/pageable staging when a
  region exceeds pinned_host; host-resident weights + coalesced Transfer/Evict
  when state exceeds VRAM; pin packs for DMA; omit zero mock_transfer_delay_s
  on real CUDA (was capacity-spinning). bench/oversized_model.py 1.5× VRAM:
  ~1.1 s/fwd (was ~76 s).

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md

TensorTorrent 0.2.4

0.2.4

• Compile-path timing breakdown (specialize_timing) and make bench-perf.
• Faster specialize plumbing: CPU-first region measure, optional accelerator
  shards, concurrent-first fusion bake-off, incremental planner local search.
• Dataflow direct path eligibility includes XPU; prune unused Python/Rust APIs.

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md

TensorTorrent 0.2.3

0.2.3

• Disable activation buffer reuse under streaming parameter stores (shared slot
  views could overwrite live activations and produce NaN logits).

https://github.com/alhussein-jamil/TensorTorrent/blob/main/CHANGELOG.md