Releases
v0.4.8rc2-dev31
Compare
Sorry, something went wrong.
No results found
efschu
released this
23 Jun 14:54
Engine-driven multi-group transfer optimizations
Wire-Format (Tier 1, always on)
Replaces pickle.HIGHEST_PROTOCOL with torch.save on the worker→server wire
b'L' magic byte + payload; legacy pickle blobs auto-detected on first byte
torch.save reaches ~1.6-1.8 GiB/s on production-sized (~50 MiB) bf16 tensors
vs pickle's ~0.9 GiB/s -- ~2x speedup on serialize + deserialize
Cumulative effect on the 14 GiB / 4-group workload: ~1.5-2s shaved per store
Multi-Stream D2D (Tier 1, opt-in)
LMCACHE_MP_MULTI_STREAM_D2D=1 runs each group's GPU→CPU gather on its own
CUDA stream
Default is OFF because on consumer GPUs the PCIe bus is shared and
per-stream sync overhead exceeds the overlap win (4x slower in our test)
Available for A/B testing on different hardware
Cleanup
Removed duplicate _serialize_single_group_chunks (was defined twice)
Removed dead _to_serializable_chunk and tagged-tuple wire format
Removed duplicate _pinned_pool initialization
Compatibility
Backward-compatible wire format : dev<=30 pickle blobs still decode
No L2 cache wipe required when upgrading
Same COMMIT_STORE_GROUP request shape; the new format is auto-detected
You can’t perform that action at this time.