Repository navigation
v0.4.8rc2-dev23
Fix for msgspec 4 GiB bin limit: per-group COMMIT_STORE_GROUP messages.
Root cause: combined multi-group cpu_data blob exceeded msgspec msgpack bin limit (4 GiB) for large hybrid models like Qwen3-27B with GDN layers.
Fix: New RequestType.COMMIT_STORE_GROUP ([key, instance_id, group_idx, cpu_data]) — worker sends one message per group instead of one combined blob. Each per-group blob fits within the msgspec limit.
Also includes PR LMCache#3813 review fixes:
- fp8 dtype (e4m3fn, e5m2) support in _serialize_multi_group_chunks via view(torch.uint8)
- assert replaced with explicit ValueError check in worker_transfer.py
- group count validation in _commit_store_multi_group before indexing
- 16 unit tests covering serialization, roundtrip, and dtype preservation