v0.2.0 — Multi-chunk elastic growth

What's new

SharedStore now grows automatically. Instead of allocating a single fixed-size shared memory block, the store splits into a control block (header + index) and separate data chunks that are created on demand when the current chunk fills up.

API changes

# Before
store = SharedStore.create("pipeline", size_mb=256, max_entries=1024)
store = SharedStore.connect("pipeline", locks, max_entries=1024)

# After
store = SharedStore.create("pipeline", chunk_size_mb=64, max_entries=1024)
store = SharedStore.connect("pipeline", locks)  # reads config from header

Details

• Elastic growth: put() tries all existing chunks. If none have space, it creates a new shared memory segment ({name}_{i}) and retries.
• Lazy chunk discovery: Connected processes open chunks on demand when get() encounters a virtual offset pointing to a chunk not yet opened locally.
• Virtual offsets: Index entries store chunk_index * chunk_data_size + local_offset. Encoding/decoding is transparent to the index layer.
• Per-chunk allocators: Each chunk has its own ChunkHeader (free_list_head, data_size) and BlockAllocator. No free-list links span chunks.
• Format version 2: New StoreHeader layout (chunk_data_size, chunk_count fields) and ChunkHeader (32 bytes).

Performance

No regression on the core hot path (single-chunk put/get). Auto-growth adds ~1ms per new chunk creation (OS syscall overhead). Zero-copy get() latency is unaffected regardless of which chunk the data lives in (~4 us).

Tests

64 tests passing, including:

• Auto-growth (fill one chunk, verify second chunk created)
• Multi-chunk get/delete across chunks
• connect() reading config from header
• Concurrent chunk growth from multiple processes