v2.1.1

v2.1.1 - ARM64 Mac build fix

v2.0.0 - Multi-Column Storage & GPU Acceleration

What's New

Multi-Column Storage

• Pack heterogeneous tensors (text, image, audio) in one file
• Selective column loading
• Per-column compression settings

Column Manipulation

• Add/update/delete/rename columns without repacking

Zero-Copy Memory Mapping

• 50-90% memory reduction with mmap
• Page-aligned chunks for efficiency

v0.1.0

Release Notes - v0.1.0

Initial public release of Decompressed.

What's New

Section-Based Packing

Added pack_cvc_sections() for combining multiple arrays of different sizes into a single file with metadata. This solves the main limitation where chunk metadata required arrays to align with chunk boundaries.

sections = [
    (wikipedia_embeddings, {"source": "wikipedia"}),
    (arxiv_embeddings, {"source": "arxiv"}),
]
pack_cvc_sections(sections, "combined.cvc", chunk_size=10_000)

# Load just arXiv
vectors = load_cvc_range("combined.cvc", section_key="source", section_value="arxiv")

The loader automatically handles chunks that span section boundaries and extracts only the requested data.

Metadata Filtering

Extended load_cvc_range() to accept section_key and section_value parameters for filtering. You can now load specific sections without writing custom filtering logic.

Core Features

• FP16 and INT8 compression with 2-4× size reduction
• CPU-based decompression (GPU backends via Triton in development)
• Chunked file format for streaming large datasets
• Per-chunk and per-section metadata support

Installation

pip install decompressed

For GPU support (once available):

pip install decompressed[gpu]

Known Issues

• CUDA backend not yet implemented (planned)
• Triton backend available but requires manual installation
• No pre-built wheels yet - package builds from source

Documentation

See README.md for API documentation and examples.

Breaking Changes

None (initial release).