v0.2.6

mmap-chunker-core v0.2.6

Prebuilt native libraries for C, Python, Go, and other FFI consumers,
plus standalone mmap-chunker CLI archives.

Native archives contain the C header, dynamic library, static library, and licenses.
CLI archives contain the versioned mmap-chunker executable and licenses.
Every archive has an accompanying .sha256 checksum file.

Python wheels and the sdist for this version are published on PyPI
(project mmap-chunker-core), not duplicated here.

What's Changed

• Fix DataTrove streaming parity and prove real single-file parallelism by @trungminhdo4-glitch in #27
• release: prepare v0.2.6 by @trungminhdo4-glitch in #28

Full Changelog: v0.2.5...v0.2.6

v0.2.5

mmap-chunker-core v0.2.5

Prebuilt native libraries for C, Python, Go, and other FFI consumers,
plus standalone mmap-chunker CLI archives.

Native archives contain the C header, dynamic library, static library, and licenses.
CLI archives contain the versioned mmap-chunker executable and licenses.
Every archive has an accompanying .sha256 checksum file.

Python wheels and the sdist for this version are published on PyPI
(project mmap-chunker-core), not duplicated here.

What's Changed

• test: add real multi-file worker proof by @trungminhdo4-glitch in #22
• feat: DataTrove single-file adoption proof by @trungminhdo4-glitch in #24
• feat: Python wheel distribution (mmap-chunker-core) by @trungminhdo4-glitch in #25
• release: prepare v0.2.5 (Python distribution + OIDC Trusted Publishing) by @trungminhdo4-glitch in #26

Full Changelog: v0.2.4...v0.2.5

v0.2.4

mmap-chunker-core v0.2.4

Prebuilt native libraries for C, Python, Go, and other FFI consumers,
plus standalone mmap-chunker CLI archives.

Native archives contain the C header, dynamic library, static library, and licenses.
CLI archives contain the versioned mmap-chunker executable and licenses.
Every archive has an accompanying .sha256 checksum file.

What's Changed

• feat(cli): add ordered multi-file record partitioning by @trungminhdo4-glitch in #21
• release: prepare v0.2.4 by @trungminhdo4-glitch in #23

Full Changelog: v0.2.3...v0.2.4

v0.2.3

mmap-chunker-core v0.2.3

Prebuilt native libraries for C, Python, Go, and other FFI consumers,
plus standalone mmap-chunker CLI archives.

Native archives contain the C header, dynamic library, static library, and licenses.
CLI archives contain the versioned mmap-chunker executable and licenses.
Every archive has an accompanying .sha256 checksum file.

What's Changed

• feat: add CLI worker range selection by @trungminhdo4-glitch in #17
• feat: add configurable CLI partition delimiter byte by @trungminhdo4-glitch in #18
• ci: distribute standalone mmap-chunker CLI archives by @trungminhdo4-glitch in #19
• release: prepare v0.2.3 by @trungminhdo4-glitch in #20

Full Changelog: v0.2.2...v0.2.3

v0.2.2

mmap-chunker-core v0.2.2

Prebuilt native libraries for C, Python, Go, and other FFI consumers.

Each archive contains the C header, dynamic library, static library, and licenses.
Every archive has an accompanying .sha256 checksum file.

What's Changed

• ci: harden post-release compatibility gates by @trungminhdo4-glitch in #1
• test: add independent scanner differential oracles by @trungminhdo4-glitch in #2
• ci: migrate actions to Node 24 runtimes by @trungminhdo4-glitch in #3
• test: establish trustworthy benchmark foundation by @trungminhdo4-glitch in #4
• refactor: unify internal chunk planning state by @trungminhdo4-glitch in #5
• test: add bounded differential fuzzing by @trungminhdo4-glitch in #6
• ci: enforce ABI and package contracts by @trungminhdo4-glitch in #7
• test: add Python C ABI parity oracle by @trungminhdo4-glitch in #8
• ci: add focused ASan FFI lane by @trungminhdo4-glitch in #9
• docs: prove record-aligned worker partitioning by @trungminhdo4-glitch in #10
• fix: bound extreme record partition requests by @trungminhdo4-glitch in #11
• Add Linux cross-language ABI conformance gate by @trungminhdo4-glitch in #12
• Harden Linux x86_64 release GLIBC floor by @trungminhdo4-glitch in #13
• test: add decision-grade performance baseline by @trungminhdo4-glitch in #14
• feat: add partition CLI by @trungminhdo4-glitch in #15
• release: prepare v0.2.2 by @trungminhdo4-glitch in #16

New Contributors

• @trungminhdo4-glitch made their first contribution in #1

Full Changelog: v0.2.1...v0.2.2

v0.2.1

mmap-chunker-core v0.2.1

Prebuilt native libraries for C, Python, Go, and other FFI consumers.

Each archive contains the C header, dynamic library, static library, and licenses.
Every archive has an accompanying .sha256 checksum file.

Full Changelog: v0.2.0...v0.2.1

mmap-chunker-core v0.2.0

Highlights

• Safe dependency-free SWAR delimiter search
• Arithmetic fixed-size chunking with O(1) boundary metadata
• Record-aligned N-way partition planning for parallel consumers
• C ABI evolved additively to v1.2
• Expanded Linux, Windows, macOS and Rust 1.77 validation

Fixed-size chunking

• mmap_engine_scan_fixed(handle, chunk_size) — C ABI function
• Exact fixed byte ranges, no delimiter scan required
• Arithmetic chunk boundaries via scanner::fixed_chunk_count and scanner::fixed_chunk_bounds
• O(1) layout metadata — boundary positions computed arithmetically, no scan
• Zero-copy CChunkView views preserved
• Last chunk may be shorter; chunk size 0 clamps to 1

Record partition planning

• mmap_engine_partition_records(handle, num_partitions, delimiter) — C ABI function
• Approximately balanced N-way ranges computed from absolute file-position targets
• Record-aligned boundaries: no record is split across partitions
• Absolute-target algorithm prevents cumulative size drift across partitions
• Giant records may reduce actual partition count (boundaries collapse, no empty partitions)
• All partition boundaries computed independently from floor(file_len * i / N)
• Metadata cost: O(N) scan, bounded to file length
• Library plans ranges; it does not create worker threads/processes

Performance

• Internal single-byte delimiter search now uses safe 64-bit SWAR
• Corrected benchmarks showed substantial improvements in byte search and in-memory boundary construction on tested Windows x86_64 workloads

C ABI v1.2

Additive API additions — existing v1.0 functions remain available:

mmap_engine_scan_fixed          — fixed-size arithmetic chunking
mmap_engine_partition_records   — record-aligned N-way partition planning

New capability bits:

CAP_FIXED_SIZE_CHUNKING   (bit 3)
CAP_RECORD_PARTITIONING   (bit 4)

mmap_engine_abi_version() returns 0x00010002.

Validation

• 110 Rust tests (108 unit + 2 integration)
• 30 external C ABI assertions
• Ubuntu, Windows, macOS CI
• Rust 1.77 MSRV
• ABI symbol regression gate
• Zero Rust runtime dependencies

mmap-chunker-core v0.1.0

mmap-chunker-core v0.1.0

Initial public release of a zero-dependency Rust library for memory-mapped file chunking with a stable C ABI.

Highlights

• Cross-platform native mmap: Windows (\CreateFileMappingW) and POSIX (\mmap/munmap) implementations
• Zero-copy chunk views: \CChunkView\ references mapped memory directly — no data copying
• Configurable delimiter scanning: single-byte delimiters (newline, comma, tab, pipe, NUL, etc.)
• Stable C ABI v1: 8 public functions with additive-only compatibility policy
• ABI version discovery: runtime \mmap_engine_abi_version()\ returns (major << 16) | minor\
• Capability detection: \mmap_engine_capabilities()\ returns feature bitmask (zero-copy, configurable delimiter, error strings)
• Thread-local error diagnostics: \mmap_engine_last_error()\ provides human-readable error strings per thread
• Panic containment: all FFI boundaries use \catch_unwind; \mmap_engine_free()\ aborts on unrecoverable panic
• Static + dynamic library: \staticlib\ and \cdylib\ crate types

Validated platforms

Platform	Rust tests	C ABI consumer	Status
Windows x86_64 GNU	47/47	15/15 (local)	Passed
Linux x86_64 (Ubuntu)	47/47	15/15 (CI)	Passed
macOS	—	—	Targeted, unvalidated

Test coverage

• 47 Rust tests (45 unit + 2 integration) including property tests for concatenation, gap-freedom, determinism, monotonic offsets, and delimiter variants
• 15 external C ABI assertions via \�xamples/c_consumer.c\ (compiled with gcc 15.2.0 locally + cc in Ubuntu CI)
• 53 Python ctypes integration tests (local, companion module)

Dependencies

Zero runtime dependencies. Pure Rust with direct OS syscall FFI.

Files

• \src/lib.rs, \src/mmap.rs, \src/scanner.rs, \src/ffi.rs\ — core library
• \mmap_chunker.h\ — public C header with full API docs, threading contract, and ABI stability notes
• \�xamples/c_consumer.c\ — real external C ABI consumer (15 assertions)
• \ ests/c_abi_test.rs\ — Rust-side C ABI integration tests
• \ ests/benchmark.rs\ — mmap vs \s::read\ performance comparison

Known limitations

• Full-file mmap only (no windowed mapping)
• Single-byte delimiter only
• Read-only access
• Eager chunk computation (no lazy/streaming iteration)