Data Format Lab v0.2.0-rc1

Reproducible full dataset assets and equivalence benchmark evidence candidate. v0.1.0 remains unchanged. Benchmark result assets will be appended after the macOS ARM full runs and package checks complete.

Data Format Lab v0.1.0

Data Format Lab v0.1.0

The first evidence release of a reproducible research lab for testing data-format and database performance claims under explicit contracts.

Evidence contract

• Frozen dataset: github-stars-2026-07-03, 2,331 rows, 13 typed columns
• Source SHA-256: 39cc70109d9dddf947257584e15f2f9a6bc97dcdf0a7bf939c26cccbcda0e22e
• Canonical hash: 1bf35022cce6d752f7959907b6a60d4024123e045a270f4aa286acbefbe4ca39
• Independent lanes: fair storage, format-specific claims, and prompt tokens
• Separate macOS ARM and Linux x86_64 runs; no cross-hardware ranking
• Two runs per lane and platform

Findings

• Parquet zstd-19 was the smallest fair artifact at 176,713 bytes; default Parquet had the lowest final warm read-all p50 on both platforms.
• Vortex beat the matched Parquet implementation in every tested stress-scan operation; this supports the tested workload, not a universal database claim.
• Lance FTS returned top-20 results around 1 ms on macOS and 1.4 ms on Linux, while Lance data size varied between repeated writes. The FTS index stayed exactly 418,696 bytes.
• TsFile was 313,744 bytes versus 5,777,023 bytes for Parquet on the 1,000,000-row time-series workload, but TsFile writes were much slower.
• Compact TSV used the fewest prompt tokens: 93,377 with o200k_base and 92,756 with cl100k_base. The taxonomy dictionary is included.

No universal winner is declared. See the versioned evidence summary and platform reports:
https://github.com/Anionix/data-format-lab/tree/v0.1.0/reports/v0.1.0

Assets

Each benchmark archive contains manifest.json, results.json, report.md, and the referenced artifacts. SHA256SUMS covers the frozen dataset and all 12 evidence archives. FastLanes, Nimble, and AnyBlox remain documented non-ranking research attempts.