Data Format Lab v0.1.0
The first evidence release of a reproducible research lab for testing data-format and database performance claims under explicit contracts.
Evidence contract
- Frozen dataset: github-stars-2026-07-03, 2,331 rows, 13 typed columns
- Source SHA-256: 39cc70109d9dddf947257584e15f2f9a6bc97dcdf0a7bf939c26cccbcda0e22e
- Canonical hash: 1bf35022cce6d752f7959907b6a60d4024123e045a270f4aa286acbefbe4ca39
- Independent lanes: fair storage, format-specific claims, and prompt tokens
- Separate macOS ARM and Linux x86_64 runs; no cross-hardware ranking
- Two runs per lane and platform
Findings
- Parquet zstd-19 was the smallest fair artifact at 176,713 bytes; default Parquet had the lowest final warm read-all p50 on both platforms.
- Vortex beat the matched Parquet implementation in every tested stress-scan operation; this supports the tested workload, not a universal database claim.
- Lance FTS returned top-20 results around 1 ms on macOS and 1.4 ms on Linux, while Lance data size varied between repeated writes. The FTS index stayed exactly 418,696 bytes.
- TsFile was 313,744 bytes versus 5,777,023 bytes for Parquet on the 1,000,000-row time-series workload, but TsFile writes were much slower.
- Compact TSV used the fewest prompt tokens: 93,377 with o200k_base and 92,756 with cl100k_base. The taxonomy dictionary is included.
No universal winner is declared. See the versioned evidence summary and platform reports:
https://github.com/Anionix/data-format-lab/tree/v0.1.0/reports/v0.1.0
Assets
Each benchmark archive contains manifest.json, results.json, report.md, and the referenced artifacts. SHA256SUMS covers the frozen dataset and all 12 evidence archives. FastLanes, Nimble, and AnyBlox remain documented non-ranking research attempts.