Skip to content

v0.7.3

Choose a tag to compare

@justapithecus justapithecus released this 13 Feb 00:23
· 104 commits to main since this release
fe55cc4

O(1) snapshot resolution and complexity sweep

Summary

Eliminates all known O(N) hotspots across Dataset and Volume read/write paths. Cold-start Latest() on remote stores drops from ~30 minutes (5,800 manifests on R2) to ~2 seconds. No API changes; existing data is compatible without migration.

Highlights

  • Persistent latest pointer: Datasets and Volumes write a latest file for O(1) cold-start resolution (1 Get instead of N manifest downloads)
  • Pointer-before-manifest protocol: Pointer write failure aborts the commit, eliminating stale pointer bugs across process restarts
  • O(1) Snapshot(ctx, id) on HiveLayout: Canonical manifest at the non-partitioned path replaces O(N) partition scan
  • O(log B) Volume block lookup: Binary search on sorted blocks replaces linear scan in ReadAt
  • O(N + K log K) block merge: Merge-insert algorithm avoids re-sorting the full cumulative block set on each commit
  • Optimized scan fallback: latestByScan loads only the last manifest (1 List + 1 Get), self-heals by writing the pointer

Upgrade Notes

  • No API changes — all improvements are internal and transparent
  • No migration required — pre-v0.7.3 datasets and volumes work without modification
  • Automatic self-healing — on first write after upgrade, the latest pointer and canonical manifest (HiveLayout) are created automatically
  • Safe to upgrade from v0.7.2

References

Full Changelog: v0.7.2...v0.7.3

What's Changed

Full Changelog: v0.7.2...v0.7.3