Skip to content

v0.4.0 - Production-Ready Index Engine

Choose a tag to compare

@anidotnet anidotnet released this 06 Jun 17:07
· 61 commits to main since this release

Summary

This release makes the index engine production-ready for high-volume, ordered workloads such as an email client's initial sync. It eliminates an O(n²) index build, fixes long-standing key-ordering bugs that made integer/float range and descending-sorted index queries return wrong results on the persistent store, and corrects Value's numeric ordering.

⚠️ Breaking Changes

  • On-disk format (indexes + keys). Two storage-format changes mean databases created with 0.3.x must be rebuilt (indexes are derived data; re-create the database or drop and re-create indexes):
    • Non-unique simple and compound indexes now use a flat composite-key layout
    • nitrite-fjall-adapter now serializes keys with an order-preserving codec instead of bincode

Key Improvements

  • O(n²) non-unique index build fixed - inserts and removals are now O(1) point operations
  • Integer/float range queries now correct - byte-order issues eliminated with order-preserving codec
  • Value numeric ordering fixed - negative and very large integers now sort correctly
  • Descending sort correctness - order_by now always applies explicit field sort

Performance

Per-message insert into a collection with indexes (release build, Durability::Periodic):

  • 2,000 messages: 0.11s (~17,800 msg/s) — was 2.26s (885 msg/s)
  • 10,000 messages: 0.24s (~41,600 msg/s) — was 38.0s (263 msg/s)
  • 50,000 messages: 1.12s (~44,600 msg/s) — was ~16 min (extrapolated)

See CHANGELOG.md for full details.