Skip to content

4.2.0

Latest
Latest

Choose a tag to compare

View all tags
@zdenek-jonas zdenek-jonas released this 28 Jul 09:36
· 41 commits to main since this release
4.2.0
15c7151
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Features

  • Storage restart: an embedded storage can be started again after it has been shut down [#701]
  • Data integrity (opt-in, disabled by default)
    • Chunk checksums on data files, verified on startup and on demand [#717] [#728]
    • Store-time reference validation with optional self-healing [#739] [#742]
    • Configurable escalation for zombie object ids found by the garbage collector [#743]
  • Power-loss durability: fsync barriers applied on file rollover and commit [#751]
  • GigaMap
    • Annotation-based index generation for Lucene and JVector, including a compile-time index metamodel [#692] [#695] [#699]
    • Structural index lifecycle: removal of indices and constraints across all index families, redefinition for bitmap indices [#694]
    • reindex() to rebuild all indices from the current entity state, plus Lucene back-fill when an index is registered on a populated GigaMap [#714] [#716] [#718] [#724]
    • Entity-id based update() / apply(), so Lucene-only and JVector-only maps can be reindexed without a bitmap index [#713] [#722]
    • External-directory Lucene index commits are now coupled to store() instead of committing eagerly [#715] [#719]
    • Opt-in null embeddings for the vector index, and BitmapIndices.addWithoutInitialization for lazy registration without back-fill [#748] [#759]

Bugfixes

  • Storage garbage collector: mid-cycle registration race, multi-channel data-loss windows, broken task chain after a failed prepended task, destructive re-execution of partially aborted sweeps, and dangling object ids caused by concurrent data imports [#734] [#736] [#738] [#746] [#755] [#770]
  • Transactions file compaction reworked; the previous implementation could produce a transaction log that failed timestamp validation on the next start and destroyed history on a crash mid-rewrite [#760]
  • Backup and rollback hardening: byte-count validation on all backup-relevant copy and write sites, in-flight copies superseded by a truncation, and store rollback consistency [#744] [#749] [#775]
  • Storage engine: duplicate entity-cache entries on type id change, wrong hash key when rebuilding the type-in-file table, missing bounds check for variable-length members in BinaryReferenceTraverser, and a task broker reference that prevented auto-shutdown [#726] [#740] [#741] [#767]
  • GigaMap index correctness
    • Bitmap indices: Error on update/set/replace, exact-match results of binary composite indices, BinaryIndexerString collisions on keys containing NUL, null-valued Boolean-indexed entities, multi-value indexers dropping pre-existing shared values, notNull() on composite indexers, and remove with short key arrays [#685] [#686] [#687] [#688] [#689] [#690] [#691] [#756] [#776]
    • Use-after-free and double-free in bitmap segment decompression [#765] [#773]
    • Stale indices after class evolution no longer delete a committed entity, and stale-key removal now fails loudly [#762] [#771]
    • Anonymous indexer name resolution after reload, and restart idempotence of generated @Unique constraints [#693] [#702]
  • GigaMap consistency and concurrency
    • Read-lock leaks and deadlocks during iteration; the iteration contract is now enforced [#697] [#700]
    • Silent lost updates: dirty segments are retained in release() and segments mutated by apply() / update() are pinned [#737] [#757]
    • Exceptions thrown from user code (indexers, update logic) no longer leave the map inconsistent, and replace() / apply() validate their current argument [#705] [#758]
    • Lucene reader is no longer reopened on the write path [#777]
    • JVector: abandoned vector indices (thread and graph) are reclaimed, and the graph is no longer fully rebuilt on complete() or on shutdown persist [#754] [#764] [#774]
  • Configuration: correct Redis file system URI prefix, Azure accountName typo (deprecated alias retained), and null guards for optional cloud storage credentials [#732]

Other Changes

  • Documentation on mutex lock behavior, continuous-backup restore consistency and the sweep-retry contract [#732] [#745]
  • Fixes for the example projects [#733]

Migration

Warning

The transaction log compaction was reworked [#760]. A transactions file compacted by 4.2.0 uses only the already existing entry kinds, so older versions can still read it.

While rewriting the log, however, 4.2.0 first puts the complete compacted content into a checksummed swap file next to the transactions file (<transactions file name>_compaction.swp) and heals an interrupted rewrite from it on the next start. Versions prior to 4.2.0 do not know this file. If a crash interrupts a compaction and the storage is then opened with an older version, the rewrite is not healed and the leftover swap file is ignored.

After a crash during compaction, open the storage with 4.2.0 once before downgrading, and do so in writable mode - a read-only start refuses to perform the repair write.

Documentation

https://docs.eclipsestore.io/manual/4.2.0

See also https://github.com/eclipse-serializer/serializer/releases/tag/4.2.0

Assets 2
Loading