Skip to content

v0.4.0

Choose a tag to compare

@tvanreenen tvanreenen released this 09 Aug 06:10
· 25 commits to main since this release
ee07ca0

Summary

Version 0.4.0 makes persistence explicit, self-describing, and safer to update. Earlier archives stored vectors and metadata but omitted the settings needed to interpret those vectors correctly. This release records that configuration in every new archive and clearly separates creating, opening, saving, and reloading a store.

Explicit persistence lifecycle

  • Add VectorStore.open(path) to construct a store from a versioned archive and restore its dimensions and normalization mode.
  • Let save(path) perform the first save or a Save As operation and bind that destination. Later save() calls update the bound archive.
  • Add reload() as a deliberate refresh from disk. Failed reads or validation leave current in-memory state unchanged.
  • Keep creating a new in-memory store separate from opening one on disk so file access and persistence boundaries remain visible in application code.

Versioned, self-describing archives

  • Write archive format version 1 with format_version, dimensions, normalize, vectors, and metadata fields.
  • Validate the complete archive before changing live store state, including configuration, array dtypes and shapes, row counts, finite values, and normalized-store zero-vector rules.
  • Reject unsupported versions and malformed archives clearly rather than guessing omitted configuration or partially applying valid fields.
  • Continue preserving each metadata item as one opaque row payload.

Safer archive replacement

  • Write each save to a uniquely named temporary archive in the destination directory, close it, and replace the destination with os.replace.
  • Preserve the previous complete archive when writing or replacement fails and clean up temporary files after failures.
  • Bind a new Save As destination only after its archive is written successfully.

This provides atomic visibility: readers see the previous complete archive or the new complete archive instead of a partially written file. It does not provide file locking, multi-writer coordination, or a universal power-loss durability guarantee.

Migration window

  • Constructor file_path=, instance load(), and direct context-manager persistence remain available in 0.4 with FutureWarning. They will be removed in 0.5.
  • The deprecated context manager now saves only after a successful block. It does not save or suppress the exception when the block raises.
  • Older archives containing only vectors and metadata remain readable through the configuration-aware legacy API. Loading one warns, and its next save rewrites it as format version 1.
  • VectorStore.open() intentionally rejects an unversioned archive because that file cannot report its original dimensions or normalization semantics.

Applications should migrate an old archive once with 0.4 using its original configuration or recreate it from source data. The legacy reader is removed in 0.5 rather than maintained indefinitely. See the persistence migration guide for side-by-side API replacements and conversion instructions.

Runtime compatibility

  • Support Python 3.11 through 3.14. Python 3.10 remains supported by the 0.3 release series but is not supported by 0.4.
  • Raise the minimum NumPy version from 1.21.3 to 1.23.2, the earliest release that supports Python 3.11.
  • Test every supported Python version and the minimum Python/NumPy combination in GitHub Actions.

Upgrade notes

  • Search, insertion, retrieval, clearing, normalization, and metadata behavior are unchanged from 0.3.2.
  • Code using VectorStore.open(), save(path), save(), and reload() is on the persistence API intended for 0.5.
  • Transitional entry points continue to work in 0.4 but warn so the required 0.5 migration is visible during testing.
  • Metadata still uses NumPy's pickle-backed object-array loading. Only open archives produced by your application or another trusted source.
  • Mutable public state, repeated-add performance, deterministic tie ordering, and a formal thread-safety contract remain planned for later releases.

See the repository changelog for the complete release record.

Full comparison: v0.3.2...v0.4.0