Skip to content

ParqDB 0.2.0rc3

Pre-release
Pre-release

Choose a tag to compare

@petrizhang petrizhang released this 21 Aug 08:36
· 8 commits to main since this release
b38b0f6

0.2.0rc2 was rejected by installed-wheel verification before PyPI publication. 0.2.0rc3 contains that feature set plus the release-gate and dependency fixes below.

[0.2.0rc3] - 2026-08-21

Fixed

  • The vendored Arrow Parquet runtime now uses Apache Thrift 0.23, removing the
    vulnerable unbounded compact-protocol varint decoder from release wheels.
  • Installed-wheel tests now resolve repository fixtures independently of the
    process working directory, preserving source-tree isolation during release
    verification.
  • Pull requests now build and exercise an installed wheel on Python 3.11,
    including the embedded DataFusion namespace-coexistence check, before a
    release tag can be created.

[0.2.0rc2] - 2026-08-21

Added

  • LVQ4/LVQ8 builds now emit a strict, immutable top-level manifest.json and
    package-local centroid objects, so one snapshot directory can be copied
    directly to public HTTP storage without repacking.
  • The new @parqdb/browser client performs global leaf-centroid routing,
    explicit postings row-group selection through HTTP Range requests, and
    LVQ distance/top-k execution in WebAssembly, returning source keys plus
    _distance without listing objects or joining the source table.
  • A new parqdb publish command builds or reuses a static IVF-LVQ4/LVQ8
    index, publishes its source table and model assets to an immutable local or
    S3-compatible prefix, and verifies public HTTP Range and CORS behavior.

Changed

  • IVF training now defaults to a persisted two-level hierarchy. Postings are
    grouped into root-aligned cid_bucket files, discovered through an immutable
    manifest, and pruned with explicit Parquet row-group access plans. Root
    training samples 512 points per root, re-seeds empty roots from populated
    donor partitions, then assigns the exact leaf budget proportionally to the
    sampled root populations. It falls back after bounded empty-root recovery to
    flat leaf training followed by a synthetic valid hierarchy.
  • Python index discovery and removal now stay behind the table-centered API;
    internal catalog and native repository bridges are no longer exposed through
    session facade attributes.
  • Browser queries coalesce and parallelize HTTP Range reads while keeping
    metadata prefetch and the in-browser byte cache bounded.
  • The optional publication stack uses the smaller tokenizers runtime instead
    of the full Python Transformers package while preserving pinned MiniLM token
    and embedding parity.

Fixed

  • Publication build work is now bound to the exact source and configuration,
    so a changed input cannot silently reuse a stale index. Interrupted embedding
    output is committed atomically and cannot be mistaken for a complete source.
  • Static publication cleans partial local and S3 writes on failure and reports
    post-commit public HTTP verification failures without suggesting an unsafe
    retry against the same immutable destination.
  • Dependency auditing now covers every locked optional extra in CI and release
    jobs; the vulnerable h2 and Transformers dependency versions are removed.

Removed

  • The pre-1.0 Session.to_arrow() compatibility alias; use collect() for a
    materialized Arrow table or stream() for incremental batches.