ParqDB 0.2.0rc3

0.2.0rc2 was rejected by installed-wheel verification before PyPI publication. 0.2.0rc3 contains that feature set plus the release-gate and dependency fixes below.

[0.2.0rc3] - 2026-08-21

Fixed

• The vendored Arrow Parquet runtime now uses Apache Thrift 0.23, removing the
  vulnerable unbounded compact-protocol varint decoder from release wheels.
• Installed-wheel tests now resolve repository fixtures independently of the
  process working directory, preserving source-tree isolation during release
  verification.
• Pull requests now build and exercise an installed wheel on Python 3.11,
  including the embedded DataFusion namespace-coexistence check, before a
  release tag can be created.

[0.2.0rc2] - 2026-08-21

Added

• LVQ4/LVQ8 builds now emit a strict, immutable top-level manifest.json and
  package-local centroid objects, so one snapshot directory can be copied
  directly to public HTTP storage without repacking.
• The new @parqdb/browser client performs global leaf-centroid routing,
  explicit postings row-group selection through HTTP Range requests, and
  LVQ distance/top-k execution in WebAssembly, returning source keys plus
  _distance without listing objects or joining the source table.
• A new parqdb publish command builds or reuses a static IVF-LVQ4/LVQ8
  index, publishes its source table and model assets to an immutable local or
  S3-compatible prefix, and verifies public HTTP Range and CORS behavior.

Changed

• IVF training now defaults to a persisted two-level hierarchy. Postings are
  grouped into root-aligned cid_bucket files, discovered through an immutable
  manifest, and pruned with explicit Parquet row-group access plans. Root
  training samples 512 points per root, re-seeds empty roots from populated
  donor partitions, then assigns the exact leaf budget proportionally to the
  sampled root populations. It falls back after bounded empty-root recovery to
  flat leaf training followed by a synthetic valid hierarchy.
• Python index discovery and removal now stay behind the table-centered API;
  internal catalog and native repository bridges are no longer exposed through
  session facade attributes.
• Browser queries coalesce and parallelize HTTP Range reads while keeping
  metadata prefetch and the in-browser byte cache bounded.
• The optional publication stack uses the smaller tokenizers runtime instead
  of the full Python Transformers package while preserving pinned MiniLM token
  and embedding parity.

Fixed

• Publication build work is now bound to the exact source and configuration,
  so a changed input cannot silently reuse a stale index. Interrupted embedding
  output is committed atomically and cannot be mistaken for a complete source.
• Static publication cleans partial local and S3 writes on failure and reports
  post-commit public HTTP verification failures without suggesting an unsafe
  retry against the same immutable destination.
• Dependency auditing now covers every locked optional extra in CI and release
  jobs; the vulnerable h2 and Transformers dependency versions are removed.

Removed

• The pre-1.0 Session.to_arrow() compatibility alias; use collect() for a
  materialized Arrow table or stream() for incremental batches.

ParqDB 0.2.0rc1

[0.2.0rc1] - 2026-08-18

Added

• Reusable IVF centroid artifacts used by source, LVQ4, and LVQ8 logical
  indexes.
• Cosine distance and list<double> source-vector support through canonical
  float conversion.
• Portable source, LVQ4, and LVQ8 conformance fixtures under IVF schema
  version 1.
• A reproducible GIST benchmark runner for ParqDB and Faiss Flat, SQ4, and SQ8
  comparisons.
• Native, process-scoped index build coordination with observable status,
  bounded parallelism, failure retention, and retry.
• Bounded incremental Arrow IPC encoding and decoding with transport-level
  backpressure and no full-result buffering.
• An HTTP transport and Python ASGI server using the same session facades,
  source and index lifecycle, error hierarchy, and managed query streams as
  embedded execution. Server-side source registration uses an explicit URI
  prefix allowlist.
• A parqdb serve deployment command and TOML configuration, with a safe
  loopback-only default template generated by parqdb config init.
• Portable, source-free index packages that can be registered against an exact
  source through the embedded or HTTP table API.

Changed

• The public Python API now uses portable session and table facades instead of
  inheriting DataFusion objects. DataFusion remains available through an
  explicit embedded-only escape hatch.
• ParqDB now has one supported DataFusion execution path. The public backend,
  builder, Spark, and StarRocks extension surfaces have been removed while the
  open index specification remains available to other engines.
• IVF postings no longer copy full source vectors. The public IVF
  configuration selects source, lvq4, or lvq8 encoding and
  l2_squared or cosine distance.
• Portable SQL terminals now reject DDL, DML, COPY, and session-mutating
  statements before execution.
• Index names are scoped by their owning table identifier. Build status and
  transport errors expose stable failure codes for embedded/HTTP parity.
• Server implementation is organized into server, transport, and runtime
  packages. The public parqdb.server factory remains unchanged.
• Embedded sessions and server deployment now use a required local root plus
  an optional warehouse for index relations. SQLite catalog placement is no
  longer a public configuration parameter.
• A session uses one warehouse for all index metadata and relations. Published
  locations are warehouse-relative, while each catalog owns its source
  bindings independently.

Fixed

• LVQ distance evaluation now honors non-zero Arrow array offsets when cached
  postings batches are sliced.

Removed

• The versioned backend and builder SDKs, capability registry, experimental
  Spark and StarRocks modules, and their public configuration objects.
• catalog=, index_root=, Session.indexes, and the top-level
  open_index_catalog API. Index lifecycle is table-centered.

Relify 0.1.0rc2

Fixed

• Embedded DataFusion DataFrame text and HTML representations now resolve the bundled formatter from the Relify namespace in clean wheel installations.

See PR #6.

Relify 0.1.0rc1

Added

• A portable specification for index catalogs, immutable metadata snapshots,
  Parquet and Iceberg relation references, IVF-Flat schemas, and vector-query
  semantics.
• Shared conformance fixtures for valid and invalid metadata, index tables, and
  ordered IVF results, including independent execution through DuckDB.
• An embedded DataFusion session with persistent Parquet table registration,
  SQLite-backed table and index catalogs, and support for files, directories,
  and nested wildcard patterns over local filesystems, S3, and HDFS.
• Parallel Rust IVF-Flat construction with bounded-memory deterministic
  sampling and k-means training, SIMD/GEMM distance kernels, composite source
  keys, optional vector storage in postings, and configurable Parquet output.
• A table-centered Python query API with automatic or explicit index
  selection, SQL-string filters, projection, nprobes, large result limits,
  exact-search fallback, and transparent source-table resolution.
• Portable Arrow collection plus lazy DataFusion DataFrame, executable SQL,
  EXPLAIN, and runtime analysis terminals for local vector queries.
• Asynchronous index creation and refresh, source-scoped status and waiting,
  immutable snapshot publication, resident index caching, and
  reachability-based orphan removal.
• A Spark Classic backend that builds IVF indexes in Iceberg, queries Parquet
  and Iceberg index tables through native PySpark DataFrame plans, and shares
  published indexes with the local backend.
• A query-only StarRocks backend that executes complete IVF plans over Arrow
  Flight SQL while binding every Iceberg source and index table to its exact
  snapshot.
• A versioned third-party backend SDK with lazy entry-point discovery, typed
  capability reports, shared query resolution and canonical schema validation,
  and reusable backend contract tests.
• An independent builder SDK and shared table lifecycle that allow Local,
  Spark, StarRocks, and third-party query sessions to select compatible
  construction engines without a backend/build capability matrix.
• Bundled example datasets, runnable Local, Spark, and StarRocks examples, and
  a capability-driven integration-test framework for optional environments.
• Reproducible persisted-build and large-k recall-latency benchmarks with
  Faiss comparison.
• Verified Maturin wheels with locked Python and Rust dependencies, CycloneDX
  SBOMs, license and vulnerability audits, and isolated build-and-search smoke
  tests.

Security

• Canonical URI validation, warehouse confinement, immutable metadata writes,
  catalog compare-and-swap publication, cross-process build coordination, and
  conservative garbage-collection retention protect index state from partial
  publication and premature deletion.