Skip to content

v0.44.0

Choose a tag to compare

@github-actions github-actions released this 14 Oct 12:56
· 54 commits to main since this release

This release adds typed pruning APIs and tools for building and testing external-data workers. The vgi-protocol, vgi, and vgi-client crates are released together at 0.44.0; the VGI wire surface is unchanged.

Worker APIs

  • Infer conservative Arrow-typed column domains with PushdownFilters::column_domain(), prune candidate rows from a subset of known columns with may_match(), and test domain membership with ColumnDomain::may_contain().
  • Preserve integer precision, unsigned values, decimal and timestamp types, exclusive bounds, SQL NULLs, and dictionary values. Floating-point comparisons now follow DuckDB's signed-zero and NaN semantics.
  • Choose exact-filtering ownership with FunctionMetadata::with_filter_handling(FilterHandling::{Disabled, Sdk, Worker}). SDK filtering runs before projection, including table-in-out finalization and buffering output, and preserves row provenance.
  • Emit execution-correlated logs and scan counters with ExecutionDiagnostics, and return typed source errors that distinguish missing objects, denied reads, outages, and malformed data.

Tests and examples

  • Enable vgi-client's optional testing feature for TestWorker, typed filter fixtures, exact schema and row-multiset checks, and pushed-scan comparisons against an independent reference.
  • Use the new partitioned JSONL external-data worker as a small example of streaming, pruning, filtering, source errors, and diagnostics over native stdio.
  • Added guides for filtering and pruning, testing a worker, and observing execution diagnostics.

Other changes since 0.43.0

  • Cache decoded window partitions with bounded retention.
  • Return canonical RPC error codes for SDK validation failures.
  • Clean up all HTTP test servers when the integration harness exits.
  • Refresh the quickstart dependency versions to VGI 0.44, vgi-rpc 0.31, and Arrow 60.

Migration

Existing filter_pushdown: true, auto_apply_filters: true declarations remain valid and select SDK filtering. Workers that enable pushdown without an exact-filtering owner must select FilterHandling::Sdk or explicitly guarantee worker-side evaluation with FilterHandling::Worker; otherwise discovery/init fails. Required filters sent to a Disabled function are rejected.

For table-in-out output whose cardinality differs from its input, provide explicit parent_rows when SDK filtering removes rows. This prevents incorrect correlation with input rows.

See filter ownership and pruning, worker testing, and the external-data example.