Repository navigation
v0.44.0
This release adds typed pruning APIs and tools for building and testing external-data workers. The vgi-protocol, vgi, and vgi-client crates are released together at 0.44.0; the VGI wire surface is unchanged.
Worker APIs
- Infer conservative Arrow-typed column domains with
PushdownFilters::column_domain(), prune candidate rows from a subset of known columns withmay_match(), and test domain membership withColumnDomain::may_contain(). - Preserve integer precision, unsigned values, decimal and timestamp types, exclusive bounds, SQL NULLs, and dictionary values. Floating-point comparisons now follow DuckDB's signed-zero and NaN semantics.
- Choose exact-filtering ownership with
FunctionMetadata::with_filter_handling(FilterHandling::{Disabled, Sdk, Worker}). SDK filtering runs before projection, including table-in-out finalization and buffering output, and preserves row provenance. - Emit execution-correlated logs and scan counters with
ExecutionDiagnostics, and return typed source errors that distinguish missing objects, denied reads, outages, and malformed data.
Tests and examples
- Enable
vgi-client's optionaltestingfeature forTestWorker, typed filter fixtures, exact schema and row-multiset checks, and pushed-scan comparisons against an independent reference. - Use the new partitioned JSONL external-data worker as a small example of streaming, pruning, filtering, source errors, and diagnostics over native stdio.
- Added guides for filtering and pruning, testing a worker, and observing execution diagnostics.
Other changes since 0.43.0
- Cache decoded window partitions with bounded retention.
- Return canonical RPC error codes for SDK validation failures.
- Clean up all HTTP test servers when the integration harness exits.
- Refresh the quickstart dependency versions to VGI 0.44, vgi-rpc 0.31, and Arrow 60.
Migration
Existing filter_pushdown: true, auto_apply_filters: true declarations remain valid and select SDK filtering. Workers that enable pushdown without an exact-filtering owner must select FilterHandling::Sdk or explicitly guarantee worker-side evaluation with FilterHandling::Worker; otherwise discovery/init fails. Required filters sent to a Disabled function are rejected.
For table-in-out output whose cardinality differs from its input, provide explicit parent_rows when SDK filtering removes rows. This prevents incorrect correlation with input rows.
See filter ownership and pruning, worker testing, and the external-data example.