Skip to content

v0.3.9 limited python support for predicate pushdown

Choose a tag to compare

@changhiskhan changhiskhan released this 25 Feb 07:31
· 4891 commits to main since this release

By default pyarrow compute Expressions doesn't serialize to sql strings. This patch release enables a limited set of filter pushdowns via python. Supported syntax:

  1. field references
  2. Operators: > < >= <= = == !=
  3. conjunctions / disjunctions

This enables querying via duckdb without needing to load the whole dataset into memory first.

e.g., duckdb.query("SELECT * FROM dataset WHERE id=5")

What's Changed

  • [Rust] Handle double equals in filter by @eddyxu in #639

Full Changelog: v0.3.8...v0.3.9