Skip to content

[python][daft] Fix predicate conversion for unsupported expressions#7980

Merged
JingsongLi merged 1 commit into
apache:masterfrom
QuakeWang:fix-daft-predicate-unsupported-literals
May 27, 2026
Merged

[python][daft] Fix predicate conversion for unsupported expressions#7980
JingsongLi merged 1 commit into
apache:masterfrom
QuakeWang:fix-daft-predicate-unsupported-literals

Conversation

@QuakeWang
Copy link
Copy Markdown
Contributor

Purpose

Daft predicate conversion used None to represent both unsupported expressions and literal values. As a result, comparison pushdown could treat unsupported expressions as literals, for example converting a cast expression into id = None.

String predicates also called str(...) on visited arguments, so column or unsupported arguments could be pushed as literal strings. This could mark filters as pushed and remove them from Daft remaining filters, changing query semantics.

This patch adds explicit internal markers for literals and unsupported expressions. Predicates are only pushed when the non-column side is a real scalar literal, and string predicates now require real string literals. Unsupported or mixed expressions stay in Daft remaining filters.

Tests

  • python -m pytest paimon-python/pypaimon/tests/daft/daft_data_test.py -k 'FilterPushdown'
  • python -m pytest paimon-python/pypaimon/tests/daft/daft_data_test.py

@JingsongLi JingsongLi closed this May 27, 2026
@JingsongLi JingsongLi reopened this May 27, 2026
Copy link
Copy Markdown
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This is a well-designed safety fix. The core insight is correct: using None as a sentinel for both "unsupported expression" and "literal value" was a type confusion that could silently push incorrect predicates and remove them from Daft's remaining filter list — changing query semantics.

The _Literal / _Unsupported marker approach is clean and makes the type distinction explicit throughout the visitor. The _is_pushable_scalar guard also correctly rejects None literals and collection types from pushdown.

Test coverage for the mixed-expression scenarios is good.

+1

@JingsongLi JingsongLi merged commit ad151c0 into apache:master May 27, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants