Skip to content

[spark] Fix min and max query failure on metadata columns - #9431

Merged
JingsongLi merged 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/fix-agg-psdown
Aug 28, 2026
Merged

[spark] Fix min and max query failure on metadata columns#9431
JingsongLi merged 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/fix-agg-psdown

Conversation

@ArnavBalyan

Copy link
Copy Markdown
Member

Purpose

  • Min and max queries directly aggregating Paimon metadata columns fail.
  • This is because agg pushdown is checking the physical schema for metadata columns. However, metadata columns are not in the physical schema, instead derived from split metadata as virtual columns.
  • Skip pushdown for such columns and let Spark aggregate to avoid the failure.

Tests

  • UT

@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit 8688a0e into apache:master Aug 28, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants