Skip to content

[opt](paimon) support prune complex type in paimon jni reader - #66573

Merged
yiguolei merged 1 commit into
apache:branch-4.1from
zhangstar333:paimon-prune-complex
Aug 7, 2026
Merged

[opt](paimon) support prune complex type in paimon jni reader#66573
yiguolei merged 1 commit into
apache:branch-4.1from
zhangstar333:paimon-prune-complex

Conversation

@zhangstar333

@zhangstar333 zhangstar333 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Problem Summary:

before disable prune complex struct/array/map type in paimon. now support those in paimon jni reader, so could enable it.

|   0:VPAIMON_SCAN_NODE(116)                                                                                                                                                            |
|      table: test_paimon_variant.test_paimon_spark.jni_complex_column_pruning                                                                                                          |
|      predicates: (element_at(profile[#1], 'zip') >= 200000), (element_at(element_at(events[#2], 1), 'score') >= 90), (element_at(element_at(attributes[#3], 'primary'), 'code') = 20) |
|      inputSplitNum=1, totalFileSize=0, scanRanges=1                                                                                                                                   |
|      partition=1/0                                                                                                                                                                    |
|      cardinality=2, numNodes=1                                                                                                                                                        |
|      nested columns:                                                                                                                                                                  |
|        profile:                                                                                                                                                                       |
|          origin type: struct<city:text,zip:int,street:text>                                                                                                                           |
|          pruned type: struct<city:text,zip:int>                                                                                                                                       |
|          all access paths: [profile.city, profile.zip]                                                                                                                                |
|          predicate access paths: [profile.zip]

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@zhangstar333
zhangstar333 requested a review from yiguolei as a code owner August 7, 2026 07:00
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhangstar333

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 50.00% (1/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

1 similar comment
@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@yiguolei
yiguolei merged commit 688e3ce into apache:branch-4.1 Aug 7, 2026
30 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants