Skip to content

[fix](be) Keep Paimon JNI predicate compatibility#65328

Open
Gabriel39 wants to merge 1 commit into
apache:masterfrom
Gabriel39:codex/fix-paimon-predicate-compat
Open

[fix](be) Keep Paimon JNI predicate compatibility#65328
Gabriel39 wants to merge 1 commit into
apache:masterfrom
Gabriel39:codex/fix-paimon-predicate-compat

Conversation

@Gabriel39

Copy link
Copy Markdown
Contributor

Format V2 Paimon JNI reader now falls back to legacy split-level paimon_predicate when scan-level predicate is missing or empty. Added BE unit coverage for scan priority, legacy fallback, and missing predicate failure. Validation: build-support/clang-format.sh passed; ./run-be-ut.sh --run --filter=PaimonJniReaderTest* was attempted but local CMake is blocked by incomplete thirdparty/installed dependencies after missing Protobuf library.

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Format V2 Paimon JNI reader only accepted scan-level paimon_predicate and rejected scan ranges that still carried the predicate in the legacy split-level field. This made the v2 path stricter than the v1 reader and could break rolling upgrades or older planning paths that still populate the split-level predicate. The fix centralizes predicate selection so scan-level predicate remains preferred, while v2 falls back to the split-level predicate when the scan-level value is missing or empty. The scanner params and validation now share the same selection rule.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - Added BE unit tests for scan-level priority, split-level fallback when scan-level is missing or empty, and clear failure when both locations are missing. Ran build-support/clang-format.sh. Attempted ./run-be-ut.sh --run --filter='PaimonJniReaderTest*', but local BE UT CMake configuration is blocked by incomplete thirdparty/installed dependencies after reaching missing Protobuf library.
- Behavior changed: Yes. Format V2 Paimon JNI reader now accepts legacy split-level paimon_predicate as a fallback for compatibility.
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

@Gabriel39

Copy link
Copy Markdown
Contributor Author

/review

@Gabriel39 Gabriel39 marked this pull request as ready for review July 7, 2026 10:05
@Gabriel39 Gabriel39 requested a review from yiguolei as a code owner July 7, 2026 10:05

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review complete. I did not find a blocking issue worth an inline review comment in this PR.

Critical checkpoint conclusions:

  • Goal and correctness: the change restores format-v2 Paimon JNI predicate compatibility by preferring scan-level paimon_predicate and falling back to the legacy split-level field when the scan-level value is missing or empty. validate_scan_range() and build_scanner_params() now use the same selection rule.
  • Scope: the implementation is small and localized to the v2 Paimon JNI reader plus focused BE unit coverage.
  • Parallel paths: v1 Paimon JNI already prefers scan-level predicate and falls back to the split-level field; the v2 path is now aligned. Current FE planning still sets scan-level predicate/options and serialized table at scan level, and Java PaimonJniScanner consumes the resulting paimon_predicate parameter uniformly.
  • Concurrency/lifecycle: no new concurrency, global state, thread lifecycle, or lock behavior was introduced.
  • Configuration/session propagation: no new config item was added. Existing scan-level serialized_table, storage properties, and Paimon options propagation were checked around PaimonScanNode and FileQueryScanNode.
  • Compatibility: the split-level predicate fallback is a compatibility improvement for older planning paths or rolling-upgrade scenarios that still carry the predicate in TPaimonFileDesc.
  • Error handling: missing predicate still fails during scan-range validation, and the later scanner-param build asserts the same invariant after validation.
  • Tests: the added BE unit tests cover scan-level precedence, missing scan-level fallback, empty scan-level fallback, and the missing-both error. The new test file is included by the BE test glob and BE UT intentionally compiles with -fno-access-control, so the protected test helper access is consistent with local test conventions.
  • Validation limits: I did not run BE UT locally because this GitHub Actions checkout is missing .worktree_initialized, thirdparty/installed, and thirdparty/installed/bin/protoc; review validation was static.

User focus: no additional user-provided review focus was present.

Subagent conclusions:

  • optimizer-rewrite proposed OPT-001 about protected member access in the new test. I dismissed it with evidence because doris_be_test is compiled with -fno-access-control and be/test/AGENTS.md documents that BE-UT ignores access control.
  • tests-session-config found no new candidate and independently noted the same BE UT access-control configuration.
  • Convergence round 1 ended with both optimizer-rewrite and tests-session-config replying NO_NEW_VALUABLE_FINDINGS for the same ledger and empty proposed inline comment set.

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 100.00% (14/14) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.53% (23198/41037)
Line Coverage 40.08% (225148/561725)
Region Coverage 36.03% (177743/493314)
Branch Coverage 37.09% (79021/213045)

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run performance

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run external

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants