Skip to content

[fix](scan) Fix adaptive load batch sizing#63243

Closed
mrhhsg wants to merge 1 commit into
apache:branch-4.1from
mrhhsg:codex/fix-load-adaptive-batch-size-4.1
Closed

[fix](scan) Fix adaptive load batch sizing#63243
mrhhsg wants to merge 1 commit into
apache:branch-4.1from
mrhhsg:codex/fix-load-adaptive-batch-size-4.1

Conversation

@mrhhsg
Copy link
Copy Markdown
Member

@mrhhsg mrhhsg commented May 14, 2026

Summary

  • update file scan load path to feed converted blocks into adaptive batch size prediction before and after varchar truncation
  • add debug hooks and regression coverage to make the compact-with-sequence rowset layout reproducible
  • keep local regression configuration aligned with the active test cluster

Root Cause

The load path initialized adaptive batch sizing but did not update the predictor after converting loaded source blocks. As a result, the reader could keep using the initial small probe size, creating many small segments and exposing the compaction/delete-bitmap timing issue in unique_with_mow_c_p0/test_compact_with_seq2.

Validation

  • git diff --check
  • git diff --cached --check
  • ./run-regression-test.sh --run -d unique_with_mow_c_p0 -s test_compact_with_seq2

@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@mrhhsg mrhhsg changed the title [codex] Fix adaptive load batch sizing [fix](scan) Fix adaptive load batch sizing May 14, 2026
@mrhhsg mrhhsg closed this May 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants