Skip to content

branch-4.0: [fix] fix routine_load_jobs first_error_msg on non-master FE and flaky ann_index_basic#65942

Merged
morningman merged 2 commits into
apache:branch-4.0from
morningman:fix-branch40-p0-routineload-ann-index-20260723
Jul 23, 2026
Merged

branch-4.0: [fix] fix routine_load_jobs first_error_msg on non-master FE and flaky ann_index_basic#65942
morningman merged 2 commits into
apache:branch-4.0from
morningman:fix-branch40-p0-routineload-ann-index-20260723

Conversation

@morningman

Copy link
Copy Markdown
Contributor

What this PR does

Two independent fixes for P0 regression failures on branch-4.0 (surfaced in internal CI build 203615).

1. information_schema.routine_load_jobs returns empty error columns on a non-master FE (product fix)

FIRST_ERROR_MSG / ERROR_LOG_URLS of a routine load job are transient, in-memory fields maintained only on the master FE (set in RoutineLoadJob.executeTaskOnTxnStatusChanged on task-txn commit; the follower replay path never sets them).

information_schema.routine_load_jobs is served by the local (coordinating) FE, so on a non-master FE these columns come back empty, while SHOW ROUTINE LOAD (forwarded to the master) shows them. In a multi-FE cluster the two views are inconsistent, which made test_routine_load_first_error_msg fail (~4/5 runs).

Fix: route the routine_load_jobs scan to the master FE, the same way information_schema.tables is already routed, gated by the existing enable_schema_scan_from_master_fe session variable (default true). No BE change, no test change needed.

2. Flaky ann_index_basicsql_ip_asc empty result (test fix)

Queries whose top-n cannot be evaluated by the ANN index (Asc inner_product, Desc l2, small-predicate top-n) fall back to a brute-force scan. Right after an INSERT the freshly written raw rows can be briefly invisible to that scan, so the ordered query intermittently returns nothing.

Fix: add a scan-visibility gate after each INSERT that waits until a real row read (select id, not count(*) which may take a metadata shortcut) returns the expected number of rows. .out and qt_ tags are unchanged.

Verification

  • FE change is minimal and type-checked; not built/e2e-locally — relying on PR CI.
  • Root causes confirmed by code path + CI history (routine_load 4/5 fail, ann_index ~2/20).

🤖 Generated with Claude Code

morningman and others added 2 commits July 23, 2026 11:26
…rom the master FE

FIRST_ERROR_MSG and ERROR_LOG_URLS of a routine load job are transient,
in-memory fields that are only maintained on the master FE (they are set in
RoutineLoadJob.executeTaskOnTxnStatusChanged when a task txn commits; the
follower replay path never sets them).

information_schema.routine_load_jobs is served by the local (coordinating) FE,
so when the query lands on a non-master FE these columns come back empty, while
SHOW ROUTINE LOAD (which forwards to the master) shows them. In a multi-FE
cluster this makes the two views inconsistent and makes
test_routine_load_first_error_msg flaky.

Route the routine_load_jobs scan to the master FE, the same way the
information_schema.tables scan is already routed, so the result is consistent
with SHOW ROUTINE LOAD. Gated by the existing enable_schema_scan_from_master_fe
session variable (default true).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… visibility window

ann_index_basic intermittently failed at the sql_ip_asc check, returning an
empty result. Queries whose top-n cannot be evaluated by the ANN index (Asc
inner_product, Desc l2, small-predicate top-n) fall back to a brute-force scan;
right after an INSERT the freshly written raw rows can be briefly invisible to
that scan, so the ordered query returns nothing.

Add a scan-visibility gate after each INSERT that waits until a real row read
(select id, not count(*) which may take a metadata shortcut) returns the
expected number of rows. This does not touch the .out file or the qt_ tags.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉
Increment coverage report
Complete coverage report

@morningman
morningman merged commit bbc1d2a into apache:branch-4.0 Jul 23, 2026
26 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants