Skip to content

[fix](be) Release deletion vector rows on read errors#65351

Open
Gabriel39 wants to merge 1 commit into
apache:masterfrom
Gabriel39:codex/iceberg-dv-fault-injection-tests
Open

[fix](be) Release deletion vector rows on read errors#65351
Gabriel39 wants to merge 1 commit into
apache:masterfrom
Gabriel39:codex/iceberg-dv-fault-injection-tests

Conversation

@Gabriel39

@Gabriel39 Gabriel39 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Deletion vector and position-delete cache builders allocated rows/maps with raw new and returned nullptr on read or parse failures. Those error paths leaked partially built cache values because KVCache::get only takes ownership when the builder returns a non-null pointer. This PR uses RAII while building DeleteRows and DeleteFile, releases ownership only after successful construction, and adds fault-injection/regression coverage for V1 Iceberg deletion-vector and position-delete failures, V1 Paimon Parquet/ORC deletion-vector failures, the shared Iceberg helper, and the format v2 TableReader path.

Release note

None

Check List (For Author)

  • Test: Regression test / Unit Test
    • Added BE unit tests for V1/shared Iceberg deletion vector helper fault and stress paths.
    • Added BE unit tests for V1 Iceberg reader deletion-vector and position-delete read failure cache-builder paths.
    • Added BE unit tests for V1 Paimon Parquet/ORC reader deletion vector read failure cache-builder paths.
    • Added BE unit tests for format v2 Iceberg deletion vector fault paths.
    • Extended Iceberg deletion vector regression to cover both scanner v1 and scanner v2 debug point fault injection.
    • Ran build-support/clang-format.sh locally.
    • Ran PATH=/usr/local/opt/llvm@16/bin:$PATH build-support/check-format.sh locally.
    • Ran git diff --check locally.
    • On gabriel@10.26.20.3 in /mnt/disk3/gabriel/Workspace/dev1/doris_pr65351_ut, ran ./run-be-ut.sh --run --filter="IcebergReaderTest.v1_position_delete_read_error_releases_cache_entry:IcebergReaderTest.v1_deletion_vector_read_error_releases_cache_entry:PaimonDeletionVectorTest.V1*:IcebergDeleteFileReaderHelperTest.*:IcebergV2ReaderTest.IcebergDeletionVector*:IcebergV2ReaderTest.IcebergTableReader*DeletionVector*"; 25 tests passed.
    • Attempted build-support/run-clang-tidy.sh --build-dir be/cmake-build-debug-dev-perf locally with CLANG_TIDY_BINARY=/usr/local/opt/llvm@16/bin/clang-tidy; blocked by the local macOS bash 3.2 missing mapfile.
  • Behavior changed: No
  • Does this need documentation: No

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Gabriel39 Gabriel39 requested a review from yiguolei as a code owner July 8, 2026 03:37
@Gabriel39 Gabriel39 force-pushed the codex/iceberg-dv-fault-injection-tests branch from 05e82db to 18b2449 Compare July 8, 2026 03:39
@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

Gabriel39 added a commit to Gabriel39/incubator-doris that referenced this pull request Jul 8, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#65351

Problem Summary: Deletion vector and position-delete cache builders allocated DeleteRows with raw new and returned nullptr on read or parse failures. Those error paths leaked the partially built DeleteRows object and left the failure behavior under-tested in the V1 Iceberg and Paimon readers. Use RAII while building DeleteRows and release ownership only after successful construction. Add UT coverage for V1 Iceberg reader deletion vector read failures, V1 Paimon Parquet/ORC deletion vector read failures, the shared Iceberg helper fault paths, and the format v2 TableReader deletion vector fault paths.

### Release note

None

### Check List (For Author)

- Test: Regression test / Unit Test

    - Added BE unit tests for V1/shared Iceberg deletion vector helper fault and stress paths.

    - Added BE unit tests for V1 Iceberg reader deletion vector read failure cache-builder path.

    - Added BE unit tests for V1 Paimon Parquet/ORC reader deletion vector read failure cache-builder paths.

    - Added BE unit tests for format v2 Iceberg deletion vector fault paths.

    - Extended Iceberg deletion vector regression to cover both scanner v1 and scanner v2 debug point fault injection.

    - Ran build-support/clang-format.sh locally.

    - Ran PATH=/usr/local/opt/llvm@16/bin:$PATH build-support/check-format.sh locally.

    - Ran git diff --check locally.

    - On gabriel@10.26.20.3 in /mnt/disk3/gabriel/Workspace/dev1/doris_pr65351_ut, ran ./run-be-ut.sh --run --filter="IcebergReaderTest.v1_deletion_vector_read_error_releases_cache_entry:PaimonDeletionVectorTest.V1*:IcebergDeleteFileReaderHelperTest.*:IcebergV2ReaderTest.IcebergDeletionVector*:IcebergV2ReaderTest.IcebergTableReader*DeletionVector*"; 24 tests passed.

    - Attempted build-support/run-clang-tidy.sh --build-dir be/cmake-build-debug-dev-perf locally with CLANG_TIDY_BINARY=/usr/local/opt/llvm@16/bin/clang-tidy; blocked by the local macOS bash 3.2 missing mapfile.

- Behavior changed: No

- Does this need documentation: No
@Gabriel39 Gabriel39 force-pushed the codex/iceberg-dv-fault-injection-tests branch from 18b2449 to a597245 Compare July 8, 2026 05:32
@Gabriel39

Copy link
Copy Markdown
Contributor Author

/review

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

@Gabriel39 Gabriel39 changed the title [test](iceberg) Add deletion vector fault injection tests [fix](be) Release deletion vector rows on read errors Jul 8, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed for PR 65351 at head a597245.

I found one ownership issue that should be fixed before merge. The PR converts the deletion-vector DeleteRows cache builders to RAII, but the V1 Iceberg position-delete DeleteFile cache builder in the same method still leaks when _read_position_delete_file fails.

Critical checkpoint conclusions:

  • Goal/test coverage: The deletion-vector read-error leak paths are covered by new unit/regression tests, but the parallel V1 Iceberg position-delete read-error path remains uncovered and still has the same cache-builder ownership problem.
  • Scope/focus: The main code change is small and focused; the missed issue is a directly adjacent parallel path in touched code.
  • Concurrency/lifecycle: KVCache::get serializes creation under its shard lock; no new concurrency issue found. The remaining issue is lifecycle/ownership on a builder failure path.
  • Config/session/compatibility: No production config or serialization compatibility change found. The regression toggles enable_file_scanner_v2 and restores it in finally.
  • Parallel paths: V1 Iceberg DV, V1 Paimon DV, and format-v2 DV builders use unique_ptr correctly; V1 Iceberg position-delete DeleteFile builder is the missed parallel path.
  • Tests/results: I did not run BE/regression tests in this runner because .worktree_initialized, thirdparty/installed, and thirdparty/installed/bin/protoc are missing. Static review only.
  • Observability, persistence, data writes: Not materially affected by this PR.

User focus: no additional user-provided review focus was present.

Subagent conclusions: optimizer-rewrite reported OR-001 as a duplicate of MAIN-001 and found no additional optimizer/rewrite or reader-semantics issue. tests-session-config reported TSC-001 as duplicate/supporting evidence for MAIN-001 and found no additional tests/session/config issue. Convergence round 1 ended with both live subagents replying NO_NEW_VALUABLE_FINDINGS for the same ledger and proposed final comment set.

@@ -660,10 +666,10 @@ Status IcebergReaderMixin<BaseReader>::_position_delete_base(
SCOPED_TIMER(_iceberg_profile.delete_rows_sort_time);
_iceberg_delete_rows =
_kv_cache->template get<DeleteRows>(data_file_path, [&]() -> DeleteRows* {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This RAII conversion fixes the DeleteRows builder here, but _position_delete_base still has the same leak pattern in the DeleteFile cache builder just above it: auto* position_delete = new DeleteFile, then _read_position_delete_file(...), and on any non-OK status it returns nullptr. KVCache::get only takes ownership when the builder returns a non-null pointer, so a missing/corrupt position-delete file still leaks the partially built DeleteFile. Please wrap that builder in std::unique_ptr<DeleteFile> too and release only after _read_position_delete_file succeeds.

Gabriel39 added a commit to Gabriel39/incubator-doris that referenced this pull request Jul 8, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#65351

Problem Summary: Deletion vector and position-delete cache builders allocated rows/maps with raw `new` and returned `nullptr` on read or parse failures. Those error paths leaked partially built cache values because `KVCache::get` only takes ownership when the builder returns a non-null pointer. Use RAII while building `DeleteRows` and `DeleteFile`, and release ownership only after successful construction. Add fault-injection/regression coverage for V1 Iceberg deletion-vector and position-delete failures, V1 Paimon Parquet/ORC deletion-vector failures, the shared Iceberg helper, and the format v2 TableReader path.

### Release note

None

### Check List (For Author)

- Test: Regression test / Unit Test

    - Added BE unit tests for V1/shared Iceberg deletion vector helper fault and stress paths.

    - Added BE unit tests for V1 Iceberg reader deletion vector and position-delete read failure cache-builder paths.

    - Added BE unit tests for V1 Paimon Parquet/ORC reader deletion vector read failure cache-builder paths.

    - Added BE unit tests for format v2 Iceberg deletion vector fault paths.

    - Extended Iceberg deletion vector regression to cover both scanner v1 and scanner v2 debug point fault injection.

    - Ran build-support/clang-format.sh locally.

    - Ran PATH=/usr/local/opt/llvm@16/bin:$PATH build-support/check-format.sh locally.

    - Ran git diff --check locally.

    - On gabriel@10.26.20.3 in /mnt/disk3/gabriel/Workspace/dev1/doris_pr65351_ut, ran ./run-be-ut.sh --run --filter="IcebergReaderTest.v1_position_delete_read_error_releases_cache_entry:IcebergReaderTest.v1_deletion_vector_read_error_releases_cache_entry:PaimonDeletionVectorTest.V1*:IcebergDeleteFileReaderHelperTest.*:IcebergV2ReaderTest.IcebergDeletionVector*:IcebergV2ReaderTest.IcebergTableReader*DeletionVector*"; 25 tests passed.

    - Attempted build-support/run-clang-tidy.sh --build-dir be/cmake-build-debug-dev-perf locally with CLANG_TIDY_BINARY=/usr/local/opt/llvm@16/bin/clang-tidy; blocked by the local macOS bash 3.2 missing mapfile.

- Behavior changed: No

- Does this need documentation: No
@Gabriel39 Gabriel39 force-pushed the codex/iceberg-dv-fault-injection-tests branch from a597245 to 658e7ee Compare July 8, 2026 06:07
@Gabriel39

Copy link
Copy Markdown
Contributor Author

/review

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part 1.3 checkpoint conclusions:

  • Goal/test proof: the RAII ownership changes address the deletion-vector/position-delete cache leak goal, and negative coverage was added, but one new V1 regression assertion is invalid and is called out inline.
  • Scope/focus: the code changes are focused on cache-builder ownership and fault-injection coverage.
  • Concurrency/lifecycle: no new shared concurrency or static/global lifecycle risk was found; the main lifecycle change improves ownership by using unique_ptr until cache publication succeeds.
  • Config/compatibility/parallel paths: no new persistent format, thrift, or rolling-upgrade compatibility concern was found. V1/V2 Iceberg and Paimon paths were checked; the V1 should_stop test path needs correction.
  • Error handling/data correctness/performance/observability: error propagation and cache insertion behavior are mostly preserved; no additional observability appears necessary for this leak fix.
  • Tests/results: BE unit and regression coverage were added, but the V1 should_stop regression should be fixed before relying on it.

Subagent conclusions: optimizer-rewrite and tests-session-config proposed no candidates in their initial pass. After MAIN-1 and the proposed final comment set were recorded, final convergence round 1 ended with both subagents replying NO_NEW_VALUABLE_FINDINGS for the same ledger state.

User focus: no additional user-provided review focus was supplied.

Validation: static review only in this runner. .worktree_initialized and thirdparty/installed were missing, so I did not run local builds or tests.

Gabriel39 added a commit to Gabriel39/incubator-doris that referenced this pull request Jul 8, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: apache#65351

Problem Summary: Deletion vector and position-delete cache builders allocated rows/maps with raw `new` and returned `nullptr` on read or parse failures. Those error paths leaked partially built cache values because `KVCache::get` only takes ownership when the builder returns a non-null pointer. Use RAII while building `DeleteRows` and `DeleteFile`, and release ownership only after successful construction. Add fault-injection/regression coverage for V1 Iceberg deletion-vector and position-delete failures, V1 Paimon Parquet/ORC deletion-vector failures, the shared Iceberg helper, and the format v2 TableReader path.

### Release note

None

### Check List (For Author)

- Test: Regression test / Unit Test

    - Added BE unit tests for V1/shared Iceberg deletion vector helper fault and stress paths.

    - Added BE unit tests for V1 Iceberg reader deletion vector and position-delete read failure cache-builder paths.

    - Added BE unit tests for V1 Paimon Parquet/ORC reader deletion vector read failure cache-builder paths.

    - Added BE unit tests for format v2 Iceberg deletion vector fault paths.

    - Extended Iceberg deletion vector regression to cover both scanner v1 and scanner v2 debug point fault injection.

    - Ran build-support/clang-format.sh locally.

    - Ran PATH=/usr/local/opt/llvm@16/bin:$PATH build-support/check-format.sh locally.

    - Ran git diff --check locally.

    - On gabriel@10.26.20.3 in /mnt/disk3/gabriel/Workspace/dev1/doris_pr65351_ut, ran ./run-be-ut.sh --run --filter="IcebergReaderTest.v1_position_delete_read_error_releases_cache_entry:IcebergReaderTest.v1_deletion_vector_read_error_releases_cache_entry:PaimonDeletionVectorTest.V1*:IcebergDeleteFileReaderHelperTest.*:IcebergV2ReaderTest.IcebergDeletionVector*:IcebergV2ReaderTest.IcebergTableReader*DeletionVector*"; 25 tests passed.

    - Attempted build-support/run-clang-tidy.sh --build-dir be/cmake-build-debug-dev-perf locally with CLANG_TIDY_BINARY=/usr/local/opt/llvm@16/bin/clang-tidy; blocked by the local macOS bash 3.2 missing mapfile.

- Behavior changed: No

- Does this need documentation: No
@Gabriel39 Gabriel39 force-pushed the codex/iceberg-dv-fault-injection-tests branch from 658e7ee to bc11dbe Compare July 8, 2026 06:33
@Gabriel39

Copy link
Copy Markdown
Contributor Author

/review

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed for PR 65351 at head bc11dbe.

I found one remaining test issue that should be fixed before relying on this regression coverage: the new scanner-v2 should_stop case expects a SQL exception for an END_OF_FILE status that the scanner framework treats as normal EOS.

Critical checkpoint conclusions:

  • Goal/test proof: the RAII cache-builder changes address the intended leak pattern across the touched V1/V2 Iceberg and Paimon paths, with added unit/regression coverage. One new regression assertion is invalid and is called out inline.
  • Scope/focus: the production change is small and focused on cache-builder ownership and fault injection.
  • Concurrency/lifecycle: KVCache::get still serializes cache creation under the shard lock; the ownership changes improve failure-path lifecycle by releasing only after successful construction.
  • Config/session/compatibility: no production config, serialization, storage-format, or rolling-upgrade compatibility change was found. The regression toggles enable_file_scanner_v2 and restores it in finally.
  • Parallel paths: V1 Iceberg DV/position-delete, V1 Paimon DV, and format-v2 DV paths were checked. The older V1 should_stop thread is addressed in current head, but the current V2 should_stop SQL assertion has the same EOF-as-EOS problem.
  • Error handling/data correctness/performance/observability: error propagation and cache insertion behavior are otherwise preserved; no additional observability appears necessary for this leak fix.
  • Tests/results: git diff --check was clean for the PR files. I did not run BE/regression tests because .worktree_initialized, thirdparty/installed, and thirdparty/installed/bin/protoc are missing in this runner. build-support/check-format.sh could not run because the runner has clang-format 18 instead of the required v16.

User focus: no additional user-provided review focus was supplied.

Subagent conclusions: optimizer-rewrite found no new valuable candidate beyond existing threads. tests-session-config proposed TSC-1; I accepted it as M-1 and prepared the inline comment. Final convergence round 1 ended with both live subagents replying NO_NEW_VALUABLE_FINDINGS for the same ledger and proposed final comment set.

### What problem does this PR solve?

Issue Number: None

Related PR: apache#65351

Problem Summary: Deletion vector and position-delete cache builders allocated rows/maps with raw `new` and returned `nullptr` on read or parse failures. Those error paths leaked partially built cache values because `KVCache::get` only takes ownership when the builder returns a non-null pointer. Use RAII while building `DeleteRows` and `DeleteFile`, and release ownership only after successful construction. Add fault-injection/regression coverage for V1 Iceberg deletion-vector and position-delete failures, V1 Paimon Parquet/ORC deletion-vector failures, the shared Iceberg helper, and the format v2 TableReader path.

### Release note

None

### Check List (For Author)

- Test: Regression test / Unit Test

    - Added BE unit tests for V1/shared Iceberg deletion vector helper fault and stress paths.

    - Added BE unit tests for V1 Iceberg reader deletion vector and position-delete read failure cache-builder paths.

    - Added BE unit tests for V1 Paimon Parquet/ORC reader deletion vector read failure cache-builder paths.

    - Added BE unit tests for format v2 Iceberg deletion vector fault paths.

    - Extended Iceberg deletion vector regression to cover both scanner v1 and scanner v2 debug point fault injection.

    - Ran build-support/clang-format.sh locally.

    - Ran PATH=/usr/local/opt/llvm@16/bin:$PATH build-support/check-format.sh locally.

    - Ran git diff --check locally.

    - On gabriel@10.26.20.3 in /mnt/disk3/gabriel/Workspace/dev1/doris_pr65351_ut, ran ./run-be-ut.sh --run --filter="IcebergReaderTest.v1_position_delete_read_error_releases_cache_entry:IcebergReaderTest.v1_deletion_vector_read_error_releases_cache_entry:PaimonDeletionVectorTest.V1*:IcebergDeleteFileReaderHelperTest.*:IcebergV2ReaderTest.IcebergDeletionVector*:IcebergV2ReaderTest.IcebergTableReader*DeletionVector*"; 25 tests passed.

    - Attempted build-support/run-clang-tidy.sh --build-dir be/cmake-build-debug-dev-perf locally with CLANG_TIDY_BINARY=/usr/local/opt/llvm@16/bin/clang-tidy; blocked by the local macOS bash 3.2 missing mapfile.

- Behavior changed: No

- Does this need documentation: No
@Gabriel39 Gabriel39 force-pushed the codex/iceberg-dv-fault-injection-tests branch from bc11dbe to 6e94309 Compare July 8, 2026 06:57
@Gabriel39

Copy link
Copy Markdown
Contributor Author

run buildall

@Gabriel39

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed for PR 65351 at head 6e94309.

I did not find any new substantiated blocking issue in the current head. I am not adding inline comments: the three live prior concerns are already present in existing threads, and the current patch appears to address them by converting the remaining position-delete builder to RAII and by replacing the regression EOF should_stop expectations with non-EOF injected errors for scanner V1 and V2.

Part 1.3 checkpoint conclusions:

  • Goal/test proof: The PR goal is to avoid leaking partially built delete-vector/position-delete cache values on read/parse errors. Current head uses std::unique_ptr while building DeleteRows/DeleteFile and releases only on successful cache insertion paths. The patch adds BE unit coverage for Iceberg/Paimon V1, Iceberg helper, and format-v2 TableReader fault paths, plus regression coverage for scanner V1/V2 injected read failures.
  • Scope/focus: The production changes are narrow and limited to cache-builder ownership, shared Iceberg DV decode reuse, and debug-point injection for test coverage.
  • Concurrency/lifecycle: KVCache::get still serializes creation under the shard mutex and only owns non-null returned pointers. The RAII locals are destroyed on failed create callbacks and released only on success. No new thread or long-lived shared lifecycle is introduced.
  • Configuration/session: No new config item is added. The regression temporarily changes enable_file_scanner_v2 and restores it in finally; BE debug points are also cleared in finally.
  • Compatibility/storage/protocol: No storage format, persisted metadata, thrift protocol, or mixed-version contract changes were found.
  • Parallel paths: V1 Iceberg, V1 Paimon Parquet/ORC, shared Iceberg helper, and format-v2 Iceberg DV paths were reviewed. Scanner V1 and scanner V2 regression paths are both covered by non-EOF injected errors in the current head.
  • Testing: Added tests target the relevant error paths. I could not run them in this runner because the checkout is missing .worktree_initialized, thirdparty/installed, and the base/parent commit needed for local diff checks; validation here is static plus GitHub PR patch/API inspection.
  • Observability/performance: No new observability need was identified. The ownership changes do not add material hot-path work beyond RAII wrapper construction.
  • User focus: No additional user-provided review focus was present.

Subagent conclusions:

  • optimizer-rewrite: no optimizer/rewrite or parallel join/aggregate candidate found; convergence round 1 returned NO_NEW_VALUABLE_FINDINGS.
  • tests-session-config: no test/session/config candidate found beyond existing threads; convergence round 1 returned NO_NEW_VALUABLE_FINDINGS.

Final opinion: comment-only review with no new inline comments for the current head.

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29641 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6e94309a6cae89956f8a2b00b995dfffeaa3eeb4, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17644	4044	4004	4004
q2	2023	332	202	202
q3	10301	1491	826	826
q4	4681	472	336	336
q5	7536	861	583	583
q6	182	171	136	136
q7	783	842	630	630
q8	9358	1609	1547	1547
q9	5671	4412	4385	4385
q10	6810	1799	1542	1542
q11	518	347	321	321
q12	730	549	419	419
q13	18102	3341	2789	2789
q14	268	266	241	241
q15	q16	792	787	718	718
q17	985	972	994	972
q18	7205	5799	5568	5568
q19	1332	1299	1042	1042
q20	779	634	556	556
q21	5873	2717	2521	2521
q22	440	355	303	303
Total cold run time: 102013 ms
Total hot run time: 29641 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4355	4300	4261	4261
q2	295	321	215	215
q3	4603	4985	4342	4342
q4	2047	2137	1347	1347
q5	4497	4299	4342	4299
q6	229	175	128	128
q7	1752	2048	1815	1815
q8	2522	2248	2212	2212
q9	8119	8163	7873	7873
q10	4751	4724	4268	4268
q11	567	404	379	379
q12	792	776	557	557
q13	3258	3646	3035	3035
q14	297	325	283	283
q15	q16	706	736	653	653
q17	1351	1324	1315	1315
q18	7932	7397	7248	7248
q19	1158	1140	1106	1106
q20	2200	2203	1937	1937
q21	5230	4575	4473	4473
q22	551	455	399	399
Total cold run time: 57212 ms
Total hot run time: 52145 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 180177 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6e94309a6cae89956f8a2b00b995dfffeaa3eeb4, data reload: false

query5	4332	619	486	486
query6	460	241	205	205
query7	4863	577	340	340
query8	336	197	169	169
query9	8768	4149	4117	4117
query10	474	355	300	300
query11	5907	2350	2141	2141
query12	161	102	102	102
query13	1268	587	473	473
query14	6274	5311	4988	4988
query14_1	4357	4333	4321	4321
query15	223	206	180	180
query16	1011	411	446	411
query17	972	732	592	592
query18	2438	485	366	366
query19	215	200	158	158
query20	116	111	104	104
query21	235	161	135	135
query22	13657	13564	13372	13372
query23	17583	16634	16219	16219
query23_1	16317	16244	16309	16244
query24	7535	1800	1329	1329
query24_1	1321	1299	1281	1281
query25	552	465	395	395
query26	1338	368	206	206
query27	2626	588	370	370
query28	4429	2004	2027	2004
query29	1097	634	496	496
query30	334	258	232	232
query31	1135	1119	1000	1000
query32	111	65	61	61
query33	529	333	266	266
query34	1170	1111	653	653
query35	775	793	673	673
query36	1403	1393	1207	1207
query37	153	112	96	96
query38	1884	1696	1642	1642
query39	937	917	875	875
query39_1	865	881	873	873
query40	235	154	135	135
query41	67	63	64	63
query42	94	92	92	92
query43	328	343	286	286
query44	1449	786	768	768
query45	204	186	176	176
query46	1061	1191	752	752
query47	2354	2361	2259	2259
query48	389	389	298	298
query49	586	440	315	315
query50	1036	422	313	313
query51	10856	10661	11022	10661
query52	85	87	75	75
query53	256	273	208	208
query54	283	228	239	228
query55	79	70	66	66
query56	283	301	292	292
query57	1440	1393	1316	1316
query58	277	249	256	249
query59	1649	1716	1455	1455
query60	292	269	242	242
query61	151	150	146	146
query62	695	645	584	584
query63	244	206	206	206
query64	2827	1034	851	851
query65	4847	4812	4776	4776
query66	1850	508	382	382
query67	29515	29533	29401	29401
query68	2947	1597	1025	1025
query69	402	300	271	271
query70	1096	931	962	931
query71	358	311	300	300
query72	3105	2824	2392	2392
query73	809	763	410	410
query74	5097	4951	4804	4804
query75	2620	2591	2223	2223
query76	2350	1170	773	773
query77	346	381	294	294
query78	12404	12245	11753	11753
query79	1368	1115	767	767
query80	711	544	461	461
query81	479	323	292	292
query82	560	156	123	123
query83	385	311	289	289
query84	318	155	126	126
query85	916	642	498	498
query86	356	295	300	295
query87	1853	1827	1773	1773
query88	3689	2788	2809	2788
query89	467	404	357	357
query90	1924	203	196	196
query91	198	188	158	158
query92	61	62	57	57
query93	1642	1560	974	974
query94	602	367	299	299
query95	790	587	478	478
query96	1036	763	344	344
query97	2690	2665	2569	2569
query98	226	209	199	199
query99	1139	1171	1022	1022
Total cold run time: 264934 ms
Total hot run time: 180177 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 25.11 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6e94309a6cae89956f8a2b00b995dfffeaa3eeb4, data reload: false

query1	0.01	0.01	0.01
query2	0.10	0.04	0.05
query3	0.26	0.14	0.14
query4	1.60	0.14	0.14
query5	0.28	0.22	0.22
query6	1.24	1.08	1.08
query7	0.03	0.01	0.01
query8	0.05	0.03	0.04
query9	0.38	0.32	0.33
query10	0.55	0.58	0.58
query11	0.19	0.14	0.13
query12	0.18	0.15	0.14
query13	0.47	0.48	0.46
query14	1.02	1.01	0.99
query15	0.63	0.59	0.59
query16	0.33	0.33	0.32
query17	1.10	1.10	1.12
query18	0.22	0.21	0.20
query19	2.03	1.98	1.99
query20	0.02	0.01	0.01
query21	15.45	0.21	0.13
query22	4.77	0.05	0.05
query23	16.15	0.31	0.12
query24	2.99	0.42	0.32
query25	0.11	0.05	0.04
query26	0.72	0.20	0.16
query27	0.05	0.03	0.03
query28	3.58	0.94	0.55
query29	12.46	3.97	3.19
query30	0.27	0.15	0.16
query31	2.76	0.60	0.32
query32	3.23	0.59	0.48
query33	3.18	3.21	3.24
query34	15.59	4.22	3.57
query35	3.55	3.51	3.52
query36	0.55	0.45	0.44
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.18	0.16	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 96.63 s
Total hot run time: 25.11 s

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 80.00% (24/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.10% (29288/40065)
Line Coverage 56.70% (316599/558381)
Region Coverage 53.18% (263453/495438)
Branch Coverage 54.23% (115668/213310)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 80.00% (24/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.11% (29291/40065)
Line Coverage 56.71% (316630/558381)
Region Coverage 53.18% (263468/495438)
Branch Coverage 54.22% (115667/213310)

@Gabriel39

Copy link
Copy Markdown
Contributor Author

run external

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 93.33% (28/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 74.82% (29978/40065)
Line Coverage 58.85% (328619/558381)
Region Coverage 55.55% (275238/495438)
Branch Coverage 56.90% (121372/213310)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants