Skip to content

perf: early termination for right semi/anti hash joins#21775

Open
SubhamSinghal wants to merge 1 commit intoapache:mainfrom
SubhamSinghal:semi-anti-early-termination
Open

perf: early termination for right semi/anti hash joins#21775
SubhamSinghal wants to merge 1 commit intoapache:mainfrom
SubhamSinghal:semi-anti-early-termination

Conversation

@SubhamSinghal
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

For RightSemi/RightAnti hash joins, the probe phase only needs to know whether at least one matching build row exists per probe row. However, the current code:

  1. Traverses the entire collision chain, emitting every (probe, build) pair
  2. Runs equal_rows_arr on all pairs (take() + eq_dyn_null(), allocating intermediate arrays)
  3. Deduplicates via get_semi_indices to keep just the first match per probe row

For skewed data with N duplicate build rows per key, this produces N pairs per probe row only to keep 1 — O(N) wasted work in both chain traversal and equality verification.

What changes are included in this PR?

New get_first_match method on JoinHashMapType trait (join_hash_map.rs):

  • Walks the collision chain for a hash value, stops at the first entry where a caller-provided predicate returns true
  • Returns Option<u64> — the matching build row index, or None
  • Shared implementation in get_first_match_impl, reused by JoinHashMapU32, JoinHashMapU64, and PruningJoinHashMap

New process_probe_batch_right_semi_anti function (stream.rs):

  • For each probe row, calls get_first_match with an equality predicate built from JoinKeyComparator::is_equal (zero-allocation per-row comparison)
  • Collects matched probe indices (RightSemi) or unmatched probe indices (RightAnti)
  • Builds output via existing build_batch_from_indices

Are these changes tested?

Covered by existing tests:

  • 791 hash join unit tests (all pass)
  • 14 SLT join test files (all pass)
  • TPC-H SF1 benchmark: no regressions, Q4/Q21 (semi/anti patterns) show minor improvements

No new tests added — this is a pure optimization that produces identical results. The existing test suite comprehensively covers RightSemi/RightAnti join.

Are there any user-facing changes?

No.

TPC-H Benchmark result:

Query main (ms) new (ms) Change
Q1 2.96 2.89 ~same
Q2 3.12 2.80 ~same
Q3 1.70 1.77 ~same
Q4 1.23 1.15 -7%
Q5 2.11 2.04 ~same
Q6 0.69 0.75 ~same
Q7 2.58 2.51 ~same
Q8 2.94 2.91 ~same
Q9 2.19 2.03 ~same
Q10 2.20 2.15 ~same
Q11 1.95 1.91 ~same
Q12 1.38 1.36 ~same
Q13 1.23 1.10 -11%
Q14 1.18 1.12 ~same
Q15 2.28 2.22 ~same
Q16 1.76 1.70 ~same
Q17 1.37 1.26 ~same
Q18 1.61 1.45 -10%
Q19 2.28 2.05 -10%
Q20 2.01 1.81 -10%
Q21 2.72 2.23 -18%
Q22 1.89 1.75 ~same

@github-actions github-actions Bot added the physical-plan Changes to the physical-plan crate label Apr 22, 2026
@Dandandan
Copy link
Copy Markdown
Contributor

run benchmark tpch10

@Dandandan
Copy link
Copy Markdown
Contributor

run benchmark tpch_mem

@adriangbot
Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4294002015-1739-4dcxj 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing semi-anti-early-termination (f436877) to 64619a6 (merge-base) diff using: tpch10
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot
Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4294008461-1740-8w4n8 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing semi-anti-early-termination (f436877) to 64619a6 (merge-base) diff using: tpch_mem
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot
Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details

Comparing HEAD and semi-anti-early-termination
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                              HEAD ┃       semi-anti-early-termination ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │    23.62 / 23.93 ±0.37 / 24.65 ms │    23.24 / 23.77 ±0.33 / 24.29 ms │     no change │
│ QQuery 2  │    10.46 / 10.68 ±0.19 / 11.01 ms │    10.54 / 10.75 ±0.24 / 11.21 ms │     no change │
│ QQuery 3  │    10.51 / 10.75 ±0.23 / 11.10 ms │    10.55 / 10.66 ±0.08 / 10.76 ms │     no change │
│ QQuery 4  │    15.89 / 16.72 ±0.47 / 17.25 ms │    15.63 / 15.86 ±0.18 / 16.06 ms │ +1.05x faster │
│ QQuery 5  │    27.09 / 27.96 ±0.66 / 28.92 ms │    26.83 / 27.57 ±0.67 / 28.42 ms │     no change │
│ QQuery 6  │       6.88 / 7.03 ±0.14 / 7.30 ms │       6.70 / 6.81 ±0.11 / 7.00 ms │     no change │
│ QQuery 7  │    48.69 / 51.53 ±1.61 / 52.90 ms │    51.82 / 52.05 ±0.13 / 52.21 ms │     no change │
│ QQuery 8  │    13.51 / 13.99 ±0.26 / 14.20 ms │    14.19 / 14.36 ±0.09 / 14.44 ms │     no change │
│ QQuery 9  │    29.10 / 30.91 ±2.12 / 35.05 ms │    30.77 / 31.69 ±1.02 / 33.65 ms │     no change │
│ QQuery 10 │    19.30 / 20.77 ±1.25 / 22.63 ms │    20.16 / 20.90 ±0.51 / 21.72 ms │     no change │
│ QQuery 11 │       5.96 / 6.13 ±0.14 / 6.36 ms │       6.05 / 6.18 ±0.11 / 6.37 ms │     no change │
│ QQuery 12 │    16.44 / 17.10 ±0.39 / 17.64 ms │    16.69 / 16.99 ±0.19 / 17.23 ms │     no change │
│ QQuery 13 │    14.34 / 14.85 ±0.32 / 15.36 ms │    14.28 / 14.49 ±0.11 / 14.57 ms │     no change │
│ QQuery 14 │       5.93 / 6.03 ±0.11 / 6.25 ms │       5.80 / 5.86 ±0.05 / 5.92 ms │     no change │
│ QQuery 15 │    10.50 / 10.76 ±0.17 / 11.00 ms │    10.70 / 10.91 ±0.11 / 11.00 ms │     no change │
│ QQuery 16 │    10.22 / 10.33 ±0.08 / 10.44 ms │    10.23 / 10.40 ±0.12 / 10.55 ms │     no change │
│ QQuery 17 │    42.66 / 43.36 ±0.52 / 44.15 ms │    42.27 / 43.99 ±0.95 / 45.05 ms │     no change │
│ QQuery 18 │    64.92 / 66.93 ±1.33 / 68.32 ms │    68.49 / 70.47 ±1.73 / 73.12 ms │  1.05x slower │
│ QQuery 19 │    14.04 / 14.30 ±0.23 / 14.71 ms │    14.57 / 15.66 ±1.63 / 18.90 ms │  1.09x slower │
│ QQuery 20 │    15.33 / 15.51 ±0.16 / 15.77 ms │    15.77 / 16.16 ±0.25 / 16.44 ms │     no change │
│ QQuery 21 │ 103.88 / 104.56 ±0.64 / 105.40 ms │ 102.03 / 104.51 ±1.82 / 107.66 ms │     no change │
│ QQuery 22 │       8.52 / 8.61 ±0.06 / 8.68 ms │       8.61 / 8.77 ±0.09 / 8.89 ms │     no change │
└───────────┴───────────────────────────────────┴───────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary                          ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD)                          │ 532.75ms │
│ Total Time (semi-anti-early-termination)   │ 538.81ms │
│ Average Time (HEAD)                        │  24.22ms │
│ Average Time (semi-anti-early-termination) │  24.49ms │
│ Queries Faster                             │        1 │
│ Queries Slower                             │        2 │
│ Queries with No Change                     │       19 │
│ Queries with Failure                       │        0 │
└────────────────────────────────────────────┴──────────┘

Resource Usage

tpch_mem — base (merge-base)

Metric Value
Wall time 5.0s
Peak memory 8.2 GiB
Avg memory 5.6 GiB
CPU user 21.1s
CPU sys 0.4s
Peak spill 0 B

tpch_mem — branch

Metric Value
Wall time 5.0s
Peak memory 8.2 GiB
Avg memory 5.6 GiB
CPU user 21.4s
CPU sys 0.5s
Peak spill 0 B

File an issue against this benchmark runner

@adriangbot
Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details

Comparing HEAD and semi-anti-early-termination
--------------------
Benchmark tpch_sf10.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                               HEAD ┃        semi-anti-early-termination ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │  333.96 / 335.79 ±2.86 / 341.50 ms │  328.36 / 329.99 ±1.09 / 331.79 ms │     no change │
│ QQuery 2  │  138.99 / 140.85 ±1.57 / 143.63 ms │  134.27 / 139.28 ±3.50 / 144.06 ms │     no change │
│ QQuery 3  │  295.37 / 307.39 ±8.82 / 321.16 ms │  295.38 / 299.16 ±2.75 / 302.67 ms │     no change │
│ QQuery 4  │  155.66 / 157.89 ±1.44 / 160.09 ms │  155.88 / 157.34 ±1.70 / 160.61 ms │     no change │
│ QQuery 5  │ 435.19 / 447.46 ±11.31 / 465.97 ms │  429.06 / 435.01 ±5.83 / 445.00 ms │     no change │
│ QQuery 6  │  138.88 / 140.64 ±1.38 / 142.81 ms │  133.20 / 135.28 ±2.49 / 140.17 ms │     no change │
│ QQuery 7  │ 560.53 / 588.11 ±17.88 / 608.11 ms │  545.29 / 551.81 ±5.40 / 559.75 ms │ +1.07x faster │
│ QQuery 8  │ 468.69 / 485.40 ±17.34 / 518.45 ms │  472.69 / 475.88 ±2.38 / 479.88 ms │     no change │
│ QQuery 9  │  666.53 / 673.69 ±6.63 / 686.24 ms │ 670.13 / 682.40 ±11.63 / 702.82 ms │     no change │
│ QQuery 10 │  339.00 / 343.22 ±4.08 / 349.08 ms │  337.24 / 340.61 ±2.65 / 344.33 ms │     no change │
│ QQuery 11 │  106.98 / 109.80 ±2.00 / 112.95 ms │  106.14 / 110.97 ±5.77 / 122.29 ms │     no change │
│ QQuery 12 │ 200.88 / 212.51 ±11.10 / 230.85 ms │  202.63 / 208.80 ±7.06 / 220.44 ms │     no change │
│ QQuery 13 │  311.71 / 319.97 ±6.48 / 328.94 ms │  306.64 / 310.98 ±3.63 / 315.44 ms │     no change │
│ QQuery 14 │  185.10 / 188.72 ±3.58 / 194.24 ms │  185.46 / 190.28 ±4.88 / 196.51 ms │     no change │
│ QQuery 15 │  337.45 / 340.72 ±2.91 / 346.13 ms │  330.59 / 335.06 ±4.07 / 342.26 ms │     no change │
│ QQuery 16 │     79.45 / 81.85 ±1.83 / 83.80 ms │     80.48 / 83.89 ±4.83 / 93.50 ms │     no change │
│ QQuery 17 │ 767.89 / 778.33 ±10.67 / 794.94 ms │ 747.88 / 766.45 ±10.77 / 779.34 ms │     no change │
│ QQuery 18 │ 777.30 / 794.18 ±11.69 / 812.81 ms │ 768.05 / 796.04 ±19.39 / 820.18 ms │     no change │
│ QQuery 19 │ 272.07 / 290.19 ±15.45 / 311.58 ms │ 269.37 / 283.09 ±19.41 / 320.35 ms │     no change │
│ QQuery 20 │  309.53 / 315.01 ±4.39 / 320.75 ms │  309.15 / 313.77 ±4.08 / 320.53 ms │     no change │
│ QQuery 21 │  827.98 / 832.92 ±3.75 / 837.27 ms │  824.73 / 833.47 ±6.86 / 842.06 ms │     no change │
│ QQuery 22 │     83.65 / 87.67 ±2.35 / 90.19 ms │     81.69 / 84.65 ±1.79 / 86.47 ms │     no change │
└───────────┴────────────────────────────────────┴────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                          ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                          │ 7972.31ms │
│ Total Time (semi-anti-early-termination)   │ 7864.21ms │
│ Average Time (HEAD)                        │  362.38ms │
│ Average Time (semi-anti-early-termination) │  357.46ms │
│ Queries Faster                             │         1 │
│ Queries Slower                             │         0 │
│ Queries with No Change                     │        21 │
│ Queries with Failure                       │         0 │
└────────────────────────────────────────────┴───────────┘

Resource Usage

tpch10 — base (merge-base)

Metric Value
Wall time 45.0s
Peak memory 12.0 GiB
Avg memory 8.2 GiB
CPU user 426.8s
CPU sys 21.3s
Peak spill 0 B

tpch10 — branch

Metric Value
Wall time 40.0s
Peak memory 12.3 GiB
Avg memory 8.5 GiB
CPU user 422.2s
CPU sys 20.4s
Peak spill 0 B

File an issue against this benchmark runner

@Dandandan
Copy link
Copy Markdown
Contributor

TPC-H Benchmark

@SubhamSinghal this looks too fast to be correct?

@SubhamSinghal
Copy link
Copy Markdown
Contributor Author

SubhamSinghal commented Apr 22, 2026

@Dandandan might be some mistake on my side for calculating benchmark. but the optimisation looks fine right. Planning to file follow-up PR if direction looks right.

@Dandandan
Copy link
Copy Markdown
Contributor

run benchmark tpcds

@adriangbot
Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4298095287-1757-mfdzk 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing semi-anti-early-termination (f436877) to 64619a6 (merge-base) diff using: tpcds
Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot
Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details

Comparing HEAD and semi-anti-early-termination
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                      HEAD ┃               semi-anti-early-termination ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │               6.70 / 7.19 ±0.79 / 8.77 ms │               6.92 / 7.56 ±0.92 / 9.37 ms │  1.05x slower │
│ QQuery 2  │         144.76 / 145.37 ±0.62 / 146.54 ms │         145.15 / 145.94 ±0.62 / 146.90 ms │     no change │
│ QQuery 3  │         112.42 / 113.99 ±1.10 / 115.56 ms │         114.41 / 114.87 ±0.35 / 115.28 ms │     no change │
│ QQuery 4  │     1256.31 / 1418.71 ±82.23 / 1475.30 ms │     1249.94 / 1264.22 ±13.72 / 1286.88 ms │ +1.12x faster │
│ QQuery 5  │         172.70 / 176.40 ±2.34 / 179.23 ms │         174.37 / 175.96 ±0.98 / 176.85 ms │     no change │
│ QQuery 6  │        833.48 / 870.43 ±22.48 / 904.22 ms │        828.00 / 858.85 ±31.84 / 917.70 ms │     no change │
│ QQuery 7  │         335.56 / 339.69 ±2.57 / 342.72 ms │         348.05 / 349.63 ±1.49 / 351.89 ms │     no change │
│ QQuery 8  │         112.17 / 112.88 ±0.69 / 114.09 ms │         116.29 / 117.29 ±1.05 / 119.32 ms │     no change │
│ QQuery 9  │         105.19 / 106.60 ±2.22 / 111.01 ms │         103.14 / 111.60 ±9.55 / 130.26 ms │     no change │
│ QQuery 10 │         101.19 / 102.20 ±0.55 / 102.65 ms │         105.20 / 106.64 ±1.10 / 108.58 ms │     no change │
│ QQuery 11 │      947.25 / 1016.66 ±37.54 / 1049.07 ms │       848.24 / 895.69 ±58.75 / 1003.71 ms │ +1.14x faster │
│ QQuery 12 │            43.48 / 44.45 ±1.22 / 46.74 ms │            43.18 / 43.69 ±0.54 / 44.42 ms │     no change │
│ QQuery 13 │         386.44 / 391.07 ±2.69 / 393.77 ms │         394.05 / 399.24 ±5.97 / 409.35 ms │     no change │
│ QQuery 14 │         987.30 / 992.06 ±3.72 / 996.14 ms │        981.07 / 992.33 ±7.59 / 1000.48 ms │     no change │
│ QQuery 15 │            14.95 / 15.17 ±0.21 / 15.51 ms │            16.04 / 16.15 ±0.15 / 16.44 ms │  1.07x slower │
│ QQuery 16 │               7.53 / 7.65 ±0.13 / 7.87 ms │               8.05 / 8.14 ±0.11 / 8.36 ms │  1.06x slower │
│ QQuery 17 │         222.09 / 223.00 ±0.65 / 223.85 ms │         227.33 / 233.65 ±5.44 / 242.24 ms │     no change │
│ QQuery 18 │         122.77 / 123.58 ±0.75 / 124.82 ms │         122.00 / 123.48 ±1.19 / 125.63 ms │     no change │
│ QQuery 19 │         164.01 / 164.74 ±0.82 / 166.34 ms │         162.81 / 164.32 ±1.05 / 165.89 ms │     no change │
│ QQuery 20 │            13.94 / 14.18 ±0.21 / 14.49 ms │            13.71 / 13.96 ±0.19 / 14.25 ms │     no change │
│ QQuery 21 │            19.87 / 20.24 ±0.24 / 20.62 ms │            19.65 / 20.22 ±0.47 / 20.70 ms │     no change │
│ QQuery 22 │         518.34 / 524.33 ±5.06 / 533.66 ms │        474.09 / 485.76 ±10.26 / 503.99 ms │ +1.08x faster │
│ QQuery 23 │        828.28 / 882.16 ±42.97 / 932.40 ms │        831.60 / 864.33 ±32.48 / 915.31 ms │     no change │
│ QQuery 24 │         376.09 / 380.17 ±2.47 / 382.74 ms │         399.22 / 402.39 ±2.13 / 405.91 ms │  1.06x slower │
│ QQuery 25 │         333.72 / 337.38 ±3.15 / 342.54 ms │         332.96 / 341.56 ±7.46 / 353.09 ms │     no change │
│ QQuery 26 │            77.75 / 78.67 ±0.75 / 80.05 ms │            76.42 / 77.19 ±0.40 / 77.49 ms │     no change │
│ QQuery 27 │               7.10 / 7.23 ±0.14 / 7.49 ms │               6.88 / 6.98 ±0.12 / 7.22 ms │     no change │
│ QQuery 28 │         149.80 / 150.31 ±0.43 / 150.87 ms │        149.61 / 158.99 ±13.28 / 184.30 ms │  1.06x slower │
│ QQuery 29 │         273.72 / 278.23 ±5.92 / 289.95 ms │         271.87 / 277.85 ±7.08 / 291.64 ms │     no change │
│ QQuery 30 │            41.28 / 41.79 ±0.32 / 42.18 ms │            41.25 / 42.05 ±1.01 / 43.99 ms │     no change │
│ QQuery 31 │         162.97 / 165.08 ±1.72 / 167.50 ms │         165.93 / 166.58 ±0.43 / 167.19 ms │     no change │
│ QQuery 32 │            12.97 / 13.33 ±0.29 / 13.69 ms │            13.23 / 13.55 ±0.25 / 14.00 ms │     no change │
│ QQuery 33 │         138.34 / 139.21 ±0.62 / 140.20 ms │         140.93 / 142.29 ±1.55 / 144.33 ms │     no change │
│ QQuery 34 │               6.78 / 6.97 ±0.26 / 7.50 ms │               6.97 / 7.13 ±0.19 / 7.49 ms │     no change │
│ QQuery 35 │          98.30 / 100.36 ±1.43 / 101.84 ms │         100.67 / 102.72 ±2.90 / 108.46 ms │     no change │
│ QQuery 36 │               6.45 / 6.57 ±0.20 / 6.98 ms │               6.38 / 6.59 ±0.23 / 6.93 ms │     no change │
│ QQuery 37 │               8.11 / 8.19 ±0.07 / 8.28 ms │               8.05 / 8.28 ±0.16 / 8.55 ms │     no change │
│ QQuery 38 │            83.66 / 85.19 ±0.93 / 86.53 ms │            85.99 / 88.97 ±2.30 / 92.93 ms │     no change │
│ QQuery 39 │         117.76 / 120.23 ±2.29 / 124.25 ms │         118.88 / 122.21 ±4.33 / 130.64 ms │     no change │
│ QQuery 40 │         101.55 / 104.23 ±2.25 / 108.09 ms │         102.43 / 108.11 ±4.12 / 113.41 ms │     no change │
│ QQuery 41 │            13.93 / 14.08 ±0.25 / 14.57 ms │            14.02 / 14.31 ±0.21 / 14.66 ms │     no change │
│ QQuery 42 │         105.71 / 107.19 ±1.46 / 109.91 ms │         107.51 / 108.55 ±0.97 / 110.15 ms │     no change │
│ QQuery 43 │               5.74 / 5.82 ±0.12 / 6.04 ms │               5.65 / 5.75 ±0.14 / 6.02 ms │     no change │
│ QQuery 44 │            11.50 / 11.58 ±0.09 / 11.71 ms │            11.59 / 11.70 ±0.10 / 11.85 ms │     no change │
│ QQuery 45 │            47.35 / 48.30 ±0.56 / 49.08 ms │            48.14 / 48.73 ±0.37 / 49.14 ms │     no change │
│ QQuery 46 │              8.37 / 8.83 ±0.69 / 10.19 ms │               8.51 / 8.68 ±0.20 / 9.07 ms │     no change │
│ QQuery 47 │        787.78 / 829.63 ±21.77 / 849.72 ms │        697.83 / 804.78 ±63.41 / 865.93 ms │     no change │
│ QQuery 48 │         279.41 / 286.45 ±4.11 / 291.75 ms │         291.06 / 294.21 ±2.74 / 299.34 ms │     no change │
│ QQuery 49 │         250.75 / 251.71 ±0.92 / 253.33 ms │         254.85 / 256.81 ±2.06 / 260.59 ms │     no change │
│ QQuery 50 │         198.17 / 204.15 ±5.50 / 213.09 ms │         209.14 / 216.12 ±5.08 / 224.85 ms │  1.06x slower │
│ QQuery 51 │         174.62 / 176.84 ±1.98 / 179.90 ms │         178.76 / 182.38 ±2.43 / 186.28 ms │     no change │
│ QQuery 52 │         106.39 / 106.98 ±0.46 / 107.65 ms │         107.87 / 109.34 ±1.68 / 112.44 ms │     no change │
│ QQuery 53 │         100.44 / 101.35 ±0.76 / 102.42 ms │         102.26 / 103.65 ±1.03 / 105.29 ms │     no change │
│ QQuery 54 │         141.79 / 143.69 ±1.12 / 144.85 ms │         144.38 / 146.13 ±1.27 / 147.94 ms │     no change │
│ QQuery 55 │         105.49 / 106.30 ±0.85 / 107.85 ms │         107.65 / 107.98 ±0.26 / 108.26 ms │     no change │
│ QQuery 56 │         138.54 / 139.63 ±0.78 / 140.93 ms │         140.11 / 141.27 ±1.04 / 143.17 ms │     no change │
│ QQuery 57 │         164.69 / 165.58 ±0.53 / 166.23 ms │         165.80 / 167.01 ±0.91 / 168.43 ms │     no change │
│ QQuery 58 │         315.71 / 317.52 ±1.65 / 320.54 ms │         310.57 / 311.93 ±1.02 / 313.18 ms │     no change │
│ QQuery 59 │         202.06 / 203.90 ±1.29 / 205.67 ms │         193.60 / 195.62 ±2.13 / 199.63 ms │     no change │
│ QQuery 60 │         145.01 / 146.11 ±1.21 / 148.32 ms │         141.89 / 143.73 ±1.33 / 145.95 ms │     no change │
│ QQuery 61 │            14.26 / 14.30 ±0.04 / 14.38 ms │            13.53 / 13.70 ±0.19 / 14.06 ms │     no change │
│ QQuery 62 │        902.28 / 929.36 ±14.24 / 944.23 ms │         858.23 / 864.41 ±5.36 / 873.38 ms │ +1.08x faster │
│ QQuery 63 │         102.56 / 103.45 ±0.51 / 104.12 ms │         102.54 / 104.59 ±1.50 / 107.07 ms │     no change │
│ QQuery 64 │         658.97 / 662.11 ±2.17 / 664.58 ms │        676.60 / 701.25 ±13.29 / 714.42 ms │  1.06x slower │
│ QQuery 65 │         242.98 / 246.54 ±4.46 / 253.95 ms │         281.76 / 285.57 ±3.21 / 291.36 ms │  1.16x slower │
│ QQuery 66 │         223.56 / 230.32 ±7.56 / 243.81 ms │        214.82 / 238.98 ±14.49 / 256.29 ms │     no change │
│ QQuery 67 │         316.44 / 323.85 ±5.71 / 333.23 ms │        294.38 / 313.51 ±11.77 / 328.83 ms │     no change │
│ QQuery 68 │               9.42 / 9.57 ±0.14 / 9.83 ms │               8.65 / 8.81 ±0.19 / 9.15 ms │ +1.09x faster │
│ QQuery 69 │         100.86 / 102.40 ±1.08 / 104.12 ms │            97.98 / 98.96 ±0.65 / 99.91 ms │     no change │
│ QQuery 70 │        309.40 / 327.22 ±14.88 / 350.35 ms │         306.78 / 320.60 ±8.67 / 332.95 ms │     no change │
│ QQuery 71 │         130.72 / 132.13 ±0.87 / 133.40 ms │         133.00 / 134.25 ±0.67 / 134.97 ms │     no change │
│ QQuery 72 │         588.84 / 593.05 ±3.72 / 599.50 ms │        578.90 / 607.73 ±23.56 / 646.34 ms │     no change │
│ QQuery 73 │               6.55 / 6.69 ±0.20 / 7.10 ms │               7.11 / 7.22 ±0.12 / 7.45 ms │  1.08x slower │
│ QQuery 74 │         553.89 / 559.67 ±3.85 / 565.51 ms │         675.74 / 679.64 ±2.94 / 682.91 ms │  1.21x slower │
│ QQuery 75 │         265.69 / 268.38 ±2.90 / 273.91 ms │         277.99 / 280.11 ±1.39 / 281.88 ms │     no change │
│ QQuery 76 │         129.99 / 130.74 ±0.83 / 132.10 ms │         128.80 / 134.01 ±3.65 / 138.39 ms │     no change │
│ QQuery 77 │         187.06 / 189.35 ±1.96 / 192.35 ms │         187.41 / 190.55 ±1.63 / 192.03 ms │     no change │
│ QQuery 78 │         346.84 / 349.54 ±2.20 / 353.17 ms │        327.08 / 340.29 ±10.14 / 351.22 ms │     no change │
│ QQuery 79 │         255.29 / 258.54 ±2.75 / 261.90 ms │         232.01 / 235.67 ±2.55 / 239.85 ms │ +1.10x faster │
│ QQuery 80 │         326.13 / 329.44 ±2.27 / 332.58 ms │         321.30 / 322.27 ±0.71 / 323.13 ms │     no change │
│ QQuery 81 │            27.15 / 27.58 ±0.39 / 28.21 ms │            25.50 / 25.83 ±0.29 / 26.23 ms │ +1.07x faster │
│ QQuery 82 │            40.93 / 41.08 ±0.14 / 41.28 ms │            39.39 / 40.03 ±0.85 / 41.71 ms │     no change │
│ QQuery 83 │            38.58 / 38.62 ±0.04 / 38.70 ms │            36.66 / 37.12 ±0.30 / 37.56 ms │     no change │
│ QQuery 84 │            47.64 / 47.86 ±0.21 / 48.21 ms │            46.41 / 46.63 ±0.16 / 46.84 ms │     no change │
│ QQuery 85 │         142.51 / 145.19 ±1.94 / 147.39 ms │         140.59 / 144.36 ±3.33 / 150.61 ms │     no change │
│ QQuery 86 │            37.13 / 37.43 ±0.36 / 38.14 ms │            38.36 / 38.83 ±0.55 / 39.82 ms │     no change │
│ QQuery 87 │               3.44 / 3.56 ±0.18 / 3.91 ms │               3.59 / 3.72 ±0.19 / 4.10 ms │     no change │
│ QQuery 88 │            99.20 / 99.49 ±0.26 / 99.83 ms │         101.68 / 102.62 ±0.79 / 103.78 ms │     no change │
│ QQuery 89 │         116.20 / 116.70 ±0.43 / 117.23 ms │         119.36 / 121.34 ±2.75 / 126.76 ms │     no change │
│ QQuery 90 │            22.01 / 22.39 ±0.28 / 22.75 ms │            23.51 / 23.86 ±0.26 / 24.12 ms │  1.07x slower │
│ QQuery 91 │            57.92 / 58.68 ±0.63 / 59.47 ms │            60.44 / 61.52 ±0.65 / 62.41 ms │     no change │
│ QQuery 92 │            55.48 / 56.45 ±0.56 / 57.21 ms │            59.21 / 60.23 ±1.25 / 62.64 ms │  1.07x slower │
│ QQuery 93 │         180.86 / 182.43 ±0.89 / 183.50 ms │         191.73 / 193.23 ±1.30 / 195.23 ms │  1.06x slower │
│ QQuery 94 │            59.85 / 60.35 ±0.43 / 60.91 ms │            63.27 / 63.81 ±0.43 / 64.34 ms │  1.06x slower │
│ QQuery 95 │         125.77 / 126.41 ±0.35 / 126.75 ms │         130.32 / 131.38 ±1.26 / 133.79 ms │     no change │
│ QQuery 96 │            66.80 / 68.86 ±1.27 / 70.68 ms │            72.37 / 73.29 ±0.95 / 74.84 ms │  1.06x slower │
│ QQuery 97 │         116.06 / 117.79 ±1.49 / 119.75 ms │         121.24 / 124.99 ±3.43 / 129.78 ms │  1.06x slower │
│ QQuery 98 │         148.94 / 149.83 ±0.83 / 151.11 ms │         149.04 / 152.45 ±1.94 / 154.70 ms │     no change │
│ QQuery 99 │ 10825.42 / 11064.26 ±186.98 / 11354.09 ms │ 10816.94 / 10984.17 ±118.98 / 11115.37 ms │     no change │
└───────────┴───────────────────────────────────────────┴───────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                          ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                          │ 31443.20ms │
│ Total Time (semi-anti-early-termination)   │ 31277.20ms │
│ Average Time (HEAD)                        │   317.61ms │
│ Average Time (semi-anti-early-termination) │   315.93ms │
│ Queries Faster                             │          7 │
│ Queries Slower                             │         16 │
│ Queries with No Change                     │         76 │
│ Queries with Failure                       │          0 │
└────────────────────────────────────────────┴────────────┘

Resource Usage

tpcds — base (merge-base)

Metric Value
Wall time 160.0s
Peak memory 6.6 GiB
Avg memory 5.6 GiB
CPU user 262.3s
CPU sys 8.0s
Peak spill 0 B

tpcds — branch

Metric Value
Wall time 160.0s
Peak memory 6.2 GiB
Avg memory 5.5 GiB
CPU user 261.3s
CPU sys 8.5s
Peak spill 0 B

File an issue against this benchmark runner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants