Skip to content

feat: parallel int sum/avg partials (#289 phase 5/6 slice 2) - #346

Merged
jdatcmd merged 1 commit into
jdatcmd:mainfrom
ChronicallyJD:feat/289-parallel-int-agg
Aug 3, 2026
Merged

feat: parallel int sum/avg partials (#289 phase 5/6 slice 2)#346
jdatcmd merged 1 commit into
jdatcmd:mainfrom
ChronicallyJD:feat/289-parallel-int-agg

Conversation

@ChronicallyJD

Copy link
Copy Markdown
Collaborator

What

Extends the parallel-aware ungrouped fold (#343) to sum/avg over int2/int4, completing the batch-eligible kinds for parallelism (count / sum / avg over int and float). count(*), avg(float8) was #343; this adds the integer measures common in analytics.

Follow-up to #343 (now merged): extends the same parallel-fold pattern to integer measures.

How

Same shape as #343 (parallel partial ColumnarAgg → Gather → core Finalize), extending only the per-worker partial emission:

  • sum(int2/int4) → its int8 partial, NULL until a value is seen so the strict int8pl combine (and its overflow check) match core's own parallel sum(int).
  • avg(int2/int4) → the int8[2] {N, sum} array int4_avg_accum builds, combined by int4_avg_combine and finalized by int8_avg to numeric. Built with construct_array + explicit INT8OID params (not construct_array_builtin, same cross-version reason as feat: parallel-aware ungrouped vectorized batch fold (#289 phase 5/6) #343's float8 array).

Tests

Integer sums and numeric avg have no float reassociation, so the parallel fold equals the serial oracle exactly (not within tolerance). parallel_vector_agg.sh gains an exact sum(k)+avg(k) check plus a plan premise that the int aggregate takes the parallel fold. 18/18 on pg15a/16a/17a/18a/19a assert.

🤖 Generated with Claude Code

Extend the parallel arm to sum/avg over int2/int4, completing the batch-eligible
kinds (count/sum/avg over int + float). sum(int) emits its int8 partial (NULL
until a value is seen, so the strict int8pl combine and its overflow check match
core's parallel sum); avg(int) emits the int8[2] {N,sum} array int4_avg_accum
builds, combined by int4_avg_combine and finalized by int8_avg -- built with
construct_array + explicit INT8OID params (not construct_array_builtin, same
cross-version reason as the float8 array).

Integer sums and numeric avg have no float reassociation, so the parallel fold
equals the serial oracle EXACTLY: parallel_vector_agg gains an exact sum(k)+avg(k)
check and a plan premise. 18/18 on pg15a/16a/17a/18a/19a assert.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UX1jrWiQsJJA1t4pkmkb4T
@jdatcmd

jdatcmd commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Verified locally on all five majors: 18/18 on PG15, 16, 17, 18 and 19. Branch already contains merged main (#343 + #344), and merged main itself gates clean across the matrix.

Two things this gets right that are worth naming, because both were learned the hard way on #343:

  • It uses construct_array with explicit type params from the start, matching core's own int4_avg_accum exactly (INT8OID, sizeof(int64), FLOAT8PASSBYVAL, TYPALIGN_DOUBLE). feat: parallel-aware ungrouped vectorized batch fold (#289 phase 5/6) #343 shipped construct_array_builtin and errored at runtime on three of five majors while building fine on all of them. Not repeated here.
  • The eligibility guard excludes sum/avg over int8/numeric because their transition state is internal, leaving them on the serial node. Declining to fold those is the right call; combining an internal transition state across workers is how you get silent wrong answers rather than crashes.

The test property is also stronger than #343's, and correctly so: integer sums and numeric division have no reassociation, so parallel must equal serial exactly rather than within a tolerance. That makes any future divergence unambiguous instead of a judgement call about how much drift is acceptable.

No objections from me.

@jdatcmd
jdatcmd merged commit 8518015 into jdatcmd:main Aug 3, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants