Repository navigation
Aggregation implementation
Implementation reference checked against f5e786f. Fundamental explanations remain in docs/arch.
compiler.cpp calls
BuildGenericAggregateFusedProgramAst from lib.cpp.
The generated program exports agg_dispatch and agg_finish_rowset together.
It is not two separately JIT-compiled update/finalize modules.
agg_dispatch(table, batch, arg, op) -> i64
agg_finish_rowset(table, output) -> i64
| op | Operation | arg |
|---|---|---|
| 0 | Initialize | Power-of-two capacity, at least 8 |
| 1 | Update batch | Unused |
| 2 | Destroy | Unused |
TAggregateProcessor in aggregate_exec.cpp
defers initialization, feeds selected batches, and invokes output finalization.
Kernel-owned output uses DestroyKernelOwnedRowSet. It must not retain pointers
into the destroyed table arena.
The 104-byte external HashTable is defined in
qumirdb.cpp and mirrored by the processor.
| Field | Storage |
|---|---|
| Keys | Probe keys |
| Ctrl | Control bytes, replacing the former probe-distance array |
| SlotId | Stable dense slot for each probe entry |
| GroupKeys | Dense keys |
| AggBuffers | Reducer-specific state arrays |
| OwnedBlocks / OwnedBlockCount / OwnedBlockCapacity | Arena registry |
| Capacity / Size | Table capacity and group count |
| NumAggs / NumKeys / KeySize | Reducer/key layout metadata |
swiss_group.oz selects portable
SWAR, ARM NEON, or SSE2 group matching. SSE2 uses 16 slots; portable/NEON use 8.
Growth happens above the 7/8 occupancy threshold. Historical rh_ and
robin_hood_ names remain in helpers even though probing uses control bytes.
aggregate_key.cpp describes concrete scalar or composite keys. gen.cpp emits hashing, equality, ownership, and typed reducer operations. Fixed-width, f64, string, nullable, and supported BinInt/decimal values use their concrete representations; reducers can read different argument columns.
The former fixed-i64-only, no-string, no-NULL limitations are obsolete. The old field-by-field notes are preserved as historical material in the migration archive.
Defaults are in settings.h.
ShouldUsePartialAggregate requires ordinary grouping, usable input/group
statistics, at least 1,048,576 input rows, and at least an 8x predicted reduction.
--no-partial-aggregates disables that selection. --cascade-aggregates enables
the separate global partial/gather/combine path; it is off by default.