Skip to content

[Bug]: unify join typed-key canonicalization across resident and spill paths #26432

Description

@XuPeng-SH

Context

Follow-up from #26408 and #26412. The ownership/lifecycle fixes in #26412 intentionally do not redesign join-key encoding.

Problem

Resident hash joins, initial spill partitioning, and re-spill must use one SQL-equality canonicalization contract. Today the paths can consistently disagree with SQL equality or with a nested-loop oracle for values such as:

  • DOUBLE +0 versus -0;
  • NaN values;
  • JSON numeric forms such as 1 versus 1.0;
  • scaled floating-point values;
  • long VECF32 values containing +0/-0.

Fixing isolated hash functions per type is insufficient: encoding, hashing, equality, partitioning, and rebuild must share the same typed-key contract.

Proposed direction

  • Define one typed-key codec/canonicalization interface used by resident build/probe, initial spill, re-spill, and remote execution.
  • Specify NULL, NaN, signed-zero, JSON numeric, vector, decimal/scale, and collation behavior explicitly.
  • Version any persisted spill representation affected by the codec.
  • Use nested-loop execution as the semantic oracle rather than comparing two hash implementations.

Acceptance criteria

  • Resident, initial-spill, and re-spill joins return identical rows for the same inputs.
  • Differential tests compare hash join variants against a nested-loop oracle for all supported key types and the edge cases above.
  • Tests cover HashJoin, DedupJoin, and RightDedupJoin, including multi-column and NULL-containing keys.
  • Remote/local execution and mixed spill rounds use the same codec version.
  • Benchmarks show no material regression for common integer/string join keys.

Related: #26408, #26412.

Metadata

Metadata

Assignees

Labels

kind/bugSomething isn't workingneeds-triageNeeds evaluation before prioritization. Not yet decided whether to proceed

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions