fix(frontend): preserve non-finite data branch floats

[gouhongshen]

What type of PR is this?

• API-change
• BUG
• Improvement
• Documentation
• Feature
• Test and CI
• Code Refactoring

Which issue(s) this PR fixes:

Fixes #26072

What this PR does / why we need it:

Root cause

Data Branch converts rows back into SQL for MERGE, PICK, and portable output. The shared float formatter emitted bare NaN, +Inf, and -Inf tokens, which the parser treated as expressions rather than FLOAT/DOUBLE values.

Generated SQL and conflict matching also lost MatrixOne's bit-preserving FLOAT/DOUBLE storage identity:

• Scalar equality cannot match NaN, collapses signed zero, and cannot distinguish NaN payloads.
• Textual NaN casts canonicalize payload bits, while cast(-0 as double) produces positive zero.
• Applying a same-key update as DELETE plus INSERT can remove or collide with a bit-distinct peer.
• PICK initially bypassed the exact update path, and UPDATE alone could not restore a source update when the destination independently deleted the row.
• Simple FLOAT/DOUBLE primary-key duplicate checks and fuzzy-filter collision recovery still consumed scalar keys, so a direct exact-key restore could fail or expose serialized bytes in duplicate diagnostics.
• Hash-diff conflict sorting and equality also consumed raw scalar FLOAT/DOUBLE keys, collapsing crossed independent changes on signed zero or distinct NaN payloads into false conflicts.

Changes

• Format non-finite FLOAT/DOUBLE values as typed casts and NaN payloads/negative zero as bit-preserving bit_cast(unhex(...)) literals.
• Use serial(...) as the shared exact identity for FLOAT/DOUBLE key components in LCA probes, staged deletes, portable predicates, MERGE, and PICK. Non-float key behavior is unchanged.
• Preserve update pairing. Same-key FLOAT/DOUBLE updates use an exact-key UPDATE; genuine inserts remain direct INSERTs.
• Apply PICK conflict policy before materialization. FAIL errors, SKIP preserves the destination state, and ACCEPT restores a proven-missing source row with one direct INSERT.
• Make simple FLOAT/DOUBLE primary-key DEDUP, constraint, intra-batch, and fuzzy-filter checks compare serialized identities. Fuzzy-filter allocation now follows the projected identity type, background collision SQL groups on serial(pk), and duplicate errors decode the original FLOAT/DOUBLE value.
• Validate staged key metadata before emitting SQL and share the exact-key logic across MERGE, PICK, and portable output.
• Give simple FLOAT/DOUBLE hash-diff conflict matching one shared bit-preserving IEEE key order for both sorting and equality; composite serialized keys retain their existing exact byte order.

Tests

Verification on head 12db1fb854, based on c58cd292d3:

• Full controlled CGo tests for pkg/frontend, pkg/sql/plan, pkg/sql/colexec, and pkg/sql/compile — pass.
• Controlled go vet for those four packages — pass.
• Focused exact-key tests cover FLOAT/DOUBLE signed zero, distinct NaN payloads, null/const vectors, exact ordering, and companion-column permutation.
• Coverage-mode package tests confirm malformed serialized-key errors, compound/hidden fuzzy-check SQL modes, projected-type fallback, and the background exact-float duplicate decode path are exercised.
• make build — pass.
• git diff --check — pass.
• Public branch_float_special_values.sql regression on this exact head — 256/256 statements pass.
• On the equivalent production head before the test-only coverage commits: alter_table_change_column.sql 923/923 and alter_table_modify_column.sql 764/764 statements pass.

The public Data Branch regression covers FLOAT, DOUBLE, and composite keys; finite values, infinities, signed zero, and distinct NaN payloads; MERGE, PICK, portable SQL, and no-PK predicates; FAIL/SKIP/ACCEPT when source update conflicts with destination delete; and crossed independent updates/deletes on bit-distinct signed-zero and NaN keys. Exact hex(serial(...)) assertions verify both changed and untouched keys.

CI follow-up

The failed proxy BVT on head ad38897235 was caused by this PR: exact serialized FLOAT/DOUBLE DEDUP identities leaked into duplicate-entry messages in ALTER TABLE cases. This head decodes those identities for diagnostics and closes the same identity contract in fuzzy-filter collision recovery. That run's Coverage failure only propagated the producer failure.

The later Coverage-only failures were also PR-caused. Head 0570eb78b8 reported 178/240 (74.17%); after the first test-only follow-up, CI's merged profile on e91247f83a reported 175/240 (72.92%) because the background fuzzy-check path remained uncovered. The matching executor/result boundary test adds 14 covered statements, taking the observed 175/240 baseline above the strict 75% gate.

Unrelated Ubuntu/x86 failures remain tracked in #26583 and #26632 through the single machine-readable blocker comment; this PR contains no workaround for them.

Residual risk

#26432 still tracks the engine-wide resident/spill typed-key inconsistency. This PR does not change that general engine contract. Data Branch and simple FLOAT/DOUBLE primary-key constraint paths avoid raw float identity matching by using exact bit/serialized encodings. Non-float paths are unchanged.

[gouhongshen]

Unrelated CI blockers in the current Ubuntu UT run are tracked in #26583 (checkpoint rollback leaves an unpublished object) and #26632 (shared hash-build budget recovery cancels its query). Evidence from this PR's failed job has been added to both issues.

[XuPeng-SH]

Deep-reviewed updated exact head 8e852a000875b8811b6b6bfee06d837ae2d6e350. The exact storage-identity fix closes MERGE and portable-output paths, but one original-issue path still bypasses it.

P1 — FLOAT/DOUBLE primary-key updates still fail through DATA BRANCH PICK

mergeDiffs and portable output call appendBatchRowsAsSQLValues, which recognizes fromUpdate and emits the new exact-key UPDATE. pickMergeDiffs, however, calls appendPickedBatchRows; that function ignores fromUpdate and still queues the pair as staged DELETE plus replacement INSERT. This is precisely the delete/reinsert cycle that the PR correctly identifies as unsafe for FLOAT/DOUBLE keys.

I reproduced it through public SQL on a service built from this exact head. The base/source/destination contain four legal, bit-distinct DOUBLE primary keys: two NaN payloads and both signed zeros. The source changes only NaN payload 1 and negative zero, then:

DATA BRANCH PICK src INTO dst
KEYS(SELECT k FROM src WHERE note IN ('nan1_updated', 'negzero_updated'))
WHEN CONFLICT ACCEPT;

returns:

ERROR 1062: Duplicate entry '0' for key 'k'

The destination transaction rolls back and retains all four original rows. Ordinary MERGE on the same representations is covered by the PR, but PICK is explicitly part of #26072 and remains broken for a real FLOAT/DOUBLE key update.

Please route PICK same-key updates through exact-key UPDATE while preserving its FAIL/SKIP/ACCEPT conflict semantics. Add public PICK regressions with bit-distinct NaN payloads and signed zero, plus a composite float-key control; verify both updated and untouched encodings with serial(...).

[gouhongshen]

XuPeng-SH's PICK review is valid and in scope. Fixed on rebased head 74cef706ec.

PICK now uses the same shared exact FLOAT/DOUBLE update classification and row application as MERGE/portable output, but only after its existing FAIL/SKIP/ACCEPT decision. Marked delete rows still drive conflict handling and are then suppressed; accepted marked inserts execute an exact-key UPDATE through serial(...), so the key never enters the unsafe delete/reinsert cycle. Non-float paths are unchanged.

Public regressions cover FLOAT, DOUBLE, and composite FLOAT/DOUBLE/int PICK updates with two NaN payloads and both signed zeros. The untouched payload-0/+0 rows remain, only payload-1/-0 rows change, and hex(serial(...)) verifies every stored key. On the rebased head: full pkg/frontend 21.159s, vet, make build, diff checks, and the inspected 119/119 BVT pass. The PR description is updated.

[XuPeng-SH]

Deep-reviewed exact head 74cef706ec05471643afb24102edc3c034648e48. The previous PICK case with an unchanged destination is fixed, but the UPDATE-only application path introduces a silent-loss case when the destination no longer contains the source key.

P1 — ACCEPT loses a FLOAT/DOUBLE source update when the destination deleted the row

dataBranchExactFloatKeyUpdateBatch classifies both halves of every same-key update on a FLOAT/DOUBLE-containing PK as an exact update. The consumer then suppresses the DELETE half, and appendOrExecuteDataBranchApplyRow turns the INSERT half into only:

UPDATE dst SET ... WHERE serial(pk) = serial(source_pk) LIMIT 1

If the destination branch independently deleted that row, the UPDATE affects zero rows. There is no INSERT fallback, so WHEN CONFLICT ACCEPT reports success but does not materialize the accepted source row.

I reproduced this through public SQL on a service built from this exact head:

CREATE TABLE base_t(k DOUBLE PRIMARY KEY, note VARCHAR(32));
INSERT INTO base_t VALUES (1.5, 'base');
DATA BRANCH CREATE TABLE src FROM base_t;
DATA BRANCH CREATE TABLE dst FROM base_t;
UPDATE src SET note = 'source_updated' WHERE k = 1.5;
DELETE FROM dst WHERE k = 1.5;

DATA BRANCH PICK src INTO dst
  KEYS(SELECT k FROM src WHERE note = 'source_updated')
  WHEN CONFLICT ACCEPT;
SELECT COUNT(*) FROM dst; -- 0, expected 1

The FAIL control correctly reports this pair as a conflict, proving ACCEPT is taking the wrong resolution rather than seeing a no-op. The same witness with an INT PK restores the source row as expected. I also reproduced the zero-row result for:

• FLOAT through PICK;
• DOUBLE through MERGE;
• a composite (FLOAT, INT) PK through MERGE.

The portable path shares the same UPDATE-only helper, so its generated script has the same missing-row state unless it is handled separately.

Please preserve exact float-key identity while closing the complete conflict state: after ACCEPT, the source row must exist whether the destination retained, modified, or deleted the ancestor row. Add public PICK and MERGE regressions for destination-delete versus source-update across FLOAT, DOUBLE, and a composite float key, with FAIL/SKIP controls; also round-trip the portable script and retain the existing NaN-payload/signed-zero identity assertions.

Fresh exact-head evidence: go build and go vet for pkg/frontend pass; the full pkg/frontend suite passes; the focused float-key regressions pass under -race -count=3. Those tests currently cover only the destination-retained branch of this state machine.

[gouhongshen]

XuPeng-SH's destination-delete review is valid and in scope. Fixed on rebased head ae748dc651.\n\nExact FLOAT/DOUBLE-key materialization is now policy-aware at the shared apply boundary: retained rows always use exact serial(...) UPDATE; source-winning ACCEPT and portable output additionally issue an exact-key conditional INSERT when the destination independently deleted the row; SKIP deliberately performs no insertion and preserves that deletion. PICK keeps its FAIL/SKIP/ACCEPT decision ahead of the shared apply path.\n\nPublic regressions cover FLOAT PICK FAIL/SKIP/ACCEPT, DOUBLE MERGE FAIL/SKIP/ACCEPT, composite FLOAT/int MERGE ACCEPT, and a portable DOUBLE round trip. The prior NaN-payload/signed-zero assertions remain. On the rebased head, full pkg/frontend (21.141s), vet, make build, diff checks, and the inspected 171/171 BVT pass. The PR description is updated.

[XuPeng-SH]

Blocking correctness issue in pkg/frontend/data_branch_output.go:2055: the new restore path uses INSERT ... SELECT, even though lines 232-236 document that INSERT ... SELECT collapses bit-distinct FLOAT/DOUBLE keys. Reproduced on exact head ae748dc: independently insert +0 and -0 into a DOUBLE PK table; update the exact -0 row on the source branch; delete exact -0 on the destination while +0 remains; then MERGE ... WHEN CONFLICT ACCEPT fails with Error 1062: Duplicate entry -0. PICK ... WHEN CONFLICT ACCEPT fails identically. The serial-based NOT EXISTS sees -0 as missing, but the INSERT ... SELECT path then compares with scalar float semantics and collides with +0. Portable output emits the same statement. Please restore missing exact-key updates through a path that preserves independent single-row INSERT semantics, and add combined missing-key/bit-distinct-peer regressions (+0/-0 and distinct NaN payloads) for MERGE, PICK, and portable output.

[gouhongshen]

XuPeng-SH's review is valid and in scope. Fixed on rebased head ad38897235.

The review correctly identified that INSERT ... SELECT cannot preserve an independently missing FLOAT/DOUBLE key beside a bit-distinct peer. Self-checking found a deeper constraint: replacing it with INSERT ... VALUES alone still failed because the primary-key DEDUP plan joined raw floats, so +0/-0 and distinct NaN payloads were still collapsed.

The clean fix closes both boundaries:

• The shared diff state machine now marks only source-update/destination-delete conflicts as missing. Retained rows use exact-serial(...) UPDATE; source-winning ACCEPT and portable output restore the proven-missing row with one direct INSERT ... VALUES; SKIP preserves the deletion. The apply layer no longer guesses existence.
• Simple FLOAT/DOUBLE primary-key duplicate checks now compare serial(...) encodings in DEDUP joins, constraint probes, and intra-batch grouping. Non-float keys are unchanged.

Public regressions combine a missing key with its untouched bit-distinct peer for both +0/-0 and two NaN payloads across MERGE, PICK, and portable SQL. All four exact hex(serial(k)) identities remain and only the selected source row changes.

Verification on base acfb8ef77c: full pkg/frontend (20.226s), full pkg/sql/plan (10.393s), focused race test (3 runs), vet, make build, diff checks, and the inspected 212/212 BVT pass. The PR description is updated.

[gouhongshen]

The proxy BVT failure on ad38897 was valid and PR-caused. The exact FLOAT/DOUBLE identity expression was correctly serialized for DEDUP, but duplicate diagnostics printed the binary key, producing the ALTER TABLE expectation mismatch; Coverage only propagated that producer failure. Fixed on rebased head 0570eb7 by decoding exact keys for errors and closing the same projected-type/serial(pk) contract in fuzzy-filter collision recovery. Rebased onto current main bb011f2 with no conflicts. Controlled tests/vet, focused race x3, make build, diff checks, DDL BVT 923/923 + 764/764, and Data Branch BVT 212/212 pass. The PR description is rewritten and current.

[gouhongshen]

The Coverage failure on 0570eb78b8 is valid and PR-caused. All producer jobs passed, but changed-code coverage was 178/240 (74.17%), below the strict 75% gate. Fixed on rebased head e91247f83a with focused tests for malformed serialized-key decoding, compound/hidden fuzzy-check SQL modes, and projected identity-type fallback; runtime behavior is unchanged. On the final base, full frontend/plan/colexec/compile tests, vet, make build, coverage-mode package tests, and diff checks pass. The PR description is updated.

[gouhongshen]

The Coverage failure on e91247f83a is valid and PR-caused. All producer jobs passed; CI's merged profile reported 175/240 (72.92%). The earlier tests brought dedup_key.go and operator.go to 100%, but the real background fuzzy-check executor/result path remained uncovered. Fixed in 105457f602 with a boundary test that verifies decoded -0 duplicate diagnostics and malformed serialized identity errors, including result cleanup and runtime executor restoration. A matching coverage-mode run adds 14 covered statements (enough to take 175/240 above the strict 75% gate); the focused test, full pkg/sql/compile, vet, and diff checks pass. The PR description is updated.

[XuPeng-SH]

The previous INSERT ... SELECT blocker is fixed, but the hash-diff conflict join still collapses bit-distinct FLOAT/DOUBLE primary keys.

handleTarDeleteAndUpdates sorts each side by the raw PK vector (mergeutil.SortColumnsByIndex) and then treats compareSingleValInVector(...) == 0 as the same key. Both paths use scalar float ordering/comparison: +0 and -0 compare equal, and distinct NaN payloads compare equal. A minimal white-box witness on this head expects non-zero for compareSingleValInVector(+0, -0) but gets 0.

This leaves an uncovered cross-change shape. For example, start with both +0 and -0 as DOUBLE PKs; update -0 on the source while independently deleting +0 on the destination, then MERGE/PICK with conflict handling. These are different storage keys, but the merge join pairs them as a conflict. Under ACCEPT, lines 1188-1196 can mark the source -0 update as restoreMissing; the new direct INSERT then collides with the still-existing -0 row. FAIL/SKIP can likewise reject or skip a non-conflict. The same shape exists for two NaN payloads.

Please make hash-diff sorting and equality share one exact, bit-preserving key order (including composite keys), and add crossed-key regressions for +0/-0 and distinct NaN payloads across MERGE/PICK and FAIL/SKIP/ACCEPT. The current tests delete the same exact key that the source updates while merely retaining a bit-distinct peer, so they do not exercise this pairing bug.

[gouhongshen]

XuPeng-SH’s hash-diff review is valid and in scope. Fixed in 12db1fb854.

The conflict merge join now uses one shared exact key identity for both ordering and equality. Simple FLOAT/DOUBLE PKs are ordered by their raw IEEE bits, so signed zero and distinct NaN payloads remain separate storage keys; non-float keys retain the existing comparator, and composite PKs retain their existing serialized-byte order. Both sides are sorted through the same helper before the merge join, eliminating the former scalar-order mismatch.

Regression coverage now crosses independent changes: source updates -0 while destination deletes +0, and source updates NaN payload 1 while destination deletes payload 0. MERGE and PICK cover FAIL/SKIP/ACCEPT, with a composite control and exact hex(serial(...)) assertions. Focused tests, full pkg/frontend, controlled CGo vet, make build, diff checks, and the public 256/256 BVT pass. The PR description is updated.

[XuPeng-SH]

Deep-reviewed the updated exact head. The previous hash-diff blocker is closed.

The conflict matcher now derives a raw IEEE-bit identity for a simple FLOAT/DOUBLE PK, sorts both sides by that uint64 identity, and uses the identical comparison in the merge join. This distinguishes +0 from -0 and distinct NaN payloads while leaving non-float keys unchanged; composite keys continue to use their existing exact serialized-byte identity. The temporary identity vector is batch-scoped and freed after sorting, so the extra O(n), 8-byte/row allocation is limited to simple float-key conflict batches.

The new public regressions cover the previously missing crossed-change shape: source updates -0/NaN payload 1 while destination deletes +0/payload 0, across MERGE and PICK with FAIL/SKIP/ACCEPT, plus a composite control and exact serial(...) assertions. This closes the false-conflict and erroneous restore/duplicate path without weakening real conflict handling.

Fresh exact-head validation passed: diff check; go list, go build, go vet, and the full pkg/frontend suite; focused exact-identity/sort tests under -race -count=3. CI is green.

[mergify]

Merge Queue Status

• ✅ Entered queue — 2026-08-04 03:43 UTC · Rule: main · triggered by rule Automatic queue on approval for main
• ✅ Checks passed · in-place
• ✅ Merged — 2026-08-04 04:48 UTC · at 6f72fe538464441a1f50a65c1b713f0327dde405 · squash

This pull request spent 1 hour 4 minutes 39 seconds in the queue, including 1 hour 4 minutes 14 seconds running CI.

Required conditions to merge

• #review-threads-unresolved = 0 [🛡 GitHub branch protection]
  • fix(frontend): preserve non-finite data branch floats #26504
• github-review-approved [🛡 GitHub branch protection]
  • fix(frontend): preserve non-finite data branch floats #26504
• any of [🛡 GitHub branch protection]:
  • check-success = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
  • check-neutral = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
  • check-skipped = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
• any of [🛡 GitHub branch protection]:
  • check-success = Matrixone CI / UT Test on Ubuntu/x86
  • check-neutral = Matrixone CI / UT Test on Ubuntu/x86
  • check-skipped = Matrixone CI / UT Test on Ubuntu/x86
• any of [🛡 GitHub branch protection]:
  • check-success = Matrixone CI / SCA Test on Linux/arm64
  • check-neutral = Matrixone CI / SCA Test on Linux/arm64
  • check-skipped = Matrixone CI / SCA Test on Linux/arm64
• any of [🛡 GitHub branch protection]:
  • check-success = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
  • check-neutral = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
  • check-skipped = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
• any of [🛡 GitHub branch protection]:
  • check-success = Matrixone Utils CI / Coverage
  • check-neutral = Matrixone Utils CI / Coverage
  • check-skipped = Matrixone Utils CI / Coverage
• any of [🛡 GitHub branch protection]:
  • check-success = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
  • check-neutral = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
  • check-skipped = Matrixone UT Coverage / UT Coverage on Ubuntu/x86

[XuPeng-SH]

Revalidated the Mergify queue merge head 6f72fe538464. Its first parent is the reviewed/fixed head 12db1fb854; its second parent is current main 2190897330. The Data Branch, colexec, compile, plan, and float-special-value test patch is byte-identical to the approved parent. The only upstream delta from the first parent is pkg/logservice/service_bootstrap_test.go, and the merge result is exactly the current-main version—there is no conflict rewrite in this PR surface.

The exact float identity/sort tests pass on this merge head, and git diff --check is clean. Approval stands.