Skip to content

feat(builder): union columns + DEFAULT for mixed-shape InsertRecords#54

Open
klaidliadon wants to merge 1 commit into
masterfrom
fix-insertrecords-mixed-shape
Open

feat(builder): union columns + DEFAULT for mixed-shape InsertRecords#54
klaidliadon wants to merge 1 commit into
masterfrom
fix-insertrecords-mixed-shape

Conversation

@klaidliadon
Copy link
Copy Markdown
Contributor

#50 rejected any batch whose rows produced different Map column sets — ,omitzero slice / ,omitempty map records that mix nil and non-nil empty values hit this constantly, forcing callers to split into per-row InsertRecord calls. Closes #53.

records := []*Order{
    {Name: "first"},                              // cols = [name]
    {Name: "second", Tags: []string{"a", "b"}},   // cols = [name, tags]
    {Name: "third",  Note: &note},                // cols = [name, note]
}
DB.SQL.InsertRecords(records, "orders")
// → INSERT INTO orders (name,note,tags) VALUES
//   ($1,DEFAULT,DEFAULT),($2,DEFAULT,$3),($4,$5,DEFAULT)

PostgreSQL accepts DEFAULT in any VALUES position. The fix: union the cols across rows, emit sq.Expr("DEFAULT") for any slot a row skipped.


Design

Drafted via three Codex review rounds. Plan at tmp/insertrecords-mixed-shape/2026-06-02-plan.md (branch-only).

Key inflection points the reviews forced:

  1. v1 had per-row empty-record rejection — kept feat(mapper): support ,omitzero tag option #50's behavior of erroring when any individual row mapped to zero cols. Codex blocker: that's now arbitrary. If another row contributes the union, a zero-column row is representable as (DEFAULT, DEFAULT, ...). Only the whole-batch empty union should error.
  2. v1 padded by slice position — fragile. v2 pads by column name via map[string]any per row.
  3. v2 hinted at SQL.InsertDefaults in the whole-batch-empty error — but that API is on the unmerged feat(builder): InsertDefaults for INSERT ... DEFAULT VALUES #52 branch. v3 hints at sq.Expr (matches feat(mapper): support ,omitzero tag option #50's existing single-row hint) so InsertRecords: union columns + emit DEFAULT for missing, instead of rejecting mixed-shape batches #53 stays independent of feat(builder): InsertDefaults for INSERT ... DEFAULT VALUES #52. Whichever lands second updates its own hint.

Reuses sqlDefault = sq.Expr("DEFAULT") from mapper.go:26. slices.Sorted(maps.Keys(colSet)) matches Map's lexical column order at mapper.go:161, so generated SQL lines up with what callers see from Map(record) directly.

Behavior table

Batch Before (#50) After (#53)
All rows same shape, non-empty works works (unchanged)
Mixed shapes, all rows non-empty rejected (drift) unions cols + DEFAULT fills
Some rows empty, some non-empty rejected (drift + empty) empty rows become all-DEFAULT
All rows empty rejected (per-row empty) rejected (whole-batch empty)
Empty slice of records rejected (existing) rejected (existing)

Test plan

  • make db-reset test-all against PostgreSQL 18.3 — all 6 packages green.
  • 7 unit tests: uniform shape (with args assertion), mixed shape with DEFAULT slots in the right places, ,omitzero mixed slices, ,omitempty mixed maps, empty row mixed with non-empty, all-rows-empty rejection, heterogeneous map[string]any records.
  • Integration TestInsertRecordsMixedShapeRoundTrip: exec a three-row heterogeneous batch against a new dedicated mixed_shape table (nullable + defaulted columns), verify each row landed with the right mix of caller values / DB defaults / NULLs.
  • go vet ./... clean.

Notes

  • ,omitempty semantics preserved: rows where ,omitempty skipped a column (e.g. []string{} on a slice field) get DEFAULT in the union slot, not the empty literal. The user tagged the column for "use DB default when missing" — that contract still holds, just now inside a batch.
  • Hint coordination: if #52 lands first, this PR's whole-batch-empty error could be updated to point at SQL.InsertDefaults instead of sq.Expr. Independent edit, no merge ordering required.

Closes #53. Follow-up to #50.

#50 rejected any batch whose rows produced different Map column sets,
with a build-time "record N columns differ from record 0" error. That
was the conservative answer: it stopped squirrel from emitting
malformed multi-row SQL where row widths didn't match.

But it's restrictive. Realistic ,omitzero (#50) and legacy ,omitempty
on map fields produce heterogeneous batches constantly. Forcing
callers to split into per-row InsertRecord calls kills the point of
batching.

Replace the drift check with union-by-name: walk rows once to compute
the column union and a per-row map[string]any of present columns,
then emit Columns(allCols...) with each row padded to the union via
sq.Expr("DEFAULT") in any slot the row skipped. PostgreSQL accepts
DEFAULT in any VALUES position, so the resulting SQL is valid for
every shape the union covers.

The per-row "empty record" rejection from #50 also goes away — a row
with no own columns can be all-DEFAULT *precisely because* another
row contributes the union. Only the whole-batch empty case still
errors, with a hint pointing at sq.Expr (matches the existing #50
hint shape on the InsertRecord single-row path).

Tests: rewrote the drift-rejection tests to assert union+DEFAULT SQL
including args; added empty-row-mixed-with-non-empty, heterogeneous
map records, and a real PG round-trip against a new mixed_shape table
that proves each row lands with the right mix of caller values, DB
defaults, and NULLs.

Planned via three Codex review rounds (per-row vs whole-batch reject
flipped, map-padding by name vs positional, test plan gaps, sq.Expr
hint vs unmerged #52 reference). Plan at
tmp/insertrecords-mixed-shape/2026-06-02-plan.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

InsertRecords: union columns + emit DEFAULT for missing, instead of rejecting mixed-shape batches

1 participant