feat(data ingest): confirm the inferred tabular schema (#185)

[LukasWodka]

Summary

Under Option A (RFC-0002 §8.1), data-ingestors owns the tabular schema-inference rules (di#349) and the CLI mirrors them so the schema it EMITS is the same answer the ingestor would compute. This PR lands the CLI-side mirror: InferSchema infers a column → SQL type schema from the CSV, the CLI emits it explicitly as spec.schema, and a venv-free parity harness pins the two implementations to the same per-column answer.

Part of #185.

What changed

• internal/push/tabular.go — InferSchema / inferColumnType mirror di#349's rule precedence (first match wins): leading-zero code → BOOLEAN → INT/BIGINT → FLOAT → DATE/DATETIME → VARCHAR(n). VARCHAR(n) is sized by rune count (MySQL character semantics), integer/float matching is ASCII-only, and framework-managed columns are skipped. Empty / id-like columns are returned so the caller can surface them as warnings.
• internal/cli/data.go — when no --schema is given, infer here and emit a.Spec.Schema → spec.schema explicitly; warn on skipped / empty-in-sample / id-like columns.
• internal/push/schema_inference_parity_test.go + testdata/schema_inference_parity.json — the value-level parity contract, vendored byte-for-byte from di#349's fixture.

How it mirrors di#349 + the venv-free parity test

The ingestor's schema_inference.infer_column_type is the source of truth (Principle 6 / backend#1009). testdata/schema_inference_parity.json is vendored verbatim from di#349's tests/fixtures/schema_inference_parity.json — edit it once and both sides are pinned to the same answer. TestSchemaInferenceParity runs it as a static, venv-free check on every PR (no Python / pandas needed in CI), so drift between the Go mirror and the Python owner fails the build.

Explicit-emit rationale

The CLI does not rely on the deployed ingestor to re-infer. It emits spec.schema authoritatively, so the type each column lands as in-cluster is the answer the user saw and can override with --schema — independent of which ingestor version happens to be running. A mis-typed column is always correctable with --schema (the SAFE direction).

Schema ref deliberately NOT bumped

This PR does not touch the vendored ingest.v1.json schema ref. Emitting spec.schema uses the existing contract; nothing here changes the wire schema, so the pinned ref stays as-is (the drift tripwire remains green).

Review-finding fixes folded in

• Mixed-timezone DATETIME parity divergence (fixed). A column of RFC3339 values with non-uniform UTC offsets (or a mix of tz-aware and tz-naive) was typed DATETIME by the CLI but VARCHAR by the ingestor — the one divergence in the unsafe direction (per-row offset silently dropped in-cluster). inferDatetime now detects a non-uniform timezone and routes to VARCHAR, mirroring _infer_datetime's None result on mixed timezones. Pinned by TestInferColumnType_TimezoneParity (verified against di#349 / pandas 3.0.3). The ASCII-only shared fixture is left untouched — di#349 carries no tz case, so adding one would diverge the vendored contract.
• Dead code removed. SerializeSchema, the InferredColumn type, and SchemaInference.Columns were built for an interactive confirm/amend step that is not part of this PR and had zero callers/readers; removed, and the comments that described that step as if it existed are corrected to match the shipped behavior.
• Stale --schema help refreshed to the full inferred type set.
• VARCHAR-width and textual-BOOLEAN behavior kept as-is — both are faithful mirrors of di#349's owned rules (Principle 6); diverging the CLI from the source of truth would be its own parity bug.

Test plan

go build ./... && go test ./... && gofmt -l . (empty) && go vet ./... — all green locally. Parity + timezone subtests pass; existing tabular / interactive tests unchanged.

---

Hand-finished after the autonomous build stalled on the venv-goldens step; the type-inference mirror + parity harness were completed and the review findings addressed by hand.

---

Note

Medium Risk
Changes how tabular column types are chosen and sent on the wire; mistakes could mis-type data at load time, though parity tests and explicit --schema mitigate this, and VARCHAR fallbacks favor the safe direction.

Overview
Replaces the CLI’s simple INT/FLOAT/VARCHAR inference with a di#349-aligned mirror of data-ingestors’ schema_inference rules, so tabular ingests without --schema get the same column types the ingestor would compute.

InferSchema now returns a SchemaInference struct (schema map plus Skipped / Empty / IDLike warnings). Classification follows first-match-wins precedence: leading-zero codes → textual BOOLEAN → INT/BIGINT → FLOAT → DATE/DATETIME → VARCHAR(n) sized by rune count, with a mixed-timezone guard so non-uniform RFC3339 offsets fall back to VARCHAR instead of tz-naive DATETIME.

data ingest infers when --schema is omitted, emits spec.schema explicitly (authoritative in-cluster), and prints warnings for risky columns. TestSchemaInferenceParity plus vendored schema_inference_parity.json pin per-column types against the ingestor contract without Python in CI.

^{Reviewed by Cursor Bugbot for commit 4b88b49. Bugbot is set up for automated code reviews on this repo. Configure here.}

[saadqbal]

Nice, careful PR — parity discipline is solid and I couldn't find a correctness bug (verified the datetime/type/all-null cases against di#349 and the current ingestor). Two small things, neither blocking:

• varcharOf sizes VARCHAR(n) from the 5000-row sample, so we've lost the old fixed VARCHAR(255) floor. A string longer than the sampled max that only shows up past the cap gets an undersized column and truncates/errors in-cluster after the upload. This faithfully mirrors di#349 so I'm not asking to change the rule — just worth a line in the inferred-schema warning (or docs) that the width is sample-derived and --schema is the escape hatch.
• cleanTokens re-caps at schemaInferenceSampleRows, but InferSchema's read loop already bounds cols[i] to 5000, so that cap only ever fires for the test-only inferColumnType. Either drop it or move it to where it's actually load-bearing, so nobody reads it as guarding the real path.