Skip to content

fix(parity): drive semseg goldens with the CLI's mask_id schema (di#358)#322

Merged
LukasWodka merged 1 commit into
developfrom
fix/parity-semseg-mask-id-schema
Jul 14, 2026
Merged

fix(parity): drive semseg goldens with the CLI's mask_id schema (di#358)#322
LukasWodka merged 1 commit into
developfrom
fix/parity-semseg-mask-id-schema

Conversation

@LukasWodka

@LukasWodka LukasWodka commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes the red "Validator goldens vs data-ingestors @ pin" check that appeared on develop after cli#286 bumped the pin past data-ingestors #358.

Root cause — a harness bug, not the CLI or the validator

di#358's MaskIdColumnValidator requires mask_id to be a declared schema column (an undeclared mask_id is dropped at ingest → stored table lacks it → the training client raises FileNotFoundError, backend#816). The CLI's real semseg ingest does declare it (spec.go: schema={"mask_id":"VARCHAR(255)"}) — but gen-validator-goldens.py only injected a schema for tabular/time-series cases, so it drove the validators without the schema the CLI sends and spuriously rejected every semseg case (including semseg-ok).

Fix

  1. gen-validator-goldens.py: mirror spec.go — inject {"mask_id":"VARCHAR(255)"} for semantic_segmentation cases (an explicit per-case schema still wins, so reject-path cases keep exercising the contract).
  2. semseg-missing-mask-id: flip ingestor_verdict accept→reject — it has no mask_id column, so post-di#358 the ingestor correctly rejects it. This is exactly the reconciliation the case's own note predicted at pin-bump time; CLI and ingestor now agree.

Verification

  • sync-validator-goldens.sh --checkvalidator goldens in sync.
  • Goldens diff is one entry (semseg-missing-mask-id: accept→reject, with the real reject message now populated). semseg-ok was already accept — the harness just stops falsely flagging it.
  • go test ./internal/push/... green (incl. the cli_verdict ↔ ingestor_verdict ↔ goldens cross-check). Ran locally against the pinned data-ingestors (semseg validator identical between the pin and current DI HEAD).

cc @divyasinghds — this is the di#358 (backend#816) contract you enforced in the validator; flagging in case you want to eyeball the reconciliation.

🤖 Generated with Claude Code


Note

Low Risk
Test harness and generated parity fixtures only; no production ingest or CLI preflight logic changes in this diff.

Overview
Aligns the validator-parity harness with how the CLI actually ingests semantic segmentation: after the data-ingestors pin passed di#358, goldens were out of sync because the Python golden generator did not pass the same mask_id schema the CLI sends.

scripts/gen-validator-goldens.py now sets schema / full_schema to {"mask_id": "VARCHAR(255)"} for semantic_segmentation cases (matching spec.go), while an explicit per-case schema still overrides—so happy paths like semseg-ok are no longer spuriously rejected in goldens.

Parity fixtures update semseg-missing-mask-id: ingestor_verdict flips from accept to reject, with an updated note that CLI and pinned ingestor now both enforce the MaskIdColumnValidator contract. goldens.json reflects the real reject message for that case.

Reviewed by Cursor Bugbot for commit cb7134d. Bugbot is set up for automated code reviews on this repo. Configure here.

The goldens-drift check went red on develop once cli#286 bumped the
data-ingestors pin past di#358, whose MaskIdColumnValidator now REQUIRES
mask_id to be a *declared schema column* (an undeclared mask_id is dropped
at ingest -> stored table lacks it -> the training client raises
FileNotFoundError, backend#816).

Root cause was in the goldens harness, not the CLI or the ingestor: the
CLI's real semseg ingest declares the mask_id column (spec.go sets
schema={'mask_id':'VARCHAR(255)'}), but gen-validator-goldens.py only
injected a schema for tabular/time-series categories. So it drove the
validators WITHOUT the schema the CLI sends, and MaskIdColumnValidator
spuriously rejected every semseg case (incl. semseg-ok). Fix: mirror
spec.go for semantic_segmentation cases (an explicit per-case schema still
wins, so the reject-path cases keep exercising the contract).

Also flips semseg-missing-mask-id's ingestor_verdict accept->reject: it has
no mask_id column, so post-di#358 the ingestor correctly rejects it (both
sides now agree). This is exactly the reconciliation the case's own note
predicted at pin-bump time. Regenerated goldens; only that one entry
changes. 'validator goldens in sync'; internal/push parity tests green.

Part of tracebloc/backend#1106 (WS-C validator mirrors).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@LukasWodka LukasWodka self-assigned this Jul 14, 2026
@LukasWodka
LukasWodka requested a review from saadqbal July 14, 2026 17:06
@LukasWodka

Copy link
Copy Markdown
Contributor Author

@BugBot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit cb7134d. Configure here.

@LukasWodka

Copy link
Copy Markdown
Contributor Author

👋 Heads-up — Code review queue is at 49 / 30

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)

@LukasWodka
LukasWodka merged commit 449d6d7 into develop Jul 14, 2026
25 checks passed
@LukasWodka
LukasWodka deleted the fix/parity-semseg-mask-id-schema branch July 14, 2026 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants