fix(data-warehouse): match the id primary-key fallback case-insensitively - #71064
Merged
Conversation
There was a problem hiding this comment.
Small, well-tested, contained fix to a case-sensitivity bug in primary-key fallback logic; diff matches description, author is on the owning team, and no risky territory (no schema/API/auth/billing/CI surface) is touched.
- Author wrote 0% of the modified lines and has 8 merged PRs in these paths (familiarity MODERATE).
- 👍 on the PR from hex-security-app[bot].
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 31L, 3F substantive, 56L/5F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1b-small (56L, 5F, single-area, fix) |
| stamphog 2.0.0b3 | .stamphog/policy.yml @ c8f67df · reviewed head c8e8d9f |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A Snowflake import with a table that has an
IDcolumn failed incremental syncs with "Primary key required for incremental syncs", and the UI didn't preselect the primary key at setup.Snowflake uppercases unquoted identifiers, so the column surfaces as
IDeverywhere.SHOW PRIMARY KEYSonly returns declared PK constraints — which Snowflake doesn't enforce and most tables don't have — so detection falls through to theid-column name fallback. That fallback compared against the literal lowercase string"id"in all three places it exists (SQL-source discovery ×2 and the sync-timeresolve_primary_keyslast resort), soIDmissed every one, no key was resolved, and the keyless-table guardrail fired. This affects any Snowflake table without a declared PK constraint, and any other source reporting uppercase column names.Changes
All three fallbacks now match the
idcolumn case-insensitively and return the column's actual stored casing (["ID"], not a hardcoded["id"]):sources/common/sql/types.py—resolve_detected_primary_keys(Postgres / MySQL / MSSQL / Redshift discovery)sources/common/sql/base.py—SQLSource._default_primary_key_from_columns(generic SQL discovery)pipelines/common/extract.py—resolve_primary_keys(sync-time last resort)Returning the actual casing matters: the merge indexes extracted batches by the real column name, so returning a lowercase
"id"for anIDcolumn would just move the same failure into the merge.How did you test this code?
Automated tests I (Claude) ran locally, all green:
test_types.pyforresolve_detected_primary_keys(parameterized: discovered-wins, lowercase fallback, uppercaseIDreturning actual casing, no-id → None) — no test covered this helper at all; the uppercase case is the production regression.TestResolvePrimaryKeysprecedence table:{"name": "ID"}resolves to["ID"]— catches both a revert to case-sensitive matching and a change to returning hardcoded lowercase.test_extract.py+sources/common/sql/tests/suites: 365 passed._default_primary_key_from_columnscarries the identical three-line change and is exercised transitively through the SQL-source discovery suites; a third copy of the same parameterized matrix wouldn't catch anything new.Automatic notifications
Docs update
No behavior documented externally changes — PK detection just works for uppercase
IDcolumns now. Affected users can also self-serve today by setting the primary key manually in the table's sync settings (user overrides take precedence).🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Claude Code session, directed by Tom, from a user report of a Snowflake
ID(typetext) column not being detected. Investigation confirmed the column type is irrelevant (the fallback only looks at names) and the uppercase identifier is the whole story. Skill invoked: /writing-tests.