fix(data-warehouse): fail early on colliding Google Sheets headers - #71982
Merged
Gilbert09 merged 2 commits intoJul 17, 2026
Conversation
Sheet headers become table column names via NamingConvention, which is case- and punctuation-insensitive. gspread only rejects exact duplicate headers, so headers that differ only by case or punctuation (e.g. "Task ID" vs "task_id") slip through and fail much later with an opaque "duplicate column name" error deep in table creation, after the sync has already started. Detect the collision when the header row is first read and raise an actionable message naming the offending headers, and map it as non-retryable so the deterministic failure isn't retried forever. Generated-By: PostHog Code Task-Id: cade8cc7-2ba9-4e90-8fb2-f130f0b98aaf
Contributor
|
Hey @Gilbert09! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
…lision Generated-By: PostHog Code Task-Id: cade8cc7-2ba9-4e90-8fb2-f130f0b98aaf
![stamphog[bot]](https://avatars.githubusercontent.com/in/3000332?s=60&v=4){kind=small}
There was a problem hiding this comment.
Small, contained fix to Google Sheets warehouse-source header validation with parameterized regression tests, authored by a member of the owning team; no data model, API contract, or ingestion-pipeline risk.
- Author wrote 0% of the modified lines and has 8 merged PRs in these paths (familiarity MODERATE).
- 👍 on the PR from hex-security-app[bot].
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 30L, 2F substantive, 64L/3F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1b-small (64L, 3F, single-area, fix) |
| stamphog 2.0.0b3 | .stamphog/policy.yml @ d080cf4 · reviewed head 0ef0a78 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When onboarding a Google Sheets data-warehouse source, sheet headers become table
column names via
NamingConvention, which is case- and punctuation-insensitive. Sotwo headers that look distinct in the sheet — for example
Task IDandtask_id—collapse to the same column name.
gspread only rejects exact duplicate headers, so these near-duplicates pass its
check and the failure surfaces much later as an opaque
duplicate column nameerrordeep in table creation, after the sync has already started. Because the failure is
deterministic, retries never recover, and the source silently stays in an error
state. Replay Vision surfaced a real onboarding session where a user got stuck in a
troubleshooting loop on exactly this: distinct-looking headers that collided after
normalization, with no message pointing at the cause.
Changes
google_sheets.py) andraise an actionable message naming the two offending headers and the column name
they collapse to. Exact duplicates are still left to gspread's own check.
get_non_retryable_errorsso thisdeterministic failure isn't retried forever.
No change to sync behavior for sheets with unique headers, and no change to which
sheets succeed — a sheet that would have failed cryptically now fails immediately
with a clear message.
How did you test this code?
Added parameterized unit tests in
test_google_sheets.py(pure-function, no DB):test_assert_unique_normalized_column_names_raises_on_normalized_collision—covers case-only, punctuation-only, and mixed collisions, and asserts the raised
message matches a non-retryable key (guards against a revert of the detection or
a message that no longer matches a key, which would retry the failure forever).
test_assert_unique_normalized_column_names_allows_valid_headers— distinctheaders, blank cells, and exact duplicates (left to gspread) do not raise.
Ran the affected tests locally with pytest (9 passed) and
ruff check/ruff format --checkon the touched files (clean). No manual end-to-end sync run.🤖 Agent context
Autonomy: Fully autonomous
Authored by Claude Code (PostHog Code) while reviewing data-warehouse new-source
onboarding sessions for recurring friction. Invoked the
/writing-testsskillbefore adding the regression tests to run them through the value gate. Chose an
early, clear hard-error over silently auto-renaming colliding columns because it
matches the Google Sheets source's existing behavior (it already rejects duplicate
headers and asks the user to fix them) and avoids changing what data lands.
Created with PostHog Code