fix(data-warehouse): accept qualified table names in postgres discovery#60257
Merged
Conversation
The multi-schema migration rewrites a row's `name` to `{schema}.{table}` in
place even when the source still has `config.schema` set to a specific
schema. Callers that pass a row's `name` back into `get_schemas(...,
names=[...])` — for example the `incremental_fields` API endpoint that
powers the sync-method config page — then hit an empty match because
`_get_discovered_tables` keys its lookup table unqualified when
`config.schema` is set. The endpoint returns 400 "Schema with name
public.foo not found".
Make `_get_discovered_tables` accept both qualified and unqualified
`names`. When a requested name has a `.` and isn't in the lookup table,
fall back to matching the unqualified suffix and return the entry under
the requested name so callers keep getting back what they asked for.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Reviews (1): Last reviewed commit: "fix(data-warehouse): accept qualified ta..." | Re-trigger Greptile |
Co-authored-by: Cursor <cursoragent@cursor.com>
estefaniarabadan
approved these changes
May 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
After the multi-schema migration qualifies a Postgres warehouse row's name in place (e.g. `tracking_link` → `public.tracking_link` via `_qualify_legacy_row`), the sync-method config page returns 400 "Schema with name public.foo not found".
Root cause: callers like the `incremental_fields` action in `ExternalDataSchemaViewset` pass the row's `name` straight back into the Postgres source via `get_schemas(config, ..., names=[instance.name])`. `_get_discovered_tables` keys its lookup dict unqualified whenever `config.schema` is set (`qualify_with_schema=False`), so the qualified name never matches. The sync workflow already self-heals this on the writer path by splitting the dotted name (`source_for_pipeline` in `postgres/source.py`), but the discovery path wasn't updated.
Changes
Make `_get_discovered_tables` accept both forms in the `names` filter. When a requested name has a `.` and isn't in the lookup table, fall back to matching the unqualified suffix and return the entry under the requested name so callers keep getting back exactly what they asked for. No behavioral change when names are already in the lookup table.
Parameterized test covers:
How did you test this code?
I'm an agent. I added a parameterized test in `test_postgres.py` covering the three lookup combinations and ran `hogli test posthog/temporal/data_imports/sources/postgres/test_postgres.py::TestPostgresSchemaDiscovery` — all 7 tests in that class pass. No manual UI testing.
Publish to changelog?
no
🤖 Agent context
Authored by Claude (Opus 4.7) in a debugging session. The user surfaced the 400 from the sync-method page after clicking "Pull new schemas" on a Postgres source. Database inspection showed the migration had qualified the row name and stored a `dwh_storage_key` to preserve the Delta path; the qualified name then failed the discovery name filter. Fixing the writer side was an option but the discovery layer is the actual mismatch point, and a layered fix at the source covers all downstream callers (`incremental_fields`, `refresh_schemas`, anything else that round-trips a row's `name`).