ENG-2635: Support namespaces for Postgres DSR execution

[JadeCara]

Ticket ENG-2635

Description Of Changes

Add namespace support for Postgres connectors, enabling a single connection to handle multiple schemas for DSR execution. This follows the established Snowflake/BigQuery pattern exactly.

Ramp has 200-300 datasets representing different Postgres schemas. Previously, each schema required a separate integration for DSRs. With namespace support, a single Postgres connection can execute DSRs across all schemas by using fully-qualified table names ("schema"."table" or "database"."schema"."table").

This also removes the stubbed retrieve_data()/mask_data() from RDSPostgresConnector, unblocking DSR execution for RDS Postgres connections.

PR 1 of 3 — Core Postgres + RDS Postgres namespace support. Google Cloud SQL Postgres (PR 2) and set_schema() reconciliation (PR 3) follow separately.

Code Changes

• src/fides/api/schemas/namespace_meta/postgres_namespace_meta.py — New PostgresNamespaceMeta schema with database_name (optional) and schema (required), auto-registered via NamespaceMeta.__init_subclass__. Returns empty fallback fields for backward compat (Postgres defaults to public schema).
• src/fides/api/service/connectors/query_configs/postgres_query_config.py — Added namespace_meta_schema, generate_table_name() for schema-qualified SQL, updated get_formatted_query_string() and get_update_stmt() to use it
• src/fides/api/service/connectors/postgres_connector.py — query_config() now fetches namespace_meta from DB, added get_qualified_table_name() for unquoted names in error handling
• src/fides/api/service/connectors/rds_postgres_connector.py — Removed stubbed retrieve_data()/mask_data() (inherits SQLConnector's implementations), query_config() passes namespace_meta, added get_qualified_table_name()
• src/fides/api/service/connectors/sql_connector.py — Added None guard in get_namespace_meta() for dry-run contexts where no DB session is available
• src/fides/service/dataset/validation_steps/namespace_meta.py — Added None guard for connection secrets; skip namespace validation when namespace_meta fields don't overlap with the connection type's schema (cross-type dataset linking)
• clients/admin-ui/src/features/integrations/integration-type-info/rdsPostgresInfo.tsx — Added "DSR Automation" to RDS Postgres tags
• tests/ops/service/connectors/test_postgres_query_config.py — New tests mirroring Snowflake test structure: parametrized namespace query generation, invalid meta validation, namespaced update statements
• tests/service/dataset_service/test_namespace_meta_validation.py — Updated for Postgres namespace support: 5 new Postgres-specific tests, cross-type namespace skip test, updated unsupported type test

Steps to Confirm

1. Run namespace query config tests

pytest tests/ops/service/connectors/test_postgres_query_config.py -v

Expected: All 9 tests pass (5 parametrized query generation, 1 invalid meta, 3 update statements).

2. Run namespace validation tests

pytest tests/service/dataset_service/test_namespace_meta_validation.py -v

Expected: All 15 tests pass, including 5 Postgres-specific and 1 cross-type namespace skip test.

3. Verify backward compatibility (no namespace = no breakage)

pytest tests/ops/service/connectors/test_queryconfig.py -v

Expected: Existing Postgres query config tests pass unchanged — datasets without namespace_meta produce unqualified table names as before.

4. Verify dataset creation with namespace_meta via API

Create a Postgres connection, then create a dataset with namespace_meta, and link it:

# Create connection
curl -X PATCH "http://localhost:8080/api/v1/connection" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '[{"name":"Test Namespace PG","key":"test_ns_pg","connection_type":"postgres","access":"read"}]'

# Create dataset with namespace_meta
curl -X POST "http://localhost:8080/api/v1/dataset/upsert" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '[{
    "fides_key": "ns_test",
    "name": "Namespace Test",
    "collections": [{"name": "orders", "fields": [
      {"name": "id", "fides_meta": {"primary_key": true, "data_type": "integer"}},
      {"name": "email", "fides_meta": {"data_type": "string", "identity": "email"}}
    ]}],
    "fides_meta": {"namespace": {"schema": "billing"}}
  }]'

# Link dataset config to connection
curl -X PATCH "http://localhost:8080/api/v1/connection/test_ns_pg/datasetconfig" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '[{"fides_key": "ns_test", "ctl_dataset_fides_key": "ns_test"}]'

Expected: All return 200/201. The datasetconfig response shows "succeeded": [...] with 1 entry.

5. Verify namespace_meta is persisted and returned

curl -X GET "http://localhost:8080/api/v1/connection/test_ns_pg/datasetconfig" \
  -H "Authorization: Bearer $TOKEN"

Expected: The response includes "fides_meta": {"namespace": {"schema": "billing"}} in the ctl_dataset of the linked dataset config.

6. Verify backward compat via API (no namespace_meta required for Postgres)

# Create and link a dataset WITHOUT namespace_meta to the same Postgres connection
curl -X POST "http://localhost:8080/api/v1/dataset/upsert" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '[{
    "fides_key": "no_ns_test",
    "name": "No Namespace",
    "collections": [{"name": "users", "fields": [
      {"name": "id", "fides_meta": {"primary_key": true, "data_type": "integer"}}
    ]}]
  }]'

curl -X PATCH "http://localhost:8080/api/v1/connection/test_ns_pg/datasetconfig" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '[{"fides_key": "no_ns_test", "ctl_dataset_fides_key": "no_ns_test"}]'

Expected: Both return 200/201 — no validation error. Postgres connections do NOT require namespace_meta (backward compatible, defaults to public schema).

7. Check RDS Postgres Admin UI tag

Navigate to the Integrations page in the Admin UI and find "RDS Postgres".

Expected: RDS Postgres shows "DSR Automation", "Discovery", and "Detection" tags.

Pre-Merge Checklist

• Issue requirements met
• All CI pipelines succeeded
• CHANGELOG.md updated
  • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
  • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
  • Updates unreleased work already in Changelog, no new entry necessary
• UX feedback:
  • All UX related changes have been reviewed by a designer
  • No UX review needed
• Followup issues:
  • Followup issues created
  • No followup issues
• Database migrations:
  • Ensure that your downrev is up to date with the latest revision on main
  • Ensure that your downgrade() migration is correct and works
    • If a downgrade migration is not possible for this change, please call this out in the PR description!
  • No migrations
• Documentation:
  • Documentation complete, PR opened in fidesdocs
  • Documentation issue created in fidesdocs
  • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
  • No documentation updates required

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
fides-plus-nightly	Ignored	Preview	Feb 26, 2026 6:09pm
fides-privacy-center	Ignored		Feb 26, 2026 6:09pm

[greptile-apps]

Greptile Summary

This PR adds namespace support for Postgres and RDS Postgres connectors, enabling a single connection to handle multiple schemas for DSR execution. The implementation follows the established Snowflake/BigQuery pattern exactly.

Key Changes:

• Created PostgresNamespaceMeta schema with database_name (optional) and schema (required) fields
• Updated PostgresQueryConfig to generate schema-qualified table names ("schema"."table" or "database"."schema"."table")
• Modified Postgres and RDS Postgres connectors to fetch and use namespace metadata
• Added get_qualified_table_name() methods for proper table existence checks
• Removed stubbed DSR methods from RDSPostgresConnector, unblocking DSR execution
• Enhanced validation to skip cross-type namespace metadata (e.g., BigQuery namespace on Postgres connection)
• Added comprehensive test coverage mirroring Snowflake tests

Backward Compatibility:

• Postgres connections without namespace_meta default to public schema
• PostgresNamespaceMeta.get_fallback_secret_fields() returns empty set for backward compatibility
• Existing tests pass unchanged

Impact:
This enables Ramp to use a single Postgres connection for 200-300 schemas instead of requiring separate integrations for each schema.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The implementation closely follows the proven Snowflake/BigQuery pattern. Comprehensive test coverage includes 9 query config tests and 5 Postgres-specific validation tests. Backward compatibility is maintained through empty fallback fields and existing tests passing unchanged. The changes are additive and well-isolated to Postgres-specific code paths.
• No files require special attention

Important Files Changed

Filename	Overview
src/fides/api/schemas/namespace_meta/postgres_namespace_meta.py	New PostgresNamespaceMeta schema with optional database_name and required schema field, following Snowflake pattern exactly. Auto-registers via init_subclass. Returns empty fallback fields for backward compatibility.
src/fides/api/service/connectors/query_configs/postgres_query_config.py	Added namespace_meta_schema and generate_table_name() for schema-qualified SQL. Updated get_formatted_query_string() and get_update_stmt() to use namespaced table names.
src/fides/api/service/connectors/postgres_connector.py	Updated query_config() to fetch namespace_meta from DB. Added get_qualified_table_name() returning unquoted names for table_exists() checks.
src/fides/api/service/connectors/rds_postgres_connector.py	Removed stubbed retrieve_data()/mask_data() to enable DSR execution. Updated query_config() to pass namespace_meta for SQL generation. Added get_qualified_table_name().
src/fides/service/dataset/validation_steps/namespace_meta.py	Added None guard for connection secrets. Implemented cross-type namespace skip logic to handle mixed dataset types linked to single connection.
tests/ops/service/connectors/test_postgres_query_config.py	Comprehensive test suite with 9 tests: 5 parametrized query generation tests, 1 invalid metadata test, 3 update statement tests covering namespaced and non-namespaced cases.

_{Last reviewed commit: 8b9718d}

[greptile-apps]

_{10 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}