fix(postgres): surface leaked transaction nesting as retryable during setup

[Gilbert09]

Problem

Error tracking surfaced a Postgres data-warehouse import failing with:

ProgrammingError: Explicit commit() forbidden within a Transaction context. (Transaction will be automatically committed on successful exit from context.)

The real stack originates in the Postgres source's metadata-discovery setup (postgres_source → source_for_pipeline), and the failing operation is the implicit commit() that psycopg runs when the with connection: setup block exits successfully. The connection in the captured event is alive but INERROR, not broken.

Root cause: during setup we run best-effort probes, one of which isolates its query in a connection.transaction() block. psycopg increments a transaction-nesting counter on enter and only decrements it on exit when the connection is still OK. A connection dropped precisely while entering or leaving that block can leave the counter incremented without flipping connection.broken (the connection reads INERROR, not BAD). psycopg's commit() refuses outright whenever that counter is non-zero, so the exit-commit raises the misleading ProgrammingError and buries the real, transient cause.

There is already a guard (_raise_if_setup_connection_broken) meant to convert exactly this masked error into a retryable dropped-connection error. It only checked connection.broken, so it missed the not-broken-but-counter-leaked case.

Error tracking issue: https://us.posthog.com/project/2/error_tracking/019f5687-17a3-7e81-b64c-07904c6b4fca

Changes

Broaden _raise_if_setup_connection_broken to fire on either signal: a broken connection or a leaked transaction-nesting counter (_num_transactions > 0). The counter is the exact predicate psycopg's commit() checks, so this preempts the misleading error precisely.

The raised error is the same retryable OperationalError the broken-connection path already used, so it's classified as a transient drop (matches _is_connection_dropped_error, matches no NonRetryableErrors substring) and the activity retries on a fresh connection instead of failing on a self-inflicted commit error.

Note

This is deliberately kept retryable. A statement error inside the probe (e.g. a statement_timeout) tears its transaction down cleanly and does not leak the counter, so this change does not turn genuine query failures into infinite retries — the counter only leaks on a connection-level problem mid-probe, which is transient.

How did you test this code?

Automated only. I (Claude) added a regression test and ran the guard's unit tests locally with the repo venv:

python -m pytest products/warehouse_sources/backend/temporal/data_imports/sources/postgres/test_postgres.py::TestRaiseIfSetupConnectionBroken -q
3 passed

New test test_leaked_transaction_counter_raises_retryable_dropped_error reproduces the production state — a non-broken connection (broken = False) with a leaked counter (_num_transactions = 1) — and asserts the guard raises a retryable error not matched by any NonRetryableErrors substring. No existing test covered the not-broken-but-counter-leaked path; the existing tests only covered broken = True and a fully healthy connection. ruff check and format are clean.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

🤖 Agent context

Autonomy: Fully autonomous

Triaged from an error-tracking webhook. I (Claude) pulled the issue's stack trace and a representative event via the PostHog MCP error-tracking tools, confirmed the failure genuinely originates in the Postgres source setup (not just serialized log context), and read psycopg's transaction.py / _connection_base.py to pin down that commit() gates purely on the transaction-nesting counter and that Transaction.__exit__ skips the decrement when the connection is not OK.

Decision: this is a fragile-code bug, not a user/upstream error, so I fixed the code rather than extending NonRetryableErrors. The masked ProgrammingError hides a transient dropped-connection condition that should stay retryable, so the fix surfaces the real retryable error rather than suppressing it. Scope kept to the existing guard; no retry-policy or unrelated changes.

Skills invoked: /writing-tests (before adding the regression test).

---

Created with PostHog Code

[github-actions]

Hey @Gilbert09! 👋

It looks like your git author email on this PR isn't your @posthog.com address (owerstom@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

• Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
• Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

[greptile-apps]

_{Reviews (1): Last reviewed commit: "fix(postgres): surface leaked transactio..." | Re-trigger Greptile}

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[stamphog]

Small, well-tested fix to a retry-detection guard in the Postgres warehouse source, by the owning-team author with strong familiarity; no risky territory, no unresolved concerns, and a bot reviewer reaction is positive.

• Author wrote 100% of the modified lines and has 38 merged PRs in these paths (familiarity STRONG).
• 👍 on the PR from greptile-apps[bot], hex-security-app[bot].

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✓	no deny categories matched
size	✓	14L, 1F substantive, 33L/2F incl. docs/generated/snapshots — within ceiling
tier	✓	T1-agent / T1b-small (33L, 2F, single-area, fix)
stamphog 2.0.0b3		.stamphog/policy.yml @ dc71029 · reviewed head 551a5b0

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-07-13 11:21 UTC	Run
prod-us	✅ Deployed	2026-07-13 11:34 UTC	Run
prod-eu	✅ Deployed	2026-07-13 11:36 UTC	Run