feat(data-warehouse): add --only-stuck and duckgres sink to queue command

[estefaniarabadan]

Problem

The manage_warehouse_queue runbook command works well for controlling the pipeline v3 queue, but its fail-run subcommand fails every active sync in scope. When operating the queue you usually only want to clear out the runs that are actually misbehaving, not healthy in-flight syncs. The command also only knew about the Postgres queue sink, so duckgres-backed runs couldn't be managed the same way.

Changes

• fail-run --only-stuck: targets only potentially bad runs, meaning runs that have been going longer than --stuck-grace-seconds and have either no queue batches or only pending/stale batches. Runs with recent batch activity are skipped and reported. Since this filter is inherently safe, --only-stuck also lifts the scoping requirement, so it can sweep across teams in one pass (still dry-run by default).
• --sink duckgres: status, fail-run, and release-locks now manage the duckgres queue too. DuckgresBatchQueue gains sync ops helpers (pending batch lookup, batch failing, lease release) mirroring the postgres_queue interface. Flags that only make sense for one sink (for example --cancel-workflow and --redis-only with duckgres) are rejected with a clear error.
• Workflow cancellation fix: fail-run --cancel-workflow now opens one shared Temporal client connection for the batch, and the connect override flags (--temporal-host, certs, namespace) actually reach the client instead of being ignored.

How did you test this code?

Automated tests only, run locally:

• products/warehouse_sources/backend/tests/management/test_manage_warehouse_queue.py: 23 passed, including new coverage for --only-stuck (only inactive runs failed, unscoped use allowed), the duckgres sink (fail-run touches only pending duckgres batches, release-locks releases only duckgres leases, status sections), sink-specific flag validation, and a regression test asserting the Temporal connect overrides reach the client.
• duckgres and postgres_queue test_jobs_db.py suites: 83 passed.
• ruff check and ruff format --check clean.

No manual run against a live queue was performed by the agent.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

Internal operational command; no user-facing docs to update.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

• Authored with PostHog Code (Claude). Skills invoked: /writing-tests.
• The --only-stuck heuristic implements runbook feedback: a run is considered stuck when it exceeds a grace period with no batches or none progressing. We kept fail-run failing everything by default and made stuck-detection opt-in, rather than changing the default behavior under operators' feet.
• Duckgres support was added as a --sink switch on the existing command instead of a separate command, so the runbook stays one entry point. The duckgres helpers were shaped to mirror the postgres_queue jobs_db interface to keep the command's code paths symmetric.

---

Created with PostHog Code

[github-actions]

Hey @estefaniarabadan! 👋

It looks like your git author email on this PR isn't your @posthog.com address (estefania.rabadan@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

• Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
• Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

[greptile-apps]

Comments Outside Diff (1)

1. products/warehouse_sources/backend/tests/management/test_manage_warehouse_queue.py, line 1133-1157 (link)

   Test only verifies the first positional arg to connect

   The fix being tested (--temporal-host/--namespace/certs are forwarded) is that the override flags actually reach the connect call. The assertion only checks connect_mock.call_args.args[0] == "temporal-fe.internal" (the host). The port, namespace, and cert kwargs go unverified. If a future refactor accidentally dropped the other arguments, the test would not catch the regression. Consider also asserting the namespace, port, and/or cert kwargs — connect_mock.call_args.kwargs["namespace"], etc.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "feat(data-warehouse): add --only-stuck a..." | Re-trigger Greptile}

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[danielcarletti]

Approving with a small ask just to make sure

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-07-13 11:21 UTC	Run
prod-us	✅ Deployed	2026-07-13 11:34 UTC	Run
prod-eu	✅ Deployed	2026-07-13 11:36 UTC	Run