fix: follow-ups to #75 (quote_ident, dlq partial-success, Nack tests)#79
Merged
fix: follow-ups to #75 (quote_ident, dlq partial-success, Nack tests)#79
Conversation
#75 landed three valuable fixes — Go consumer per-message nack, dlq_replay_all exception isolation, queue-existence check in insert_event_raw — but the SQL changes either lived in the wrong place or used patterns that don't survive the build. This PR addresses each. * sql/pgque-api/maint.sql — quote_ident(f.func_arg) is wrong: func_arg comes from pgque.maint_tables_to_vacuum() as "schema.table" (one string with a literal dot), so quote_ident wraps it as one identifier and produces vacuum "schema.table" instead of vacuum "schema"."table". Fix: split on the dot and quote each side via format('vacuum %I.%I', ...). The bug was invisible to tests because maint_tables_to_vacuum() returns nothing when autovacuum = on (the default). * sql/pgque-additions/dlq.sql — dlq_replay_all's per-event exception block used raise notice, which is hidden under most production configs (log_min_messages = warning by default). Two changes: switch to raise warning, and change the return type from bare integer to a record (replayed bigint, failed bigint, first_error text) so callers can detect partial success programmatically. Drop function if exists added before the create or replace because PG does not allow CREATE OR REPLACE to change a function's return type. docs/reference.md updated; no existing tests call dlq_replay_all. * clients/go/pgque_test.go — adds TestNack and TestConsumerHandlerNacksOnError covering the per-message nack behavior introduced in #75. Verifies a failing handler nacks its own message into pgque.retry_queue (not the DLQ, since retry_count starts below max_retries) without aborting the rest of the batch. * build/transform.sh — two of #75's source-level changes (upgrade_schema role grants flattened, insert_event_raw raises a clear "queue not found" exception) were applied directly to sql/pgque.sql, but pgque.sql is auto-generated from pgq/ + pgque-additions/ + pgque-api/ on each build. Re-applied them as post-transform patches in transform.sh so the next rebuild does not silently drop them. * sql/pgque.sql — regenerated. Verification runner reports all checks pass. Build pipeline note: anyone touching sql/pgque.sql should also touch the corresponding source under sql/pgque-additions/, sql/pgque-api/, or add a patch in build/transform.sh. The build script's self-verification catches schema-rename / search-path / queue_per_tx_limit drift but does not catch direct edits to the generated pgque.sql that lack a source. Co-Authored-By: Qiaochu Hu <110803307+hobostay@users.noreply.github.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
Owner
Author
REV ReviewCI: ✅ green (8/8 checks pass — test (14/15/16/17/18), verify, claude-review, client-smoke) Blockingnone Non-blocking
Potential
REV-style review (security, bugs, tests, guidelines, docs). SOC2 items skipped per project policy. Nothing blocking — non-blocking and potential items are good targets for v0.2.1. |
NikolayS
added a commit
that referenced
this pull request
Apr 30, 2026
client-smoke ran only TestSmoke-named tests and didn't set PGQUE_TEST_DSN, so env-gated integration tests silently skipped. That's how the Nack placeholder bug shipped green in PR #79. New client-tests job: - Sets PGQUE_TEST_DSN pointing at the docker PG. - Runs full Go suite: go test -race -v ./... - Runs full Python suite: pytest -v clients/python/tests/ - Runs full TypeScript suite: npx vitest --run (replaces tsx src/smoke.ts). Switching the TS step to vitest also unblocks PR #83, which deletes the hardcoded src/smoke.ts in favor of the vitest suite. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NikolayS
added a commit
that referenced
this pull request
Apr 30, 2026
client-smoke ran only TestSmoke-named tests and didn't set PGQUE_TEST_DSN, so env-gated integration tests silently skipped. That's how the Nack placeholder bug shipped green in PR #79. New client-tests job: - Sets PGQUE_TEST_DSN pointing at the docker PG. - Runs full Go suite: go test -race -v ./... - Runs full Python suite: pytest -v clients/python/tests/ - Runs full TypeScript suite: npx vitest --run (replaces tsx src/smoke.ts). Switching the TS step to vitest also unblocks PR #83, which deletes the hardcoded src/smoke.ts in favor of the vitest suite. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NikolayS
added a commit
that referenced
this pull request
Apr 30, 2026
client-smoke ran only TestSmoke-named tests and didn't set PGQUE_TEST_DSN, so env-gated integration tests silently skipped. That's how the Nack placeholder bug shipped green in PR #79. New client-tests job: - Sets PGQUE_TEST_DSN pointing at the docker PG. - Runs full Go suite: go test -race -v ./... - Runs full Python suite: pytest -v clients/python/tests/ - Runs full TypeScript suite: npx vitest --run (replaces tsx src/smoke.ts). Switching the TS step to vitest also unblocks PR #83, which deletes the hardcoded src/smoke.ts in favor of the vitest suite. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NikolayS
added a commit
that referenced
this pull request
Apr 30, 2026
client-smoke ran only TestSmoke-named tests and didn't set PGQUE_TEST_DSN, so env-gated integration tests silently skipped. That's how the Nack placeholder bug shipped green in PR #79. New client-tests job: - Sets PGQUE_TEST_DSN pointing at the docker PG. - Runs full Go suite: go test -race -v ./... - Runs full Python suite: pytest -v clients/python/tests/ - Runs full TypeScript suite: npx vitest --run (replaces tsx src/smoke.ts). Switching the TS step to vitest also unblocks PR #83, which deletes the hardcoded src/smoke.ts in favor of the vitest suite. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #75 addressing review items and a build-pipeline reconciliation that #75 missed.
Why a follow-up rather than rewriting #75
#75 landed three valuable fixes from @hobostay — Go consumer per-message nack,
dlq_replay_allexception isolation, queue-existence check ininsert_event_raw. The substantive logic is correct; the issues are at the edges (one regression, one visibility gap, missing test, and several edits placed in an auto-generated file). Merging #75 as-is preserved their commits cleanly; this PR layers the fixes on top.What this PR does
sql/pgque-api/maint.sql— quote_ident regression on schema-qualified VACUUMquote_ident(f.func_arg)is wrong.func_argcomes frompgque.maint_tables_to_vacuum()as"schema.table"— one string with a literal dot — soquote_identwraps the whole thing as a single identifier and producesvacuum "pgque.event_0"instead ofvacuum "pgque"."event_0". The latter is what PostgreSQL actually needs. Fix: split on the dot and quote each side viaformat('vacuum %I.%I', ...). Safer than the pre-#75 raw concatenation and correct under quoting.The bug is invisible to existing tests because
maint_tables_to_vacuum()returns nothing whenautovacuum = on(the default) — flagged in the original review.sql/pgque-additions/dlq.sql—dlq_replay_allpartial-success visibilityThe per-event exception block used
raise notice. NOTICE is hidden under most production configs (log_min_messages = warningby default), so partial failures became silent. Two changes:raise warning— visible in default logs.integerto a record(replayed bigint, failed bigint, first_error text)so callers can detect partial success programmatically.drop function if existsadded before thecreate or replacebecause PG doesn't allow CREATE OR REPLACE to change a function's return type.docs/reference.mdupdated.Breaking change: Existing callers doing
select pgque.dlq_replay_all('q')need to switch toselect replayed from pgque.dlq_replay_all('q')(or destructure all three columns). Acceptable at the v0.2 cut. No existing tests called this function.clients/go/pgque_test.go— Nack and per-message nack coverageAdds
TestNackandTestConsumerHandlerNacksOnErrorcovering the behavior #75 introduced. Verifies that a failing handler nacks its own message intopgque.retry_queue(not the DLQ —retry_countstarts belowmax_retries) without aborting the rest of the batch. Env-gated viaPGQUE_TEST_DSN, matching the existing test style.build/transform.sh— reconcile direct edits to the generatedpgque.sqlTwo of #75's source-level changes were applied directly to
sql/pgque.sql:upgrade_schema()— flattenedcreate role pgque_admin in role pgque_reader, pgque_writerinto explicitgrant ... to pgque_admincalls; changedreturn 0toreturn cnt.insert_event_raw()— raisesqueue not found: %when the lookup misses, instead of falling through to a NULL-deref later.sql/pgque.sqlis auto-generated frompgq/+sql/pgque-additions/+sql/pgque-api/bybuild/transform.sh. Direct edits to it get clobbered on the next rebuild. Re-applied both as post-transform patches intransform.shso the changes survive.sql/pgque.sql— regeneratedVerification runner reports all checks pass: schema rename, search_path on every SECURITY DEFINER, no
queue_per_tx_limit/default_with_oidsleakage, idempotent CREATE TABLE/SEQUENCE/INDEX.Build pipeline note
Anyone editing
sql/pgque.sqlshould instead edit the corresponding source undersql/pgque-additions/,sql/pgque-api/, or add a patch inbuild/transform.sh. The build's self-verification catches transformation drift but does not catch direct edits to the generated file that lack a source.Test plan
client-smokejob green (Go test compilation; integration tests env-gated)verifyjob green (transform.sh reproducibility check)\i sql/pgque.sqlsucceeds against a fresh PG18select * from pgque.dlq_replay_all('q')returns the expected three-column shape🤖 Generated with Claude Code