fix(http): send default User-Agent; harden flaky 06 http/ssrf e2e#270
Merged
Conversation
The durable HTTP client (reqwest) previously sent no User-Agent header. Some hosts reject anonymous requests: httpbingo.org (fly.io) returns HTTP 402 to any request without a User-Agent, which made the redirect test assert 3xx but receive 402, and silently let the other httpbingo tests "pass" on a 402 instead of a real success response. Product fix: - build_client() now sets a default User-Agent of "pg_durable/<version>". Nodes may still override it via an explicit User-Agent header. Test hardening (06_http_and_ssrf.sql): - Move the redirect test (Test 8) off httpbin.org (frequently unavailable; the dominant CI flake) onto httpbingo.org/status/302, inheriting the default 30s timeout. - Remove the residual-PUBLIC-grant permission test: it only exercised stock PostgreSQL grant semantics (a PUBLIC grant survives an additive per-role grant) and asserted nothing pg_durable-specific. Renumber the superuser bypass test accordingly. Harness fix (scripts/test-e2e-local.sh): - The repeat-run re-grant path (grant_e2e_df_usage) used the default include_http => false, so on runs >= 2 (after the extension is dropped and recreated) df_e2e_user lost HTTP access and 06 failed at the first df.http() call. Pass include_http => true to mirror 00_setup_playground, fixing `./scripts/test-e2e-local.sh 06_http_and_ssrf N`. Verified: 06_http_and_ssrf now passes 3/3 in repeat mode against a reused database. A separate, pre-existing BGW cold-start "pending" timeout flake on the first instance remains out of scope (tracked for a retry harness).
pinodeca
force-pushed
the
fix/flaky-06-http-phase1
branch
from
41def49 to
9aa2cf8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes the dominant CI flake in
06_http_and_ssrfand fixes a related product gap plus a repeat-run harness bug.Product fix
build_client()now sends a defaultUser-Agent: pg_durable/<version>. Previously the durable HTTP client (reqwest) sent none, and some hosts reject anonymous requests — httpbingo.org (fly.io) returns HTTP 402 to any request without a User-Agent. That made the redirect test assert3xxbut receive402, and silently let other httpbingo tests "pass" on a402instead of a real success. Nodes may still override the header explicitly.Test hardening (
tests/e2e/sql/06_http_and_ssrf.sql)httpbin.org(frequently unavailable — the dominant flake) ontohttpbingo.org/status/302.PUBLICgrant survives an additive per-role grant) and asserted nothing pg_durable-specific. Renumber the superuser-bypass test accordingly.Harness fix (
scripts/test-e2e-local.sh)grant_e2e_df_usage) used the defaultinclude_http => false. On runs ≥2 — after the extension is dropped and recreated —df_e2e_userlost HTTP access and 06 failed at the firstdf.http()call. Now passesinclude_http => trueto mirror00_setup_playground, fixing./scripts/test-e2e-local.sh 06_http_and_ssrf N.Verification
06_http_and_ssrfpasses 3/3 in repeat mode against a reused database.cargo fmt -p pg_durable -- --checkandcargo clippy --features http-allow-test-domainsclean.Out of scope
A separate, pre-existing BGW cold-start "pending" timeout flake on the first instance remains (tracked for a future retry harness).