Skip to content

test(system): a response wait only where it is the evidence - #1135

Merged
ExtraToast merged 1 commit into
mainfrom
test/response-waits-that-earn-their-place
Sep 5, 2026
Merged

test(system): a response wait only where it is the evidence#1135
ExtraToast merged 1 commit into
mainfrom
test/response-waits-that-earn-their-place

Conversation

@ExtraToast

@ExtraToast ExtraToast commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Closes #1125.

Why

Two flakes in two runs, on branches that touch neither the frontend nor the code under test:

EventPageSystemTest > logged-in user can delete existing event sign-up() FAILED
    com.microsoft.playwright.TimeoutError: Timeout 5000ms exceeded
RecoveryManagerPageSystemTest > board deletes user from member manager and restores in recovery manager() FAILED
    com.microsoft.playwright.TimeoutError: Timeout 5000ms exceeded

Both wrapped a click in page.waitForResponse and then asserted the real outcome on the very next line — the sign-up gone from the database, the account back in the pane. The wait was a weaker second copy of that assertion, and it is the copy that fails: it caps the round trip at the shared five-second budget, on the browser's view of a request whose success re-renders the element the click was on.

What this achieves

The rule is written down and the suite follows it. 19 of the 62 call sites were duplicates and go; the assertion that follows each one is unchanged and now gets the full budget instead of what the wait left it.

How

The rule lives in Polling.kt, beside the budget every wait already shares, because that is the file a test reads when it is choosing how to wait.

A wait earns its place when the response is the evidence — EventPageSystemTest reads an x-guest-access-token header, ActivationPageSystemTest reads an error body, EventEditPageSystemTest asserts the banner is fetched at its own public address and nothing observable follows. It also earns it when something touches the page before the assertion does: a navigation, another pane opened, a second form filled. There the wait is what stops the request racing whatever comes next.

It does not earn it when the assertion is the next thing. Those 19 became a plain click followed by the pollFor or Playwright assertion that was always the real check.

Worth a reviewer's attention

The sweep exposed one real race rather than causing it. The membership rendezvous writes the membership row and the MEMBER role separately; confirming the address after applying starts the membership polled for the row and then asserted the role without polling, and the response wait had been padding the gap. Removing the wait turned that into Expecting ["GUEST"] to contain ["MEMBER"]. The poll now waits for both facts, which is what the scenario is named for.

I tightened the rule twice while applying it, because the first two passes were wrong in ways only running the suite showed: a variable read more than once is a response being asserted, and an interaction between the click and the assertion means the wait is synchronising. Both are in the written rule now.

Verification

./gradlew :tests:system:test against a compose stack on the CI images, every class the sweep touched — EventPageSystemTest, EventSignUpsPageSystemTest, ActivationPageSystemTest, AddressPageSystemTest, CommitteeManagerPageSystemTest, JobManagerPageSystemTest, RecoveryManagerPageSystemTest, MembershipSignUpPageSystemTest37 tests, 37 passed.

An earlier, looser version of the sweep failed two of those tests, which is how the two extra conditions were found. The full six-shard suite runs here in CI.


Diff breakdown added removed, scaled to the largest row.

system-tests                                       +44   -142    7
  system tests       ██████░░░░░░░░░░░░░░░░░░░░    +44   -142    7

──────────────────────────────────────────────────────────────────
production                                          +0     -0
tests                                              +44   -142
total (hand-written)                               +44   -142  7 files

Two flakes in two runs, on branches touching neither the frontend nor the code
under test: EventPageSystemTest timing out on a DELETE and
RecoveryManagerPageSystemTest on a restore. Both wrapped a click in
page.waitForResponse and then asserted the real outcome on the next line, so the
wait was a weaker second copy of that assertion — and it is the copy that fails,
because it caps the round trip on the browser's view of a request whose success
re-renders the element the click was on.

The rule is written down in Polling.kt, beside the budget every wait shares, and
all 62 call sites were checked against it. Nineteen were duplicates and go. The
rest stay, for two reasons the sweep had to learn: the response carries evidence
nothing else does — a guest access token header, an error body — or something
touches the page before the assertion does, a navigation or a second form, where
the wait is what stops the request racing it.

Removing the waits exposed one real race rather than causing it: the membership
rendezvous writes the row and the MEMBER role separately, and the test polled for
the row and then asserted the role, which the wait had been padding. The poll now
waits for both.

Closes #1125
@ExtraToast ExtraToast added bug Something isn't working ci labels Sep 5, 2026
@ExtraToast ExtraToast self-assigned this Sep 5, 2026
@ExtraToast
ExtraToast merged commit 89a6f16 into main Sep 5, 2026
25 checks passed
@ExtraToast
ExtraToast deleted the test/response-waits-that-earn-their-place branch September 5, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A response wait caps a round trip the next line already proves

1 participant