Skip to content

test(workbench/e2e): poll for Model B/C options before selectOption#92

Merged
fede-kamel merged 1 commit into
mainfrom
test/anthropic-spec-bc-poll
May 6, 2026
Merged

test(workbench/e2e): poll for Model B/C options before selectOption#92
fede-kamel merged 1 commit into
mainfrom
test/anthropic-spec-bc-poll

Conversation

@fede-kamel
Copy link
Copy Markdown
Contributor

Summary

  • The Anthropic sweep flaked on tutorials feat: initial locus public release #1 and feat: initial public release of locus #2 because `configureAnthropic()` only polled for Model A's option to appear in `cfg-model` — it then fired `selectOption` on `cfg-model-b` immediately, racing the WebDriver layer's view of the populated options. Playwright retried for ~65s and timed out with Target page, context or browser has been closed.
  • Mirror the existing A poll for B and C: wait for the desired option text to be visible in the dropdown before calling `selectOption`. Same 15s budget, same `toPass` shape.
  • Verified live: the headless sweep now runs 32/32 green in 9.5 min on Anthropic with A=`claude-sonnet-4-6` and B=`claude-haiku-4-5-20251001` (full session matrix is now Anthropic 32/32 + OpenAI 32/32 + OCI v1 32/32).

Test plan

  • `npx playwright test tests/all-anthropic.spec.ts --workers=2` — 32/32 green
  • CI on this PR

The Anthropic sweep flaked on tutorials #1 and #2 because
configureAnthropic() only polled for Model A's option to appear in
the cfg-model dropdown — it then fired selectOption on cfg-model-b
immediately, racing the WebDriver layer's view of the populated
options. Playwright would retry for ~65s and eventually the test
timed out with "Target page, context or browser has been closed".

Mirror the existing A poll for B and C: wait for the desired option
text to be visible in the dropdown before calling selectOption. Same
15s budget, same toPass shape. Fixes the flake; the sweep ran
32/32 green in 9.5 min after this change.

Signed-off-by: Federico Kamelhar <federico.kamelhar@oracle.com>
@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label May 6, 2026
@fede-kamel fede-kamel merged commit b60c7b4 into main May 6, 2026
10 checks passed
@fede-kamel fede-kamel deleted the test/anthropic-spec-bc-poll branch May 6, 2026 04:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant