Skip to content

OCPBUGS-105517: Fix flaky search.spec.ts Playwright e2e tests - #16938

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
rhamilto:OCPBUGS-105517
Aug 10, 2026
Merged

OCPBUGS-105517: Fix flaky search.spec.ts Playwright e2e tests#16938
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
rhamilto:OCPBUGS-105517

Conversation

@rhamilto

@rhamilto rhamilto commented Aug 10, 2026

Copy link
Copy Markdown
Member

Analysis / Root cause:

The dev-console/search.spec.ts "Search page - Recently used" Playwright e2e tests flake in CI due to two race conditions:

  1. Dropdown not awaited after click: SearchPage.openResourcesFilter() clicks the resource filter combobox but returns immediately without waiting for the dropdown listbox (#resource-dropdown-listbox) to render. On slower CI runs, toBeVisible() assertions against dropdown contents fail because the element doesn't exist yet.

  2. Developer perspective not fully loaded: ensureDeveloperPerspective() patches the cluster config to enable the Developer perspective, then reloads until the perspective switcher toggle is no longer a static single-perspective div. However, the toggle becoming a <Select> dropdown does not guarantee the "Developer" menu option is listed — the console may still be loading perspective extensions. switchPerspective("Developer") opens the dropdown, finds no Developer option, and throws Perspective "Developer" not found in switcher menu.

Solution description:

  1. Added await expect(this.resourceDropdownList).toBeVisible() after the click in openResourcesFilter() so Playwright auto-retries until the dropdown is fully open before any test assertions run against its contents.

  2. Extended the ensureDeveloperPerspective() retry loop to open the perspective switcher dropdown and verify the "Developer" menu option is visible before considering the perspective ready. The dropdown is dismissed with Escape after each check.

Screenshots / screen recording:

Test setup:
Run the search.spec.ts Playwright e2e tests against a cluster where the Developer perspective is not enabled by default (single perspective mode).

Test cases:

  • shows recently searched resource in dropdown passes consistently
  • shows up to 5 recently searched items passes consistently
  • clears recently searched history passes consistently
  • Tests pass on clusters where Developer perspective is already enabled (no-op path)
  • Tests pass on clusters where Developer perspective needs to be enabled (patch path)

Browser conformance:

  • Chrome
  • Firefox
  • Safari (or Epiphany on Linux)

Additional info:
Jira: https://redhat.atlassian.net/browse/OCPBUGS-105517

Reviewers and assignees:

Summary by CodeRabbit

  • Tests
    • Improved end-to-end test setup by verifying the Developer perspective is available before proceeding.
    • Added a wait for the resource filter dropdown to become visible, improving test reliability.

The "Search page - Recently used" tests flake in CI for two reasons:

1. openResourcesFilter() clicks the combobox but doesn't wait for the
   dropdown listbox to render. On slower CI runs the assertion fires
   before the element exists.

2. ensureDeveloperPerspective() confirms the perspective switcher
   toggle is interactive but not that the Developer option is actually
   listed. The console may still be loading perspective extensions
   after the toggle re-renders as a dropdown.

Wait for the dropdown listbox visibility after opening, and extend the
ensureDeveloperPerspective retry loop to verify the Developer menu
option is present before proceeding.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 10, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@rhamilto: This pull request references Jira Issue OCPBUGS-105517, which is invalid:

  • expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Analysis / Root cause:

The dev-console/search.spec.ts "Search page - Recently used" Playwright e2e tests flake in CI due to two race conditions:

  1. Dropdown not awaited after click: SearchPage.openResourcesFilter() clicks the resource filter combobox but returns immediately without waiting for the dropdown listbox (#resource-dropdown-listbox) to render. On slower CI runs, toBeVisible() assertions against dropdown contents fail because the element doesn't exist yet.

  2. Developer perspective not fully loaded: ensureDeveloperPerspective() patches the cluster config to enable the Developer perspective, then reloads until the perspective switcher toggle is no longer a static single-perspective div. However, the toggle becoming a <Select> dropdown does not guarantee the "Developer" menu option is listed — the console may still be loading perspective extensions. switchPerspective("Developer") opens the dropdown, finds no Developer option, and throws Perspective "Developer" not found in switcher menu.

Solution description:

  1. Added await expect(this.resourceDropdownList).toBeVisible() after the click in openResourcesFilter() so Playwright auto-retries until the dropdown is fully open before any test assertions run against its contents.

  2. Extended the ensureDeveloperPerspective() retry loop to open the perspective switcher dropdown and verify the "Developer" menu option is visible before considering the perspective ready. The dropdown is dismissed with Escape after each check.

Screenshots / screen recording:

Test setup:
Run the search.spec.ts Playwright e2e tests against a cluster where the Developer perspective is not enabled by default (single perspective mode).

Test cases:

  • shows recently searched resource in dropdown passes consistently
  • shows up to 5 recently searched items passes consistently
  • clears recently searched history passes consistently
  • Tests pass on clusters where Developer perspective is already enabled (no-op path)
  • Tests pass on clusters where Developer perspective needs to be enabled (patch path)

Browser conformance:

  • Chrome
  • Firefox
  • Safari (or Epiphany on Linux)

Additional info:
Jira: https://redhat.atlassian.net/browse/OCPBUGS-105517

Reviewers and assignees:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Aug 10, 2026
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c2ee26ee-5d28-42b8-bb89-a04637b672b3

📥 Commits

Reviewing files that changed from the base of the PR and between 5764d40 and 3c1ded2.

📒 Files selected for processing (2)
  • frontend/e2e/pages/base-page.ts
  • frontend/e2e/pages/dev-console/search-page.ts

Walkthrough

The E2E page objects now verify that the Developer perspective is available and that the resource dropdown is visible before test execution continues.

Changes

E2E readiness checks

Layer / File(s) Summary
Developer perspective verification
frontend/e2e/pages/base-page.ts
The setup opens the perspective switcher, verifies the Developer option, and closes the menu with Escape.
Resource filter visibility assertion
frontend/e2e/pages/dev-console/search-page.ts
openResourcesFilter uses Playwright’s expect utility to verify that the resource dropdown is visible.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested labels: component/core

Suggested reviewers: shahsahil264, cajieh, logonoff

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (14 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the Jira issue and the main change: fixing flaky Playwright search tests.
Description check ✅ Passed The description covers root cause, solution, setup, test cases, browser conformance, and additional information, with only minor checklist details incomplete.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR changes only Playwright page objects; the affected search spec has static titles and no changed Ginkgo test title or dynamic value in a title.
Test Structure And Quality ✅ Passed The commit changes only two TypeScript Playwright page objects; it adds no Ginkgo It blocks, cluster resource setup, or Eventually/Consistently calls to review.
Microshift Test Compatibility ✅ Passed The pull request changes only two TypeScript Playwright page objects; it adds no Ginkgo e2e tests, so the MicroShift API compatibility check does not apply.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The patch changes only two frontend Playwright page objects. It adds no Go files or new Ginkgo e2e tests, so the SNO multi-node compatibility check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed The PR changes only Playwright page-object code in frontend/e2e/pages; it adds UI waits and perspective checks, with no manifests, operators, controllers, replicas, or scheduling constraints.
Ote Binary Stdout Contract ✅ Passed The PR changes only two Playwright TypeScript files; the parent-to-HEAD patch contains no Go, OTE, suite setup, or process-level stdout changes.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The patch changes only two TypeScript Playwright page objects; it adds no Ginkgo tests, IPv4 assumptions, or external connectivity.
No-Weak-Crypto ✅ Passed The PR adds only Playwright UI waits and perspective-menu handling; scans found no weak algorithms, crypto APIs, custom crypto, or secret comparisons.
Container-Privileges ✅ Passed The PR changes only two TypeScript source files. Added lines contain none of the checked privilege settings, and no container or Kubernetes manifest changes are present.
No-Sensitive-Data-In-Logs ✅ Passed The PR adds only UI waits and perspective-menu checks; the committed diff contains no logging calls or output of passwords, tokens, PII, hostnames, or customer data.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from cajieh and sg00dwin August 10, 2026 19:34
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 10, 2026
@rhamilto

Copy link
Copy Markdown
Member Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Aug 10, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@rhamilto: This pull request references Jira Issue OCPBUGS-105517, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@rhamilto

Copy link
Copy Markdown
Member Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-gcp-console
/test e2e-playwright

@logonoff logonoff left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 10, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Tests from second stage were triggered manually. Pipeline can be controlled only manually, until HEAD changes. Use command to trigger second stage.

@openshift-ci

openshift-ci Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: logonoff, rhamilto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@logonoff

Copy link
Copy Markdown
Member

/verified by playwright test passes

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Aug 10, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@logonoff: This PR has been marked as verified by playwright test passes.

Details

In response to this:

/verified by playwright test passes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

@rhamilto: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-gcp-console-techpreview 3c1ded2 link false /test e2e-gcp-console-techpreview

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot
openshift-merge-bot Bot merged commit 42ca40a into openshift:main Aug 10, 2026
10 of 11 checks passed
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@rhamilto: Jira Issue Verification Checks: Jira Issue OCPBUGS-105517
✔️ This pull request was pre-merge verified.
✔️ All associated pull requests have merged.
✔️ All associated, merged pull requests were pre-merge verified.

Jira Issue OCPBUGS-105517 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓

Details

In response to this:

Analysis / Root cause:

The dev-console/search.spec.ts "Search page - Recently used" Playwright e2e tests flake in CI due to two race conditions:

  1. Dropdown not awaited after click: SearchPage.openResourcesFilter() clicks the resource filter combobox but returns immediately without waiting for the dropdown listbox (#resource-dropdown-listbox) to render. On slower CI runs, toBeVisible() assertions against dropdown contents fail because the element doesn't exist yet.

  2. Developer perspective not fully loaded: ensureDeveloperPerspective() patches the cluster config to enable the Developer perspective, then reloads until the perspective switcher toggle is no longer a static single-perspective div. However, the toggle becoming a <Select> dropdown does not guarantee the "Developer" menu option is listed — the console may still be loading perspective extensions. switchPerspective("Developer") opens the dropdown, finds no Developer option, and throws Perspective "Developer" not found in switcher menu.

Solution description:

  1. Added await expect(this.resourceDropdownList).toBeVisible() after the click in openResourcesFilter() so Playwright auto-retries until the dropdown is fully open before any test assertions run against its contents.

  2. Extended the ensureDeveloperPerspective() retry loop to open the perspective switcher dropdown and verify the "Developer" menu option is visible before considering the perspective ready. The dropdown is dismissed with Escape after each check.

Screenshots / screen recording:

Test setup:
Run the search.spec.ts Playwright e2e tests against a cluster where the Developer perspective is not enabled by default (single perspective mode).

Test cases:

  • shows recently searched resource in dropdown passes consistently
  • shows up to 5 recently searched items passes consistently
  • clears recently searched history passes consistently
  • Tests pass on clusters where Developer perspective is already enabled (no-op path)
  • Tests pass on clusters where Developer perspective needs to be enabled (patch path)

Browser conformance:

  • Chrome
  • Firefox
  • Safari (or Epiphany on Linux)

Additional info:
Jira: https://redhat.atlassian.net/browse/OCPBUGS-105517

Reviewers and assignees:

Summary by CodeRabbit

  • Tests
  • Improved end-to-end test setup by verifying the Developer perspective is available before proceeding.
  • Added a wait for the resource filter dropdown to become visible, improving test reliability.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@rhamilto
rhamilto deleted the OCPBUGS-105517 branch August 11, 2026 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants