Skip to content

fix(test): increase retry for AUT and enable traces - #31248

Merged
chirag-madlani merged 1 commit into
mainfrom
fix/flaky-loginspec
Aug 10, 2026
Merged

fix(test): increase retry for AUT and enable traces#31248
chirag-madlani merged 1 commit into
mainfrom
fix/flaky-loginspec

Conversation

@chirag-madlani

@chirag-madlani chirag-madlani commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Describe your changes:

Fixes #

I worked on ... because ...

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

High-level design:

N/A — small change.

Tests:

Use cases covered

Unit tests

Backend integration tests

Ingestion integration tests

Playwright (UI) tests

Manual testing performed

UI screen recording / screenshots:

Not applicable.

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • My PR is linked to a GitHub issue via Fixes #<issue-number> above.
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.
  • For UI changes: I attached a screen recording and/or screenshots above.
  • I have added tests (unit / integration / Playwright as applicable) and listed them above.

Summary by Gitar

  • Test Improvements:
    • Enabled Playwright traces on failure in Login.spec.ts.
    • Conditionalized retry count for token renewal tests based on PLAYWRIGHT_IS_OSS.
    • Reduced token expiry timeout duration for faster test execution.

This will update automatically on new commits.

Copilot AI lite review requested due to automatic review settings August 10, 2026 05:52
@chirag-madlani
chirag-madlani requested a review from a team as a code owner August 10, 2026 05:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown
Contributor

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

  • No GitHub issue is linked. Link an issue in the Development section of the PR (or add Fixes #12345 to the description). For a same-org cross-repo issue, add Fixes open-metadata/<repo>#123 to the description.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

@github-actions github-actions Bot added safe to test Add this label to run secure Github workflows on PRs UI UI specific issues labels Aug 10, 2026
@chirag-madlani chirag-madlani added the To release Will cherry-pick this PR into the release branch label Aug 10, 2026
@gitar-bot

gitar-bot Bot commented Aug 10, 2026

Copy link
Copy Markdown
Code Review ✅ Approved 1 resolved / 1 findings

Increases test retry counts and enables Playwright traces on failure to improve test reliability. Consider increasing the wait margin slightly past the 120s token expiry to prevent flaky refresh timing.

✅ 1 resolved
Edge Case: Wait leaves only ~2s margin past 120s token expiry

📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Pages/Login.spec.ts:198-199 📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Pages/Login.spec.ts:64-77
The token expiry is configured to 120s (JWT_EXPIRY_TIME_MAP['2 minutes'] in the beforeAll), but the wait was reduced to 2611000 = 122s, leaving only a ~2s window for the refresh timer to fire and the refresh API round-trip to complete. This is a much tighter margin than the previous 180s (60s buffer) and could increase flakiness on slow CI — the added retries:2 mask rather than remove the risk. Consider a slightly larger buffer (e.g. 130-140s) that still stays well under the 5-minute test timeout. Note the comment '61s * 2' and the beforeAll comments ('3 minutes'/'3 mins') are also inaccurate given the 120s expiry.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

@chirag-madlani chirag-madlani added safe to test Add this label to run secure Github workflows on PRs and removed safe to test Add this label to run secure Github workflows on PRs labels Aug 10, 2026
@chirag-madlani
chirag-madlani added this pull request to the merge queue Aug 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

✅ Playwright Results — workflow succeeded

Validated commit 46ee4bfb0d6df9edf632697feac7ba2be5d9133b in Playwright run 31360322849, attempt 1.

✅ 46 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 0 skipped · 🧰 0 lifecycle flaky

Performance

Blocking targets: ✅ met · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 40m 2s

⏱️ Max setup 1m 34s · max shard execution 5m 8s · max shard-job elapsed before upload 9m 16s · reporting 3s

🌐 131.87 requests/attempt · 1.29 app boots/UI scenario · 0.00% common-shard skew

Optimization targets still in progress:

  • Application boot ratio was 1.29 per UI scenario (62 boots / 48 scenarios; convergence target: at most 1).
Shard Passed Failed Flaky Skipped Lifecycle failed Lifecycle flaky
✅ Shard chromium-01 46 0 0 0 0 0

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

Merged via the queue into main with commit b8e00ad Aug 10, 2026
184 of 195 checks passed
@chirag-madlani
chirag-madlani deleted the fix/flaky-loginspec branch August 10, 2026 07:34
@github-actions

Copy link
Copy Markdown
Contributor

Changes have been cherry-picked to the 2.0 branch.

github-actions Bot pushed a commit that referenced this pull request Aug 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Failed to cherry-pick changes to the 1.13 branch.
Please cherry-pick the changes manually.
You can find more details here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs To release Will cherry-pick this PR into the release branch UI UI specific issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants