Skip to content

fix(playwright): route BulkImport back to import-export lane and audit-split the atomic describe - #30834

Merged
chirag-madlani merged 1 commit into
mainfrom
fix/bulk-import-export-lane-and-audit
Aug 3, 2026
Merged

fix(playwright): route BulkImport back to import-export lane and audit-split the atomic describe#30834
chirag-madlani merged 1 commit into
mainfrom
fix/bulk-import-export-lane-and-audit

Conversation

@chirag-madlani

Copy link
Copy Markdown
Collaborator

Summary

The merge-queue plan step for PR #30697 failed with:

Atomic Playwright units exceed the 20-minute execution budget; refactor or explicitly
audit them for parallel splitting: chromium|Features/BulkImport.spec.ts|Bulk Import Export (25.8m)

Two orthogonal issues from #30458's re-enable of the suite, both fixed here.

Issue 1 — the @import-export tag was dropped along with .fixme

Before #30458:

test.describe.fixme('Bulk Import Export', { tag: '@import-export' }, () => {  })

After #30458 (current origin/main):

test.describe('Bulk Import Export', () => {  })

.fixme was removed correctly, but the { tag: '@import-export' } option went with it. Without the tag:

  • ImportExport project's grep: combineGrep(/@import-export/) doesn't match → tests are excluded from the dedicated project.
  • chromium project's grepInvert: [.., /@import-export/, ..] (active with PW_DEDICATED_IMPORT_EXPORT=true) doesn't exclude them → they land on the chromium lane.

Fix: restore the tag on the describe. The whole suite routes back to the dedicated ImportExport project → import-export lane.

Issue 2 — the atomic-unit ceiling still trips even in the import-export lane

The Bulk Import Export describe wraps six tests, each declaring test.setTimeout(300_000-600_000) (5-10 min per test). The planner's oversized_units check compares each unit's weight_ms against TARGET_MS = 20 * 60 * 1000 regardless of destination lane — so a >20-min aggregate breaks the plan step wherever it lands.

Fix: add ("Features/BulkImport.spec.ts", "Bulk Import Export") to AUDITED_PARALLEL_SUITES. discover_units then splits the describe into six per-spec parallel units. Module-scoped entity constructors (new UserClass(), new GlossaryClass() etc.) generate unique names per test instance, so each parallel unit brings its own beforeAll state without cross-worker collision.

Verification

  • 64 planning tests pass locally.
  • Reproduced the split against a synthetic test-list mirroring the CI shape — one atomic unit becomes six per-spec units (each carrying one test id).
  • Prettier + ESLint clean on the modified spec.

Test plan

  • Next merge-queue run: plan step succeeds, matrix includes an import-export shard with BulkImport specs.
  • BulkImport tests execute on the import-export lane (visible in the shard-job name), not chromium.
  • Chromium lane content drops by the BulkImport aggregate — small headroom win.

Related: #30812 (planner all-zero-history fix), #30822 (glossary hang fix).

🤖 Generated with Claude Code

…t-split the atomic describe

Merge-queue plan step for PR #30697 failed with:

  Atomic Playwright units exceed the 20-minute execution budget; refactor
  or explicitly audit them for parallel splitting:
    chromium|Features/BulkImport.spec.ts|Bulk Import Export (25.8m)

Two things went wrong when #30458 re-enabled the suite:

1. The re-enable rewrote

     test.describe.fixme('Bulk Import Export', { tag: '@Import-Export' }, ...)

   as

     test.describe('Bulk Import Export', ...)

   dropping both `.fixme` AND the `@import-export` tag. Without the tag,
   the suite fails to match `ImportExport` project's grep AND is not
   excluded by the chromium project's grepInvert — so the whole thing
   lands on the chromium lane instead of the dedicated import-export
   lane it was designed for. Restore the tag on the describe.

2. Even after the tag is restored, the entire "Bulk Import Export"
   describe is a single atomic unit. The six tests inside each set
   `test.setTimeout(300_000-600_000)` (5-10 min each), so on any real
   run the aggregate crosses the planner's 20-minute per-unit ceiling
   (TARGET_MS) regardless of lane. Add the describe to
   `AUDITED_PARALLEL_SUITES` so `discover_units` splits it into six
   per-spec parallel units. Module-scoped entity constructors generate
   unique names per test instance, so each parallel unit brings its
   own beforeAll state without cross-worker collision.

Together: the tests route to the import-export lane (dedicated workers,
tuned resources) AND no single unit can trip the atomic-ceiling gate as
the suite grows.

64 planning tests pass locally. Prettier + ESLint clean on the spec.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 3, 2026 04:55
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

  • No GitHub issue is linked. Link an issue in the Development section of the PR (or add Fixes #12345 to the description). For a same-org cross-repo issue, add Fixes open-metadata/<repo>#123 to the description.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

@github-actions github-actions Bot added safe to test Add this label to run secure Github workflows on PRs UI UI specific issues labels Aug 3, 2026
@gitar-bot

gitar-bot Bot commented Aug 3, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Restores the @import-export tag and adds BulkImport to AUDITED_PARALLEL_SUITES to fix CI execution budget failures and route tests back to the correct lane. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar | Powered by Gitar — free for open source

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Playwright CI sharding/planning by ensuring the BulkImport suite is (1) routed to the dedicated Import/Export lane again and (2) split into audited per-spec units so it no longer violates the 20-minute atomic-unit budget enforced by the shard planner.

Changes:

  • Restores the @import-export tag on the Bulk Import Export describe, so the suite is picked up by the ImportExport Playwright project and excluded from the chromium lane when PW_DEDICATED_IMPORT_EXPORT=true.
  • Adds ("Features/BulkImport.spec.ts", "Bulk Import Export") to AUDITED_PARALLEL_SUITES, enabling the planner to split the suite into per-spec parallel units instead of treating the whole describe as one oversized atomic unit.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/BulkImport.spec.ts Re-adds the @import-export tag to route the suite back to the ImportExport lane.
.github/scripts/build_playwright_shards.py Audits/splits Bulk Import Export into per-spec units to satisfy the planner’s atomic-unit runtime ceiling.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

✅ Playwright Results — workflow succeeded

Validated commit 700a18337b57f70418844a5c8357fac9dd0bbeb7 in Playwright run 30785721057, attempt 1.

✅ 554 passed · ❌ 0 failed · 🟡 2 flaky · ⏭️ 3 skipped · 🧰 0 lifecycle flaky

Performance

Blocking targets: ✅ met · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 52m 11s

⏱️ Max setup 3m 1s · max shard execution 19m 5s · max shard-job elapsed before upload 22m 0s · reporting 5s

🌐 203.56 requests/attempt · 2.81 app boots/UI scenario · 2.55% common-shard skew

Optimization targets still in progress:

  • Browser traffic was 203.56 requests per attempt (convergence target: fewer than 200).
  • Application boot ratio was 2.81 per UI scenario (1629 boots / 579 scenarios; convergence target: at most 1).
Shard Passed Failed Flaky Skipped Lifecycle failed Lifecycle flaky
🟡 Shard chromium-01 147 0 1 3 0 0
✅ Shard chromium-02 132 0 0 0 0 0
✅ Shard chromium-03 119 0 0 0 0 0
✅ Shard data-asset-rules-01 61 0 0 0 0 0
✅ Shard domain-isolation-01 14 0 0 0 0 0
✅ Shard global-state-01 34 0 0 0 0 0
🟡 Shard import-export-01 5 0 1 0 0 0
✅ Shard ingestion-01 1 0 0 0 0 0
✅ Shard reindex-01 2 0 0 0 0 0
✅ Shard search-01 10 0 0 0 0 0
✅ Shard search-rbac-01 29 0 0 0 0 0
🟡 2 flaky test(s) (passed on retry)
  • Pages/Entity.spec.tsDomain Propagation (shard chromium-01, 1 retry)
  • Features/BulkImport.spec.tsRange selection (shard import-export-01, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@ShaileshParmar11
ShaileshParmar11 added this pull request to the merge queue Aug 3, 2026
@sonika-shah
sonika-shah removed this pull request from the merge queue due to a manual request Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

🚦 Removed from the merge queue — manual (2026-08-03T09:14:36Z)

Blocked the queue: playwright-summary, integration-tests-mysql-elasticsearch

@chirag-madlani
chirag-madlani added this pull request to the merge queue Aug 3, 2026
@chirag-madlani
chirag-madlani removed this pull request from the merge queue due to a manual request Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

🚦 Removed from the merge queue — manual (2026-08-03T10:44:48Z)

Blocked the queue: playwright-summary

  • Postgresql PR Playwright E2E Tests — playwright-summary, playwright-ci-postgresql (ingestion-01), playwright-ci-postgresql (chromium-17), playwright-ci-postgresql (chromium-10)

@chirag-madlani
chirag-madlani merged commit dfa9d49 into main Aug 3, 2026
80 of 81 checks passed
@chirag-madlani
chirag-madlani deleted the fix/bulk-import-export-lane-and-audit branch August 3, 2026 10:45
chirag-madlani added a commit that referenced this pull request Aug 3, 2026
…sized-unit error

Two changes to make the tag-drop failure mode (PR #30834) actionable at
PR time rather than a merge-queue plan failure.

A. Improved `oversized_units` error message

The existing gate correctly fires when any unit exceeds TARGET_MS,
but the message ("refactor or explicitly audit them for parallel
splitting") left developers guessing between two very different fixes.
Rewrite it to spell out both:

  * Restore a `{ tag: '...' }` option that a recent edit dropped
    (points at FILE_LANE_HINTS and PR #30834 for the pattern).
  * Add the (file, describe_title) to AUDITED_PARALLEL_SUITES to
    split the describe into per-spec parallel units.

B. FILE_LANE_HINTS — filename → expected project check

A small mapping from filename regex → expected project + expected tag.
At plan time, for every planned unit whose file matches a hint but
whose project doesn't match the expected one, hard-fail with:

  Features/BulkImport.spec.ts: on `chromium` project, expected
  `ImportExport` (add `{ tag: '@Import-Export' }` to the top-level
  describe)

This catches the exact PR #30834 pattern (a re-enable rewrote
`test.describe.fixme('...', { tag: '@Import-Export' }, ...)` as
`test.describe('...', ...)`, dropping the tag along with `.fixme`) at
PR review time. Runs before the oversized-unit check so the developer
gets the specific error, not the generic one.

Seeded with one hint today — BulkImport*.spec.ts + *ImportExport*.spec.ts
→ `ImportExport` / `@import-export`. That matches 12 of the 13 files
currently in the tree with those names; the 13th is the very bug this
guard catches (see PR #30834). Add other conventions as they emerge.

7 new pytest cases cover the hint match, the ignore path, dedupe across
audit-split units, the improved oversized message, and an end-to-end
main() SystemExit reproduction of the tag-drop scenario. 71 total
planner tests pass locally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
anuj-kumary pushed a commit to anuj-kumary/OpenMetadata that referenced this pull request Aug 3, 2026
…pen-metadata#30871)

* chore(playwright): refresh timing baseline from run 30802611748

The checked-in `.github/playwright/timing-baseline.json` had drifted 12
days behind the tree — captured 2026-07-22 (sha ccc86fe), but 51
commits since then have added, renamed, or re-tagged specs. Concrete
drift as measured on origin/main today:

  * 30 current spec files have zero baseline coverage
    (Auth/SSOAuthentication.spec.ts + 25 tests, KnowledgeGraph, the 8
    Features/SearchSeparation files, etc.)
  * 1007 baseline titles (23 %) reference tests that no longer exist —
    dead weight in load_history
  * 66 files (20 % of covered) have title drift — leaf renames, @tag
    additions, describe restructures
  * Worst offenders — Pages/ExplorePageRightPanel (218/234 orphaned,
    93 % dead), Features/AdvancedSearch (107/131, 82 %),
    Pages/CustomProperties (98/149, 66 %),
    Pages/Lineage/DataAssetLineage (79/85, 93 %)

Refresh from the most recent successful full-mode merge_group run
(30802611748, 2026-08-03 09:43 UTC, sha 79df9ae). Its
`playwright-timing-history-full` artifact is retained until 2026-09-02
and reflects the CI's actual runtime under the current tree.

Metadata delta:

  * 4384 → 4409 tests (25 net additions)
  * 322 → 325 files (4 new covered: AssetHealthWidget,
    ColumnBulkOperationsTagsGlossary, IncidentManagerPagination,
    PipelineValidation)
  * 49 → 14 zero-duration entries (35 previously-skipped tests now
    have real durations captured)
  * BulkImport correctly recorded under the `ImportExport` project
    (PR open-metadata#30834 route fix reflected in the fresh capture)

Preserved unchanged from the current baseline:

  * `retainedUnstableTestIds` (68-entry curated allowlist for known
    flaky tests)
  * `retainedSourceRunId` / `retainedSourceSha` pointers

Normalized entries to the current baseline's schema
(short `title` instead of the reporter's ` › proj › file › describe ›
leaf` breadcrumb; dropped `retryDurationMs`) so the diff stays a
one-line JSON reserialization on the payload.

Also removed a stale assertion in
`test_versioned_baseline_omits_all_zero_ids_from_weights` that required
at least one zero-duration entry with outcome=expected — a bug-case
sanity check that no longer holds now that the fresh capture only
contains legitimately-skipped zero-ms entries. The load-time filtering
behavior (the meaningful part of the test) still asserts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* address Copilot review — guard against vacuous pass

`test_versioned_baseline_omits_all_zero_ids_from_weights` builds
`zero_tests = [t for t in baseline if durationMs == 0]` and then asserts
`all(t.id not in weights for t in zero_tests)`. If a future baseline
refresh captures a run with no zero-duration entries, `zero_tests` is
empty and the `all(...)` passes vacuously — the test would go green
without actually exercising the load_history filter.

Add an explicit `assert zero_tests` before the filter check with a
message telling the next baseline-refresher to either construct a
synthetic fixture or convert the test to synthetic input. Today's
refreshed baseline still has 14 skipped 0-ms entries, so the guard is
a no-op right now — it just catches the future case.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs UI UI specific issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants