Skip to content

Fixes 30704: Gate agent wizard Next on form readiness and clear the stuck dashboard service loader - #30705

Merged
ShaileshParmar11 merged 4 commits into
mainfrom
fix-serviceingestion-nightly-failures
Aug 2, 2026
Merged

Fixes 30704: Gate agent wizard Next on form readiness and clear the stuck dashboard service loader#30705
ShaileshParmar11 merged 4 commits into
mainfrom
fix-serviceingestion-nightly-failures

Conversation

@aniketkatkar97

@aniketkatkar97 aniketkatkar97 commented Jul 30, 2026

Copy link
Copy Markdown
Member

Describe your changes:

Fixes #30704

ServiceIngestion.spec.ts was failing on the nightly AUT run. I pulled the retained traces (16 zips) and found four distinct failure signatures, not one flake — two genuine product bugs plus two stale selectors. Because the affected blocks are test.describe.serial, one root cause takes out a whole service suite (e.g. Superset never reaches Update schedule options because Update description… dies first).

Failure Suites Root cause
getByTestId('schedular-schedule') times out (60 s) BigQuery, MySQL, Redshift Wizard next-button silently no-ops while the RJSF form is suspended
expect(loader).toHaveCount(0) stuck at 1 (30 s) Metabase, Superset fetchDashboardsDataModel never clears isServiceLoading
#root/dbtConfigSource__oneof_select times out Redshift RJSF native oneOf select replaced by a react-aria Select
Test Connection Done times out (180 s) Api Service CollateSaaS runner never completes the Rest workflow — infra, out of scope

1. next-button is a dead click while the Configure Ingestion form lazy-loads. AddIngestionPage / EditIngestionPage render the footer's next-button with no disabled state; pressing it calls addIngestionRef.current?.submit()workflowFormRef.current?.submit()formRef.current?.submit(). IngestionWorkflowForm wraps its RJSF <Form> in <Suspense> and passes bare React.lazy templates, so until those chunks resolve formRef.current is null and submit() does nothing — no toast, no validation error, the wizard just stays on step 1. The trace is unambiguous: edit-button at t=391905, next-button at t=392080 (175 ms later), wizard data fetches landing at t=392102 — after the click. Regressed in #28569, which moved the advance button out of the form (it used to be a native htmlType="submit", which could not race a ref).

IngestionWorkflowForm now reports readiness from a small FormReadyNotifier rendered inside the Suspense boundary (so its effect can only fire once every lazy template has resolved), AddIngestion forwards that up as onStepReadyChange, and both host pages disable next-button until the active step is ready — mirroring the footerNextDisabled pattern EmbeddedAddServicePage already uses. Step 2 (ScheduleIntervalStep) is a static import, so it reports ready immediately.

While in there: {!hideFooter && (…)} evaluates to false, and RJSF treats falsy children as "no children" (children ? children : <SubmitButton/>), so it was injecting its own stray "Submit" button into the wizard next to the real footer. Suppressed via ui:submitButtonOptions: { norender: true }.

2. Dashboard services leave a table spinner running forever. ServiceDetailsPage.fetchDashboardsDataModel set isServiceLoading(true) with no finally, unlike its sibling getOtherDetails. Its effect re-runs on activeTab, and getOtherDetails early-returns on a non-entity tab, so switching Dashboards → Agents left the flag stuck on. antd Tabs keeps the visited pane mounted, so the Dashboards <Table loading> spinner stayed DOM-attached for the rest of the page's life. Invisible to the user, but fatal to any page-wide [data-testid="loader"] assertion — and dashboard-service-specific, which is exactly why only Superset and Metabase failed.

3. Playwright fixes. New waitForIngestionWorkflowForm helper (mirrors the existing waitForServiceConnectionForm) used at every step-1 advance; openAgentScheduleStep extracted from the four duplicated blocks in updateScheduleOptions; the agent-tab loader waits in updateDescriptionForIngestedTables scoped the way addIngestionPipeline already does it; the dbt config source moved onto the existing selectOneOfOption helper; and the two [data-testid="submit-btn"] clicks replaced with next-button — that testid does not exist in this wizard (verified against @rjsf/core 5.24.13 Form.render; the button rendered there is RJSF's default and carries no testid).

selectOneOfOption itself had a latent bug found while dry-running: its Select branch preferred clicking react-aria's visually hidden native combobox (with force: true), which never opens the listbox. Only the tabs branch was exercised by passing tests. It now clicks the wrapper — the pattern ServiceBaseClass.createService already uses successfully for the ingestion-runner select — and scopes options to the visible popover.

Type of change:

  • Bug fix

High-level design:

The wizard footer lives in the host page while the form it submits lives three levels down behind a Suspense boundary, so "is this step submittable" has to travel upwards. Rather than reach into the ref, readiness is reported as a plain callback chain:

IngestionWorkflowForm  --onReady-->  AddIngestion  --onStepReadyChange-->  Add/EditIngestionPage
   (FormReadyNotifier                  (combines with                        (isDisabled={!isStepReady})
    inside <Suspense>)                  activeIngestionStep)

FormReadyNotifier is a sibling of <Form> rather than one of its children on purpose — RJSF's children slot doubles as the submit-button override, and putting anything there changes button rendering. Being inside the boundary is what makes the signal correct: any suspending template defers the whole subtree, so the effect cannot fire early.

Alternatives rejected:

  • Queue the submit and flush it when the form mounts. Hides the problem instead of fixing it, and a click that appears to do nothing for 200 ms then acts is worse UX than a briefly disabled button.
  • Test-side wait only. Leaves a real dead button in the product for anyone clicking fast.
  • Drop the lazy templates. Would undo deliberate code-splitting (Improve connector setup UI flow #28569, d82b7cf0cc).

No schema, API, or migration impact. The new props are optional, so the third AddIngestion-shaped consumer (EmbeddedAddServicePage, which already has its own gate) is unaffected.

Tests:

Use cases covered

  • Opening Edit Metadata Agent and pressing Next as soon as it is enabled advances to the Schedule Interval step (previously a silent no-op).
  • The wizard's Next button is disabled while the Configure Ingestion form is still loading, and enabled once it mounts.
  • Switching from a dashboard service's Dashboards tab to Agents no longer leaves the entity table in a loading state — including when the data model count request fails.
  • Adding a dbt agent to a Redshift service selects DBT S3 Config through the react-aria oneOf select.

Unit tests

  • I added unit tests for the new/changed logic.
  • Files added/updated:
    • src/components/Settings/Services/AddIngestion/AddIngestion.test.tsx — the configure step reports ready only after the workflow form mounts; the schedule step reports ready immediately.
    • src/pages/AddIngestionPage/AddIngestionPage.test.tsxnext-button stays disabled until the active step reports ready.
    • src/pages/ServiceDetailsPage/ServiceDetailsPage.test.tsx — the entity tab is not left loading after moving off it, on both the resolved and rejected data-model-count paths.
  • yarn test src/components/Settings/Services src/pages/AddIngestionPage src/pages/ServiceDetailsPage28 suites, 322 tests passing.
  • The ServiceDetailsPage test was confirmed to actually fail without the fix (data-service-loading="true") before being kept.

Backend integration tests

  • Not applicable (no backend API changes).

Ingestion integration tests

  • Not applicable (no ingestion changes).

Playwright (UI) tests

  • I added Playwright E2E tests for UI changes.
  • Files added/updated:
    • playwright/e2e/nightly/ServiceIngestion.spec.ts — new Edit agent wizard step navigation describe: creates a MySQL service + deployed metadata pipeline via API, opens the edit wizard, and asserts Next reaches the schedule step. No ingestion runtime needed, so it is a cheap regression guard for the 60 s timeout.
    • playwright/utils/serviceIngestion.ts, playwright/utils/serviceFormUtils.ts, playwright/support/entity/ingestion/{ServiceBaseClass,MySqlIngestionClass,PostgresIngestionClass,RedshiftWithDBTIngestionClass}.ts — helper and selector fixes described above.

Manual testing performed

The Playwright changes target the nightly AUT environment (Redshift / Superset / BigQuery credentials), which I do not have locally, so the E2E suite has not been executed on this branch — it needs a nightly run to confirm. What was verified locally:

  1. yarn test … — 28 suites / 322 tests green, and the loader test verified RED without the finally.
  2. npx tsc --noEmit — no new errors on the changed files (the one pre-existing AJV validator variance error in IngestionWorkflowForm.tsx reproduces on a stashed tree).
  3. make ui-checkstyle-changed — exit 0, no reformat diff.
  4. Traced every other caller that clicks the agent wizard's next-button (playwright/utils/autoClassification.ts, playwright/utils/profilerForm.ts, playwright/e2e/Features/StorageMetadataAgentForm.spec.ts) to confirm each already waits on the form before advancing, so the new disabled state cannot hang them.

UI screen recording / screenshots:

No visual change — the only user-visible difference is that the wizard's Next button is briefly disabled instead of dead while the form loads, and a stray RJSF "Submit" button no longer appears next to the footer. Trace evidence for the original failure, from the nightly run:

t=391905  click [data-testid="edit-button"]
t=392080  click [data-testid="next-button"]      <-- 175 ms later
t=392102  GET /services/databaseServices/name/…  <-- wizard data arrives AFTER the click
t=392316  waiting for getByTestId('schedular-schedule') … 60 s timeout

Checklist:

  • I have read the CONTRIBUTING document.

  • My PR title is Fixes <issue-number>: <short explanation>

  • My PR is linked to a GitHub issue via Fixes #<issue-number> above.

  • I have commented on my code, particularly in hard-to-understand areas.

  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed. (No schema changes.)

  • For UI changes: I attached a screen recording and/or screenshots above. (Explained above — no visual change.)

  • I have added tests (unit / integration / Playwright as applicable) and listed them above.

  • I have added a test that covers the exact scenario we are fixing. For complex issues, comment the issue number in the test for future reference.

🤖 Generated with Claude Code

Greptile Summary

The PR fixes ingestion-wizard readiness and dashboard-service loading behavior.

  • Disables the add/edit wizard’s Next button until the lazily loaded ingestion form is ready.
  • Separates dashboard data-model count state from the entity table’s loading and paging state.
  • Updates Playwright helpers and selectors to wait for form readiness and target current controls.
  • Adds unit and end-to-end regression coverage for the corrected flows.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the shared-loader issue is resolved by giving the entity fetch exclusive ownership of its loading state, and the previously questioned remount path is synchronous after the lazy modules have resolved.

Important Files Changed

Filename Overview
openmetadata-ui/src/main/resources/ui/src/components/Settings/Services/AddIngestion/AddIngestion.component.tsx Propagates active-step readiness while preserving safe synchronous remount behavior after lazy modules resolve.
openmetadata-ui/src/main/resources/ui/src/components/Settings/Services/Ingestion/IngestionWorkflowForm/IngestionWorkflowForm.tsx Signals readiness from inside the Suspense boundary and suppresses RJSF’s unintended default submit button.
openmetadata-ui/src/main/resources/ui/src/pages/AddIngestionPage/AddIngestionPage.component.tsx Gates the external wizard Next button on the active step’s readiness.
openmetadata-ui/src/main/resources/ui/src/pages/EditIngestionPage/EditIngestionPage.component.tsx Applies the same readiness gate to the edit-ingestion wizard.
openmetadata-ui/src/main/resources/ui/src/pages/ServiceDetailsPage/ServiceDetailsPage.tsx Removes dashboard count requests from shared entity-loader ownership and resets only data-model paging after failure.
openmetadata-ui/src/main/resources/ui/playwright/utils/serviceFormUtils.ts Updates react-aria one-of selection to open the visible wrapper and choose from the active popover.
openmetadata-ui/src/main/resources/ui/playwright/utils/serviceIngestion.ts Adds a reusable wait that synchronizes wizard interactions with ingestion-form readiness.

Reviews (4): Last reviewed commit: "Merge branch 'main' into fix-serviceinge..." | Re-trigger Greptile

…e loader

Fixes #30704

The Add/Edit agent wizard's footer `next-button` had no disabled state, so
pressing it before the lazily loaded RJSF templates resolved called
`submit()` against a null form ref and silently did nothing. Report step
readiness from inside the Suspense boundary and disable the button until the
form is mounted. Also suppress the stray RJSF default submit button that
appeared because a falsy `children` expression reads as "no children".

`fetchDashboardsDataModel` never cleared `isServiceLoading`, so switching
tabs on a dashboard service left the still-mounted entity tab's table
spinner running for the rest of the page's life.

On the Playwright side, add `waitForIngestionWorkflowForm` and use it at
every step-1 advance, scope the agent-tab loader waits the way
`addIngestionPipeline` already does, move the dbt config source onto the
react-aria `oneOf` select, and replace the two `submit-btn` clicks that no
longer resolve. `selectOneOfOption` now opens the select by clicking its
wrapper instead of react-aria's visually hidden native combobox, which
never opened the listbox.

Co-Authored-By: Claude <noreply@anthropic.com>
@aniketkatkar97
aniketkatkar97 requested a review from a team as a code owner July 30, 2026 12:58
Copilot AI review requested due to automatic review settings July 30, 2026 12:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

✅ PR checks passed

The linked issue has a description and all required Shipping project fields set. Thanks!

@github-actions github-actions Bot added safe to test Add this label to run secure Github workflows on PRs UI UI specific issues labels Jul 30, 2026
@aniketkatkar97 aniketkatkar97 moved this to In Review / QA 👀 in Shipping Jul 30, 2026
Copilot AI review requested due to automatic review settings July 30, 2026 13:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

`fetchDashboardsDataModel` only feeds the Data Model tab's count label, but
it was also driving `isServiceLoading` and, on error, the entity list paging
— both of which belong to `getOtherDetails`. Sharing the flag meant whichever
request settled first decided the entity table's spinner, so a fast count
query could clear it while the entity list was still in flight.

Dropping the writes fixes the stuck spinner without introducing that race,
and is simpler than clearing the flag in a `finally`.

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 30, 2026 13:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 65%
66% (77565/117518) 49.95% (46804/93685) 51.15% (14073/27511)

@aniketkatkar97
aniketkatkar97 added this pull request to the merge queue Jul 31, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 31, 2026
@aniketkatkar97
aniketkatkar97 added this pull request to the merge queue Jul 31, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🚦 Removed from the merge queue — failed_checks (2026-07-31T18:51:35Z)

These checks failed on merge-queue commit 81dc87d:

@ShaileshParmar11
ShaileshParmar11 added this pull request to the merge queue Aug 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

🚦 Removed from the merge queue — failed_checks (2026-08-01T06:20:52Z)

These checks failed on merge-queue commit d9441e4:

@ShaileshParmar11
ShaileshParmar11 added this pull request to the merge queue Aug 1, 2026
chirag-madlani pushed a commit that referenced this pull request Aug 1, 2026
…ock the merge queue (#30784)

* ci(playwright): raise the chromium shard budget to 21 minutes

The chromium lane outgrew a 19-minute shard. At the COMMON_MAX_SHARDS
ceiling of 24 the heaviest shard is predicted at 19.2m, so
assign_lane_within_budget() raises SystemExit and full-mode planning
aborts before a single test runs. Every merge_group run today failed
this way (PRs #30705, #30768, #30458, #30725, #30754), while
pull_request_target runs pass because targeted selection is far smaller.

Raise COMMON_SHARD_BUDGET_MS from 19m to 21m. At 24 shards the heaviest
is 19.2m, so the loop is guaranteed to converge at or before the
ceiling. 21m stays inside the 25m `timeout` wrapper around
`npx playwright test` and the 35m playwright-ci-postgresql job clock,
leaving ~4m of headroom.

Note the common lane now sits 1m above the dedicated lanes rather than
1m below. The strict 20-minute TARGET_MS ceiling is unaffected: it
bounds a single atomic unit, not a shard, so a 21m shard built from
units each under 20m does not trip it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(playwright): cover full-mode allocator convergence at the shard ceiling

Addresses review on #30784.

assign_lane_within_budget() was only exercised in "targeted" mode, so
neither the full-mode convergence path nor the SystemExit at
COMMON_MAX_SHARDS had coverage -- the exact code path that took the
merge queue down. Add both:

- test_full_mode_chromium_converges_at_the_shard_ceiling builds a lane
  that needs the window above 19m and asserts the allocator converges
  at or before the ceiling. Verified as a real guard: with the budget
  reverted to 19m it fails with "needs more than 24 shards ... heaviest
  shard is predicted at 20.4m".
- test_full_mode_chromium_reports_a_lane_the_ceiling_cannot_hold pins
  the SystemExit path, which had no coverage at all.

Also reword the budget comment: ~4m of headroom is relative to the 25m
playwright timeout wrapper specifically, not to the 35m job clock, which
is looser and additionally absorbs setup/teardown.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

🚦 Removed from the merge queue — failed_checks (2026-08-01T08:43:34Z)

These checks failed on merge-queue commit 79a6dd4:

@chirag-madlani
chirag-madlani added this pull request to the merge queue Aug 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

🚦 Removed from the merge queue — failed_checks (2026-08-01T14:08:56Z)

These checks failed on merge-queue commit 7787f89:

@ShaileshParmar11
ShaileshParmar11 added this pull request to the merge queue Aug 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 2, 2026
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

🚦 Removed from the merge queue — failed_checks (2026-08-02T01:09:28Z)

These checks failed on merge-queue commit 4959d00:

@ShaileshParmar11
ShaileshParmar11 added this pull request to the merge queue Aug 2, 2026
Merged via the queue into main with commit 749d226 Aug 2, 2026
84 of 85 checks passed
@ShaileshParmar11
ShaileshParmar11 deleted the fix-serviceingestion-nightly-failures branch August 2, 2026 10:05
@github-project-automation github-project-automation Bot moved this from In Review / QA 👀 to Done ✅ in Shipping Aug 2, 2026
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Changes have been cherry-picked to the 2.0 branch.

github-actions Bot pushed a commit that referenced this pull request Aug 2, 2026
…tuck dashboard service loader (#30705)

* fix(ui): gate agent wizard Next on form readiness, clear stuck service loader

Fixes #30704

The Add/Edit agent wizard's footer `next-button` had no disabled state, so
pressing it before the lazily loaded RJSF templates resolved called
`submit()` against a null form ref and silently did nothing. Report step
readiness from inside the Suspense boundary and disable the button until the
form is mounted. Also suppress the stray RJSF default submit button that
appeared because a falsy `children` expression reads as "no children".

`fetchDashboardsDataModel` never cleared `isServiceLoading`, so switching
tabs on a dashboard service left the still-mounted entity tab's table
spinner running for the rest of the page's life.

On the Playwright side, add `waitForIngestionWorkflowForm` and use it at
every step-1 advance, scope the agent-tab loader waits the way
`addIngestionPipeline` already does, move the dbt config source onto the
react-aria `oneOf` select, and replace the two `submit-btn` clicks that no
longer resolve. `selectOneOfOption` now opens the select by clicking its
wrapper instead of react-aria's visually hidden native combobox, which
never opened the listbox.

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix checkstyle

* fix(ui): stop the data model count fetch from owning entity tab state

`fetchDashboardsDataModel` only feeds the Data Model tab's count label, but
it was also driving `isServiceLoading` and, on error, the entity list paging
— both of which belong to `getOtherDetails`. Sharing the flag meant whichever
request settled first decided the entity table's spinner, so a fast count
query could clear it while the entity list was still in flight.

Dropping the writes fixes the stuck spinner without introducing that race,
and is simpler than clearing the flag in a `finally`.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit 749d226)
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Failed to cherry-pick changes to the 1.13 branch.
Please cherry-pick the changes manually.
You can find more details here.

@gitar-bot

gitar-bot Bot commented Aug 2, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Gates the agent wizard Next button on lazy-loaded form readiness and ensures dashboard service loading states clear correctly when switching tabs. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar | Powered by Gitar — free for open source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs To release Will cherry-pick this PR into the release branch UI UI specific issues

Projects

Status: Done ✅

Development

Successfully merging this pull request may close these issues.

ServiceIngestion nightly AUT failures: agent wizard Next is a no-op while the form lazy-loads, and dashboard services leave a table loader spinning

3 participants