feat(onboarding): wire the verify console to the real first-evaluation signal

[talissoncosta]

• I have read the Contributing Guide.
• I have added information to docs/ if required so people know about the feature.
• I have filled in the "Changes" section below.
• I have filled in the "How did you test this code" section below.

Changes

Contributes to #7767, consuming the first-evaluation signal from #7623.

The verify console was stubbed: it read a ?connected query param, so the connected state could be faked from the URL. It now reflects the real signal.

• New RTK Query service useEnvironmentOnboardingStatus, calling GET environments/{key}/onboarding-status/. Unauthenticated, keyed by the environment's client-side API key, served by Core rather than the edge host.
• useOnboardingConnection takes the environment key and polls that endpoint every 3s, flipping to connected once first_evaluated_at is no longer null. Edge reports the first evaluation asynchronously after the SDK's first request, so the flip is not instant; the value never reverts, so polling stops once it lands.
• The ?connected hatch is gone.

How did you test this code?

The e2e test (onboarding-tests.pw.ts) mocks onboarding-status/ and flips first_evaluated_at mid-test, so it now asserts through the real polling path rather than the query param.

Manually verified with the onboarding_quickstart_flow flag on:

• /getting-started holds on "listening" while nothing has evaluated
• Running the snippet in a scratch app flips the console to LIVE within a few seconds, no reload needed
• ?connected on the URL no longer does anything

tsc and eslint are clean on the changed files.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
flagsmith-frontend-preview	Ready	Preview, Comment	Jul 28, 2026 2:34pm
flagsmith-frontend-staging	Ready	Preview, Comment	Jul 28, 2026 2:34pm

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Ignored	Preview	Jul 28, 2026 2:34pm

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 46791b21-ba1b-4120-a788-0034c8591973

📥 Commits

Reviewing files that changed from the base of the PR and between 1ef00d8 and 00f0f1c.

📒 Files selected for processing (1)

• frontend/e2e/tests/onboarding-tests.pw.ts

---

📝 Walkthrough

Walkthrough

The frontend adds typed request and response models plus an RTK Query service for environment onboarding status. The onboarding connection hook polls this endpoint every three seconds until first_evaluated_at is returned, then reports a connected state. OnboardingFlow supplies the environment key, and the E2E tests mock the status response and wait for the LIVE state during onboarding interactions.

Estimated code review effort: 3 (Moderate) | ~20 minutes

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Docker builds report

Image	Build Status	Security report
ghcr.io/flagsmith/flagsmith-api-test:pr-8132	Finished ✅	Skipped
ghcr.io/flagsmith/flagsmith-e2e:pr-8132	Finished ✅	Skipped
ghcr.io/flagsmith/flagsmith-api:pr-8132	Finished ✅	Results ✅
ghcr.io/flagsmith/flagsmith:pr-8132	Finished ✅	Results ✅
ghcr.io/flagsmith/flagsmith-private-cloud:pr-8132	Finished ✅	Results ✅
ghcr.io/flagsmith/flagsmith-frontend:pr-8132	Finished ✅	Results ✅

[coderabbitai]

Actionable comments posted: 2

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 9f52ac8f-6cc0-4401-b563-21b5ed937f91

📥 Commits

Reviewing files that changed from the base of the PR and between 2e6017b and f08be1d.

📒 Files selected for processing (9)

• frontend/common/code-help/init/init-react-native.js
• frontend/common/code-help/init/init-react.js
• frontend/common/constants.ts
• frontend/common/services/useEnvironmentOnboardingStatus.ts
• frontend/common/types/requests.ts
• frontend/common/types/responses.ts
• frontend/e2e/tests/onboarding-tests.pw.ts
• frontend/web/components/pages/onboarding/OnboardingFlow/OnboardingFlow.tsx
• frontend/web/components/pages/onboarding/hooks/useOnboardingConnection.ts

[github-actions]

✅ private-cloud · depot-ubuntu-latest-16 — run #18791 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

 4 passed

Details

 4 tests across 3 suites
 11.8 seconds
 00f0f1c
 🔄 Run: #18791 (attempt 1)

🗂️ Previous results

✅ oss · depot-ubuntu-latest-arm-16 — run #18791 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

 1 passed

Details

 1 test across 1 suite
 44.6 seconds
 00f0f1c
 🔄 Run: #18791 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #18791 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

 1 passed

Details

 1 test across 1 suite
 38.5 seconds
 00f0f1c
 🔄 Run: #18791 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-arm-16 — run #18789 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

 5 passed

Details

 5 tests across 4 suites
 44.2 seconds
 1ef00d8
 🔄 Run: #18789 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-16 — run #18789 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

 3 passed

Details

 3 tests across 3 suites
 56.1 seconds
 1ef00d8
 🔄 Run: #18789 (attempt 1)

✅ oss · depot-ubuntu-latest-arm-16 — run #18789 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

 1 passed

Details

 1 test across 1 suite
 45.4 seconds
 1ef00d8
 🔄 Run: #18789 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #18789 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

 1 passed

Details

 1 test across 1 suite
 37.3 seconds
 1ef00d8
 🔄 Run: #18789 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-16 — run #18788 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

 2 passed

Details

 2 tests across 2 suites
 35.8 seconds
 266c600
 🔄 Run: #18788 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-arm-16 — run #18788 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

 3 passed

Details

 3 tests across 3 suites
 54.1 seconds
 266c600
 🔄 Run: #18788 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #18788 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

 1 passed

Details

 1 test across 1 suite
 42.3 seconds
 266c600
 🔄 Run: #18788 (attempt 1)

✅ oss · depot-ubuntu-latest-arm-16 — run #18788 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

 1 passed

Details

 1 test across 1 suite
 40.5 seconds
 266c600
 🔄 Run: #18788 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-16 — run #18786 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

 3 passed

Details

 3 tests across 3 suites
 35.8 seconds
 f08be1d
 🔄 Run: #18786 (attempt 1)

[github-actions]

Visual Regression

19 screenshots compared. See report for details.
View full report

[Zaimwa9]

@themis-blindfold review

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e1597317-b977-4192-a358-bea8b8534c25

📥 Commits

Reviewing files that changed from the base of the PR and between f08be1d and 1ef00d8.

📒 Files selected for processing (3)

• frontend/e2e/tests/onboarding-tests.pw.ts
• frontend/web/components/pages/onboarding/OnboardingFlow/OnboardingFlow.tsx
• frontend/web/components/pages/onboarding/hooks/useOnboardingConnection.ts

[themis-blindfold]

⚖️ Themis review: 🧹 Ship it, nits inside

The stub is properly gone and the replacement is sound. The hook latches first_evaluated_at in local state rather than reading it straight off data, which is the right call because a skipped RTK Query returns no data — so the connected state survives polling being switched off. The three waitConnected() calls added around the next-quest clicks are also necessary, not defensive padding: OnboardingNextSteps marks its body inert while locked, which removes the cards from the accessibility tree and makes getByRole('button') fail outright. The backend endpoint the service calls exists on main with matching field names and shape.

Two nits, both about edges the change opened up rather than the change itself. No blockers or majors.

Area	Score
🎯 Correctness	5/5
🧪 Test coverage	3/5
📐 Code quality	5/5
🚀 Product impact	4/5

🧹 Nits

• frontend/e2e/tests/onboarding-tests.pw.ts — the onboarding-renamed visual snapshot is now taken in an indeterminate console state.
• frontend/web/components/pages/onboarding/hooks/useOnboardingConnection.ts — polling has no ceiling and no error surface.

⚖️ Acknowledged

• Generated React/React Native snippets use const ${FEATURE_NAME} and dot access, which break for flag keys that aren't valid JS identifiers — thread resolved by @talissoncosta.

📝 Walkthrough

• common/services/useEnvironmentOnboardingStatus.ts — new RTK Query endpoint for GET environments/{key}/onboarding-status/, following the existing service template (tag types, initiate wrapper, hook export).
• common/types/{requests,responses}.ts — request arg (environmentKey) and the two-field response shape, matching the Core serialiser already on main.
• hooks/useOnboardingConnection.ts — takes the environment key, polls every 3s, latches connected on the first non-null first_evaluated_at and skips further polling; the ?connected URL hatch is removed.
• OnboardingFlow.tsx — passes environmentKey through; the same connection value still gates the terminal badge, the flag toggle and the next-step cards.
• e2e/onboarding-tests.pw.ts — mocks onboarding-status/ and flips first_evaluated_at mid-test, so LIVE is now reached through the real polling path; waitConnected() guards each next-quest click after a fresh page load.

🧪 How to verify

• Run VISUAL_REGRESSION=1 npx playwright test e2e/tests/onboarding-tests.pw.ts and compare onboarding-renamed against a baseline captured from main — that is where the snapshot-timing nit shows up.
• With onboarding_quickstart_flow on, open /getting-started and watch the network tab: onboarding-status/ should be requested every 3s and stop entirely once it returns a timestamp.
• Run the SDK snippet against the Dev key the panel shows and confirm the badge flips to LIVE, the flag toggle unlocks, and the next-quest cards become clickable without a reload.
• Confirm /getting-started?connected no longer unlocks anything.
• Leave /getting-started open for a few minutes without evaluating and confirm the request rate is what you intend for a signup that never connects.

Automate: a small unit test around useOnboardingConnection asserting the latch holds after data goes away would cover the one piece of real logic in the hook.

Product take: This is the change that makes the onboarding page honest — before it, the "verify your connection" step could only be satisfied by typing a query param, so the flag toggle and next-step cards were unreachable for a real signup. Solid, load-bearing improvement to the quickstart flow rather than a large capability.

🧭 Assumptions & unverified claims

• The E2E suite had not run for this head at review time — the image builds were still in progress and no E2E result was available, so the reworked test's assertions are unverified in CI. Unit tests and the changed-file lint job had completed successfully.
• The onboarding test self-skips unless onboarding_quickstart_flow is enabled in the E2E Flagsmith environment; whether it is could not be checked from the checkout.
• That RTK Query clears data when skip becomes true (the reason the local latch is required) is taken from documented library behaviour, not executed here — dependencies are not installed in this sandbox.

Deleting a URL query param is rarely this much of an upgrade. · reviewed at 1ef00d8

[talissoncosta]

Nit 1 fixed in 00f0f1ce8: moved the wait ahead of the snapshot and dropped the later one, which became redundant once the latch is held. Small correction on the mechanism, RTK Query fires its initial fetch on mount, so it wasn't a 3s gap, just a race with the first response.

[themis-blindfold]

Your correction is right, and it doesn't change the fix. useGetEnvironmentOnboardingStatusQuery fetches on mount, so after page.reload() (frontend/e2e/tests/onboarding-tests.pw.ts:133) the window before LIVE is the first response's latency, not a POLL_INTERVAL_MS gap — useOnboardingConnection.ts:22 only spaces out subsequent fetches. Either way the onboarding-renamed snapshot was taken before that response landed, so it could capture LISTENING or LIVE. Waiting first pins it.

The 3s interval does apply to the mid-test flip at line 99, where the component is already mounted — that one's covered by waitConnected's 15s timeout.

I can't see 00f0f1ce8 in this checkout (head is 1ef00d8), so this is on your description: dropping only line 150 is correct, since nothing between the reload and there remounts the hook. Keep the waitConnected calls after each gotoOnboarding — those are full navigations that reset firstEvaluated, so they aren't covered by the earlier wait.

[Zaimwa9]

Couple of non-blocking comments that could be left for later

[themis-blindfold]

⚖️ Themis review: 🧹 Ship it, nits inside

Delta re-review of the one new commit. It adds await waitConnected() before the onboarding-renamed visual snapshot and drops the now-redundant wait further down — exactly the fix for the snapshot race flagged earlier, and it introduces nothing new. The only thing still outstanding is the unbounded 3s poll, which is a Nit and already discussed on its thread.

Area	Score
🎯 Correctness	5/5
🧪 Test coverage	4/5
📐 Code quality	5/5
🚀 Product impact	4/5

🧹 Nits

• frontend/web/components/pages/onboarding/hooks/useOnboardingConnection.ts — polling has no ceiling, no skipPollingIfUnfocused, and no error surface; unchanged since the earlier review, see the existing thread.

📝 Walkthrough

• e2e snapshot ordering — the post-reload onboarding-renamed snapshot now waits for the console to flip back to LIVE before capturing. Observed: page.reload() at line 133 resets the hook's useState latch, and visualSnapshot takes an unmasked full-page screenshot, so the badge state was genuinely in the frame; the wait removes that race.
• redundant wait removed — the later waitConnected() before the Gradual rollout click is gone. Safe: the connection state latches for the page's lifetime, so it is still connected by the time the quest cards are clicked.

🧪 How to verify

• Run the onboarding e2e with visual regression on, with onboarding_quickstart_flow enabled, and confirm onboarding-renamed captures the LIVE badge.
• Run it a few times to confirm the snapshot no longer flips between LISTENING and LIVE across runs.
• Confirm the Gradual rollout / Remote config / Experiment card clicks still navigate, i.e. the cards are unlocked without the removed wait.

Automate: the existing onboarding-tests.pw.ts run plus the baseline comparison step covers this; no extra check needed.

Product take: No user-visible change in this commit — it stabilises the e2e visual baseline for the onboarding console. The PR as a whole is a solid improvement: the verify console now reflects a real first evaluation instead of a URL query param.

🧭 Assumptions & unverified claims

• The onboarding-renamed baseline on main was captured in the connected state (the pre-change test loaded /getting-started?connected before that point), so waiting for LIVE matches the baseline rather than diverging from it. Not verified against the stored baseline image.
• CI was still running at review time — only the Vercel docs build and preview-comments checks had completed — so the e2e and lint results for this commit are not yet observable.

A one-line await moved four lines up, and the screenshot finally stops guessing what it's looking at · reviewed at 00f0f1c