Skip to content

test(ci): harden cross-platform reliability gates - #137

Merged
slashdevcorpse merged 12 commits into
mainfrom
agent/cross-platform-test-hardening
Jul 22, 2026
Merged

test(ci): harden cross-platform reliability gates#137
slashdevcorpse merged 12 commits into
mainfrom
agent/cross-platform-test-hardening

Conversation

@slashdevcorpse

@slashdevcorpse slashdevcorpse commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Summary

  • run unit coverage across Linux, Windows, and macOS, with blocking Windows browser geometry and Linux/Windows packaged desktop lanes
  • enforce a bounded, source-registered quarantine inventory and surface platform-scoped summaries
  • add deterministic packaged desktop journeys with fake-provider, loopback, cleanup, and failure-diagnostic safeguards
  • harden browser-process lifecycle, bounded WebSocket recovery, exact project-create recovery, and file-read capacity retry behavior

Verification

  • Windows packaged desktop E2E: 7 passed
  • critical create/send repeat: 3 passed
  • focused web tests: 89 passed
  • focused desktop tests: 59 passed
  • focused script tests: 62 passed
  • focused server tests: 23 passed
  • live Windows quarantine inventory: all 62 browser files, 1 registered group, and 11 collected cases passed
  • workflow contract validation: 7 allowed and 4 disabled paths passed
  • quarantine structural validation passed
  • desktop source build passed
  • frozen lockfile install passed
  • git diff checks and production-seam checks passed

Full formatting, lint, and typecheck gates are delegated to hosted CI as required by the repository instructions.


Summary by cubic

Hardens cross-platform CI with a source-registered browser quarantine and deterministic packaged Electron E2E on Windows and Linux. Improves Windows stability with capped Vitest workers, extended targeted timeouts, and safer browser/desktop lifecycles with one-shot WS recovery.

  • New Features

    • Source-registered browser quarantine using [quarantine:<id>] in .github/quarantine.yml; Vitest derives stable/quarantine sets per platform; CI validates the registry, inventories the full suite, and posts platform-scoped summaries.
    • Packaged Electron desktop E2E via @playwright/test with a fake provider, loopback-only network guard, process-tree teardown, redacted failure diagnostics, and prebuilt-artifact checks; wired into Windows/Linux CI; repo test:e2e script added.
    • CI hardening: per-platform quality lanes on Linux/Windows/macOS, pinned actions with artifact allowlists (build outputs and redacted desktop E2E failure summaries only), browser runtime install/inventory, LF line ending normalization, --maxWorkers=50% across packages, and extended Windows staging/startup budgets.
  • Bug Fixes

    • Desktop BrowserManager: guard stale load promises, prevent adopted webviews from overwriting targets, and close renderer-owned runtimes on teardown to avoid orphaned processes.
    • WebSocket transport: one-shot recovery for safe unary calls on session interruption with targeted reconnect; enabled in wsNativeApi (server refresh, orchestration dispatch, shell reads); tests for shouldRecoverUnaryRequest.
    • Project create: recover when the command commits but the response fails; ignore archived projects during recovery.
    • Project file-read: bounded retries on capacity-exceeded errors with isProjectReadFileCapacityError and updated React Query options.
    • Terminal search: trust authoritative result-change events to avoid stale “no results” after a debounced miss.

Written for commit ddbb014. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Bug Fixes
    • Improved project recovery to exclude archived projects during duplicate project.create handling.
    • Refined project create/recover behavior to better preserve the right error and avoid incorrect recovery.
    • Added WebSocket session-interruption retry for selected orchestration/session operations.
    • Improved desktop browser tab state and teardown to prevent stale/inconsistent loading after transitions.
  • Quality
    • Expanded desktop end-to-end coverage with stronger diagnostics, log redaction, and safer process shutdown.
    • Strengthened browser test quarantine/registry validation and updated quarantined geometry coverage.
  • Chores
    • Standardized cross-platform text line endings and updated CI/test configurations/artifact ignores.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR adds browser quarantine tooling, restructures CI and packaged desktop E2E jobs, introduces a guarded fake-Codex Electron harness, and updates WebSocket recovery, project recovery, file-read retries, adopted webview handling, and test support.

Changes

CI and quarantine

Layer / File(s) Summary
Quarantine registry and workflow gates
.github/quarantine.yml, scripts/..., .github/workflows/ci.yml
Adds platform-scoped quarantine validation, inventory and summary commands, browser CI lanes, desktop E2E artifact flow, and aggregate quality gating.
Registry and workflow validation
scripts/lib/*test.ts, apps/web/vitest.browser.*
Tests registry reconciliation, selectors, workflow structure, artifact paths, and quarantine enforcement.

Desktop E2E

Layer / File(s) Summary
Packaged Electron harness and journeys
apps/desktop/e2e/*
Adds isolated Electron setup, fake Codex protocol flows, seven desktop journeys, diagnostics, network controls, and Playwright wiring.
Identity-verified teardown
apps/desktop/e2e/support/processTree.*
Adds cross-platform process snapshots, ancestry validation, descendant collection, signaling, and teardown tests.

Web runtime and browser lifecycle

Layer / File(s) Summary
WebSocket and project recovery
apps/web/src/ws*.ts, apps/web/src/lib/*
Adds one reconnect-and-replay attempt for selected unary RPCs, committed-project recovery, archived-project filtering, and capacity-specific file-read retries.
Adopted webview lifecycle
apps/desktop/src/browserManager*
Closes renderer-owned webviews during disposal, ignores stale load failures, and preserves adopted guest URL and loading state.

Platform and test support

Layer / File(s) Summary
Cross-platform test configuration
.gitattributes, .gitignore, package.json, apps/*/package.json, scripts/*
Normalizes text endings, ignores E2E outputs, adds Playwright commands, adjusts Windows smoke timing, constrains Vitest workers, and updates supporting test behavior.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.64% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly matches the main change: cross-platform CI and reliability hardening.
Description check ✅ Passed The description covers the main changes and verification, but it omits the template's explicit Why/UI Changes/Checklist sections.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
scripts/quarantine-registry.ts (1)

146-150: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract the runtime-platform mapping into the shared registry module.

runtimeQuarantinePlatform duplicates quarantinePlatform in apps/web/vitest.browser.quarantine.ts (both map process.platformQuarantinePlatform | null). Since both files already import from ./lib/quarantine-registry, hoist a single helper there so the runtime→platform mapping cannot drift between CI and the browser harness.

As per coding guidelines: "avoid duplicating logic across files and prefer reusable modules over local shortcuts."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/quarantine-registry.ts` around lines 146 - 150, Move the
process.platform-to-QuarantinePlatform mapping from runtimeQuarantinePlatform
into the shared quarantine-registry module, and export the reusable helper
there. Update both scripts/quarantine-registry.ts and
apps/web/vitest.browser.quarantine.ts to import and use that shared helper,
removing the duplicate local implementations while preserving the null result
for unsupported platforms.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/desktop/e2e/desktop.fixture.ts`:
- Around line 957-968: Update the nonNodeInvocationPids filter in the
invocationEvents flatMap so runtime.executable is accessed safely when runtime
is null. Use optional chaining or an explicit runtime guard before
Path.basename, while preserving the existing Node executable exclusion behavior.

In `@apps/desktop/e2e/fixtures/electron-bootstrap.cjs`:
- Around line 85-88: Update the NODE_OPTIONS assignment in the Electron
bootstrap initialization to preserve the existing process.env.NODE_OPTIONS value
while adding the escaped networkGuardPath --require option. Ensure inherited CI
and tooling flags remain available when main.ts builds backend child arguments,
and avoid replacing the existing value.

In `@apps/desktop/e2e/fixtures/fake-codex.ts`:
- Around line 123-127: Replace the synchronous FS.readFileSync(0, "utf8") call
in the text-generation fixture flow with asynchronous consumption of
process.stdin, awaiting the stream until it ends while discarding its contents.
Preserve the existing protocol log ordering and only write the output after
stdin consumption completes.
- Line 8: Make the side-effect import in fake-codex type-safe by adding the
project’s appropriate declaration/shim for the network-guard.cjs module, or
rename the fixture to a .cts file if that matches the existing module setup.
Preserve the import’s runtime behavior and ensure TypeScript no longer reports
an unresolved .cjs module.

---

Nitpick comments:
In `@scripts/quarantine-registry.ts`:
- Around line 146-150: Move the process.platform-to-QuarantinePlatform mapping
from runtimeQuarantinePlatform into the shared quarantine-registry module, and
export the reusable helper there. Update both scripts/quarantine-registry.ts and
apps/web/vitest.browser.quarantine.ts to import and use that shared helper,
removing the duplicate local implementations while preserving the null result
for unsupported platforms.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b4939788-de99-4a53-ae88-62902c4c152f

📥 Commits

Reviewing files that changed from the base of the PR and between 1fcda84 and a843d29.

⛔ Files ignored due to path filters (1)
  • bun.lock is excluded by !**/*.lock
📒 Files selected for processing (54)
  • .gitattributes
  • .github/quarantine.yml
  • .github/workflows/ci.yml
  • .gitignore
  • apps/desktop/e2e/desktop.e2e.ts
  • apps/desktop/e2e/desktop.fixture.test.ts
  • apps/desktop/e2e/desktop.fixture.ts
  • apps/desktop/e2e/fixtures/diagnostic-redaction.cjs
  • apps/desktop/e2e/fixtures/electron-bootstrap.cjs
  • apps/desktop/e2e/fixtures/fake-codex.ts
  • apps/desktop/e2e/fixtures/loopback.cjs
  • apps/desktop/e2e/fixtures/network-guard.cjs
  • apps/desktop/e2e/global-setup.ts
  • apps/desktop/e2e/playwright.config.ts
  • apps/desktop/e2e/support/failureDiagnostics.ts
  • apps/desktop/e2e/support/processTree.test.ts
  • apps/desktop/e2e/support/processTree.ts
  • apps/desktop/e2e/tsconfig.json
  • apps/desktop/package.json
  • apps/desktop/scripts/smoke-test-windows-job.windows.integration.test.mjs
  • apps/desktop/src/browserManager.test.ts
  • apps/desktop/src/browserManager.ts
  • apps/server/src/provider/windowsJobLauncherSourceContract.test.ts
  • apps/web/BROWSER_TEST_QUARANTINE.md
  • apps/web/package.json
  • apps/web/src/components/ChatMarkdown.test.tsx
  • apps/web/src/components/ChatView.browser.tsx
  • apps/web/src/components/Sidebar.import.test.ts
  • apps/web/src/components/chat/MessagesTimeline.test.tsx
  • apps/web/src/lib/projectCreateRecovery.test.ts
  • apps/web/src/lib/projectCreateRecovery.ts
  • apps/web/src/lib/projectCreation.test.ts
  • apps/web/src/lib/projectCreation.ts
  • apps/web/src/lib/projectReactQuery.test.ts
  • apps/web/src/lib/projectReactQuery.ts
  • apps/web/src/wsNativeApi.test.ts
  • apps/web/src/wsNativeApi.ts
  • apps/web/src/wsTransport.test.ts
  • apps/web/src/wsTransport.ts
  • apps/web/vitest.browser.geometry.config.ts
  • apps/web/vitest.browser.quarantine.ts
  • apps/web/vitest.browser.stable.config.ts
  • package.json
  • packages/contracts/package.json
  • packages/shared/package.json
  • scripts/benchmark-windows-command-discovery.test.ts
  • scripts/lib/quarantine-registry.test.ts
  • scripts/lib/quarantine-registry.ts
  • scripts/lib/release-worktree-cleanliness.test.ts
  • scripts/lib/windows-authenticode.test.ts
  • scripts/lib/workflow-contracts.test.ts
  • scripts/lib/workflow-contracts.ts
  • scripts/package.json
  • scripts/quarantine-registry.ts

Comment thread apps/desktop/e2e/desktop.fixture.ts
Comment thread apps/desktop/e2e/fixtures/electron-bootstrap.cjs
Comment thread apps/desktop/e2e/fixtures/fake-codex.ts
Comment thread apps/desktop/e2e/fixtures/fake-codex.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread scripts/lib/workflow-contracts.test.ts
Comment thread scripts/lib/workflow-contracts.test.ts
Comment thread scripts/lib/workflow-contracts.test.ts
Comment thread apps/desktop/e2e/fixtures/electron-bootstrap.cjs
Comment thread apps/web/src/lib/projectCreation.ts Outdated
Comment thread apps/desktop/e2e/fixtures/loopback.cjs Outdated
@coderabbitai coderabbitai Bot added the ready-to-merge Approved for Mergify auto-merge after required checks pass label Jul 22, 2026

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@coderabbitai coderabbitai Bot removed the ready-to-merge Approved for Mergify auto-merge after required checks pass label Jul 22, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed against the latest diff

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread scripts/quarantine-registry.ts
Comment thread apps/web/src/lib/projectReactQuery.ts
Comment thread apps/desktop/e2e/desktop.e2e.ts Outdated

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="scripts/lib/quarantine-registry.test.ts">

<violation number="1" location="scripts/lib/quarantine-registry.test.ts:85">
P3: The test name references "generated route-tree staging" but the function under test is `createQuarantineInventoryTemporaryDirectory` which creates a quarantine-inventory temp directory under `.tanstack/`. The route-tree concept belongs to the previous test's `createQuarantineInventoryEnvironment` and `SYNARA_GENERATED_ROUTE_TREE`. Consider renaming to something like "keeps quarantine inventory staging inside .tanstack" to avoid confusing readers.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread scripts/lib/quarantine-registry.test.ts

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slashdevcorpse has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@slashdevcorpse
slashdevcorpse merged commit 81350d4 into main Jul 22, 2026
21 of 23 checks passed
@slashdevcorpse
slashdevcorpse deleted the agent/cross-platform-test-hardening branch July 22, 2026 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant