Skip to content

Assert durable state instead of self-dismissing toasts in the desktop E2E suite #185

Description

@Azganoth

Summary

The assembled-desktop E2E specs assert on a toast that dismisses itself after 5 seconds. On a slow run the affordance is gone before the assertion resolves, and the suite fails even though the behavior under test succeeded. Replace the transient assertions with the durable state they stand in for, and audit the remaining specs for the same pattern.

This blocks running the suite in CI: the flake is the only thing preventing an unfiltered pull request trigger.

Affected areas

  • e2e/desktop/specs/document-lifecycle.e2e.ts — the confirmed instance.
  • e2e/desktop/specs/ — the other six specs, unaudited for the same pattern.

Intended outcome

Each spec asserts state that persists after the operation completes — on-disk contents, menu item state, editor contents, diagnostic records — rather than notification UI that expires on a timer. The scenarios keep their current coverage of the assembled boundary; nothing about what they exercise changes, only what they read to decide the operation happened.

Toast rendering itself is not the subject of these tests. If a scenario needs to prove a notification appeared at all, it should say so explicitly and tolerate the dismissal window rather than race it.

Related context

Done when

  • document-lifecycle no longer asserts the save-success toast, and the save is proven through the durable state the spec already checks.
  • The remaining six specs are audited for self-dismissing-affordance assertions, and any found are given the same treatment.
  • pnpm test:e2e:desktop passes locally from a clean baseline.
  • The forced-failure path still produces evidence and a nonzero exit code, per the CONTRIBUTING.md procedure.

Notes

The confirmed instance is document-lifecycle.e2e.ts line 33. The failing run is 31329580918. Its uploaded fixture-manifest.json recorded the document's actual sha256 equal to its expected sha256, and the failure screenshot showed the editor holding the saved marker — the save crossed the native boundary correctly and only the toast was missing. The spec took 12.7s in that run against 4.7s when passing.

Toasts auto-dismiss at Base UI's default 5000 ms, and src/lib/toast.ts sets no override.

Implementation direction

document-lifecycle already asserts the two durable outcomes immediately after the toast check: the on-disk contents reaching savedMarkdown, and the Save menu item returning to disabled. The toast assertion is the least reliable and least necessary of the three, and removing it should not need a replacement.

Out of scope

  • WebdriverIO retries or any other retry mechanism. Retrying masks the intermittent assembled-boundary regressions the suite exists to catch.
  • New scenarios or broader coverage.
  • Adding the CI job, which is tracked separately.

Metadata

Metadata

Assignees

Labels

MaintenanceChore, refactor, dependency, or test work without user-facing change

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions