Add generic e2e infrastructure for tier-gated and experimental features - #8314
Add generic e2e infrastructure for tier-gated and experimental features#8314hatton wants to merge 11 commits into
Conversation
An end-to-end test could not yet reach three kinds of Bloom behaviour: a feature behind a subscription tier, a feature behind an experimental-feature token, and anything that opens a native file or folder chooser. This adds the generic pieces, with nothing specific to any one feature. Bloom side: - New --experimental-features <tokens> switch, accepted only with --e2e. The tokens count as enabled for that run and are never written to user.config, which every Bloom of one build shares. - BloomOpenFileDialog and BloomFolderChooser answer with a path an e2e test armed through the new e2e/nextFileToChoose route, instead of showing a dialog. Honoured only under --e2e; the armed path is used once, and a missing file or one the filter refuses throws. - Under --e2e, Bloom no longer probes port 5173 for a Vite dev server, and refuses camera and microphone permission requests, which on a fresh browser profile would otherwise put up a WebView2 prompt no test can dismiss. - data-testid on the canvas tool's palette items and on LocalizableMenuItem, which also marks a tier-gated item with data-subscription-gated, because such an item stays clickable and so carries no disabled class. BloomE2E side: - launchBloom/bloomTest take subscriptionCode and experimentalFeatures options; kEnterpriseSubscriptionCode is the existing Test-727011-1339 code from SubscriptionTests.cs. The launcher picks the newest Bloom.exe. - New helpers: toolbox (open a tool, list the shown tools), canvasElements (open the Canvas tool by clicking the canvas, drag palette items, element menu and toolbar), keys (type key by key), geometry, videos (import a video through the Sign Language tool with the chooser hook), and getFeatureStatus. - AUTOMATION-DEBT.md updated: key events and the video import now have routes; two new entries for the anonymous canvas toolbar buttons and the palette drag that hangs with a real mouse. The Test branding stamps a butterfly watermark on every page so a screenshot made with the test subscription is recognisable. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
[Claude Fable 5.1 from Hatton's machine during preflight] Consulted Devin on 2026-09-04 22:15 UTC up to commit abbe372. Devin found no bugs. It raised one Investigate flag, about the palette drag being dispatched rather than driven with the real mouse; that is a documented exception, and the thread records why: #8314 (comment). No informational items. CI (pr-automation) passed. Devin review: https://app.devin.ai/review/BloomBooks/BloomDesktop/pull/8314 |
The branch that motivated this infrastructure grew a few more helpers that have nothing to do with tables. This brings over the generic parts, with the table-specific wording and code left behind. - helpers/origami.ts: drive the Edit tab's Change Layout mode: toggle it, count the sections, split one, read and choose the types an empty section offers. - helpers/pageSize.ts: read and change the book's page size and orientation through the layout-choice API. - helpers/images.ts: chooseImageFile, getImagePlacement and cropImage take an optional `within` locator, so a test can act on a picture box other than the page's first one. - helpers/bookMaking.ts: reloadPageBeingEdited leaves the page and comes back, so the page rebuilds its editing surfaces. - helpers/workspace.ts: canUndo and undo, which call the front end's workspaceBundle the way the WinForms shell does for Ctrl+Z, a key press the browser never receives. - helpers/publish.ts: the preview wait looks for a page of the book, not the language chooser, which a single-language book never shows. - PAPERCUTS.md: an e2e test cannot see a data-testid just added to the front end, because the launched Bloom reads output/browser. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Master added its own e2e hook for pre-answering the native file chooser (e2e/nextChosenFile, stored in E2eTestingApi and consumed only by FileIOApi.SelectFileUsingDialog). This branch already has one that covers every chooser, file and folder alike, because it lives in BloomOpenFileDialog and BloomFolderChooser themselves (e2e/nextFileToChoose). Resolved by keeping the one hook: the endpoint master added is gone, armFileChooser in helpers/talkingBook.ts posts to e2e/nextFileToChoose, and FileIOApi keeps master's rule of not remembering a test's folder in FilePathMemory, now as a guard on the normal path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…lpers canUndo compared the front end's answer with "", but workspaceRoot.ts answers "yes" or "fail", so the helper said yes whenever it was asked and undo's "nothing to undo" guard could never fire. It now looks for "yes", as the C# shell does. undo's comment no longer promises a wait it did not do; the caller waits for the state it expects. Leaving Change Layout mode waited for an editable text box to appear, but a page whose sections are all pictures or videos has none, so the helper hung for 60 seconds on a page that was fine. It now marks the page element in a JavaScript property before the click and waits for a page element without the mark, which is what Bloom's rebuild produces. The property is not markup, so it cannot be saved into the book. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nd menus say when their feature status is pending Under --e2e, TokensOfEnabledFeatures returned the developer's saved experimental features together with the ones the command line named, so a test that needed a feature off could not get it off, and a run depended on whatever the developer had turned on. It now returns only the command-line tokens under --e2e; the saved setting is untouched and still applies to a normal Bloom. The tests cover both the saved feature not counting and an empty command line enabling nothing. A tier-gated menu item looked enabled until its feature status arrived, so a test reading the canvas element menu the moment it opened could report a gated command as available. LocalizableMenuItem now marks such an item with data-feature-status-pending until the answer is in, and getCanvasElementMenuItems waits for the mark to clear before reading. The launcher's comment and the README described the port 5173 probe that --e2e no longer makes. The launcher now logs which Bloom.exe it runs and when it was built, because findBloomExe takes the newest build in any configuration. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
SetValue asked IsFeatureEnabled whether a feature was already saved, but under --e2e that answers from the command line and ignores the saved setting, so a feature that was saved looked absent and was appended again on every save. It now asks the saved setting itself. Found by Devin's review of the previous commit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
[Claude Fable 5.1 from Hatton's machine during preflight] Consulted Devin (review page) up to
Devin's report for each later commit repeats the earlier findings unchanged alongside the new ones. CI (pr-automation) passed for every commit. CodeRabbit has a config file in the repo but posted nothing. |
The launched Bloom serves its front end from output/browser, and nothing rebuilds that bundle when a .tsx file changes or a merge brings in someone else's; the same goes for Bloom.dll and the C# source. A run against a stale build fails the way a real regression does, in a test whose feature is not in the build yet, and nothing in the output says so. Today a whole-suite run failed one of master's own tests that way, against a bundle a day older than the commit the test needed. The fixture now compares the newest build output with the newest source file on each side before it launches Bloom, and stops the run naming the newer file and what to do about it. The bundle side counts only the files the build writes, because a running Bloom writes a history database into a template book inside the bundle folder, and that would make a stale bundle look fresh. The bundle check is skipped when BLOOM_E2E_VITE_PORT is set, since a dev server then serves the working tree. Checked once per worker process. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s and docs
IsFeatureEnabled asked whether the list of enabled tokens contained the
feature's name as a substring, so a token that merely contained it
("not-team-collections") enabled Team Collections. Both it and SetValue
now split the list and compare whole tokens. A test covers it.
The stale-build check counted every file under src/BloomBrowserUI and
src/BloomExe as source, so editing a spec, a story or a Markdown file
stopped the suite for a bundle that was current. Those files are not
part of what the build produces and are now left out. The refusal also
no longer tells an agent to run pnpm build, which AGENTS.md reserves for
the developer; it names the dev server route first and says the rebuild
is the developer's call. Both found by Devin.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The check skipped every Markdown file, but the build compiles the help and info pages from Markdown, so an edit to one of those left a stale bundle looking current. Markdown under help/ and infoPages/ now counts. The check also missed .uitest.ts files and the folders that hold only tests and their fixtures, so an edit there stopped the suite for a bundle that was fine. Those are now left out. Found by Devin. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The build compiles Markdown from the templates' ReadMe files and from the top of DistFiles as well as from the front-end tree, and the check only scanned the front-end tree, so an edit to one of those left a stale bundle looking current. The scan now covers all three. Found by Devin. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Problem
An end-to-end test (the Playwright suite in
src/BloomE2E) could not reach three kinds of Bloom behaviour. A feature behind a subscription tier stayed hidden, because the test collection had no subscription. A feature behind an experimental-feature token could only be turned on by writing the shareduser.config, which would also turn it on for the developer's own Bloom. And anything that opens a native file or folder chooser hung the run, because no automation can drive that dialog. A branch that needed all three had built them for itself, along with helpers for the Canvas tool, page layout, page size, images and undo; this PR lands the generic parts on their own, with nothing specific to any one feature.What the PR does
--experimental-features <tokens>together with--e2e. Under--e2ethe enabled features are exactly those tokens: they are never saved, and the developer's own saved experiments do not reach the run, so a test that needs a feature off gets it off by not naming it. Saving a feature from such a run leaves the saved setting as it was.e2e/nextFileToChooseroute.BloomOpenFileDialogandBloomFolderChooseranswer with that path instead of showing a dialog, only under--e2e, once per arming. A missing file or one the filter would refuse throws. This is now the one such hook: the narrower one master had added for the Talking Book import goes through it, and under--e2ethe file API no longer remembers a test's folder in the developer's machine-wide settings.--e2e, Bloom no longer probes port 5173 for a Vite dev server, and refuses camera and microphone permission requests, which on a fresh browser profile would otherwise show a WebView2 prompt no test can dismiss.subscriptionCodeandexperimentalFeaturesoptions, logs whichBloom.exeit runs and when that file was built, and refuses to launch a Bloom whose front-end bundle orBloom.dllis older than its source, naming the newer file. A stale build would otherwise fail a test exactly the way a regression does. Tier-gated specs use the existing test code fromSubscriptionTests.cs; the Test branding now stamps a butterfly on every page so such a book is visibly a test book.Devin review
This change is