Skip to content

test(goldens): close 6 handbook gaps with new Goldens (stacked on #562)#568

Merged
TaprootFreak merged 3 commits into
feat/visual-regression-handbook-gapsfrom
feat/handbook-gaps-2
May 25, 2026
Merged

test(goldens): close 6 handbook gaps with new Goldens (stacked on #562)#568
TaprootFreak merged 3 commits into
feat/visual-regression-handbook-gapsfrom
feat/handbook-gaps-2

Conversation

@TaprootFreak
Copy link
Copy Markdown
Contributor

@TaprootFreak TaprootFreak commented May 23, 2026

Summary

Stacked on #562. Closes the 6 remaining handbook→Golden mapping gaps:

# Handbook page Golden added Notes
03 software-wallet-terms welcome_page_second_step @visibleForTesting initialShowSecondStep ctor param
09 pin-confirm setup_pin_page_confirming SetupPinMode.confirm state
10 biometric-prompt biometric_prompt_sheet_default Flutter EnableBiometricBottomSheet — NOT an iOS system sheet (Maestro YAML comment misleading)
11 dashboard home_page_loaded DashboardView with mocked price chart
17 settings-backup-pin verify_pin_page_seed_backup i18n-driven subtitle, no lib refactor
26 terms legal_document_page_terms_loaded @visibleForTesting initialMarkdownContent ctor param

After this PR, the handbook can switch to Golden-sourced screenshots (Phase 1) without leaving any page unmapped.

Lib refactors

Two minimal refactors, both @visibleForTesting, both with the production default unchanged:

  • lib/screens/welcome/welcome_page.dart — adds initialShowSecondStep (default false)
  • lib/screens/legal/subpages/legal_document_page.dart — adds initialMarkdownContent (null → unchanged _loadMarkdown() path)

Driving the second-step / loaded-state via tap + pumpAndSettle was tried first but is fragile in alchemist's hook chain.

Bootstrap workflow

.github/workflows/golden-bootstrap.yaml is re-introduced temporarily, push-triggered on this branch. Generates baselines on dfx01, removed once committed.

Test plan

  • Bootstrap workflow completes green on dfx01
  • Baselines downloaded + committed
  • Bootstrap workflow removed
  • Visual Regression job green on final commit
  • Spot-check each of the 6 new baselines visually matches its handbook counterpart

@TaprootFreak
test(goldens): add 6 handbook-gap baselines (gaps 03/09/10/11/17/26)
19cf0e3 
Closes the remaining handbook coverage gaps identified in the mapping
audit done on top of PR #562:

  - 03-software-wallet-terms: WelcomePage showSecondStep=true
    (minimal lib refactor — adds @VisibleForTesting initialShowSecondStep
    ctor param, production default unchanged)
  - 09-pin-confirm: SetupPinPage with SetupPinMode.confirm
  - 10-biometric-prompt: EnableBiometricBottomSheet — Flutter widget,
    NOT the iOS LocalAuthentication system sheet (the maestro YAML
    comment was misleading; verified by reading the actual handbook
    PNG, which shows Flutter typography + the app's brand color)
  - 11-dashboard: home_page_loaded — DashboardView with mocked price
    chart (~30 points, 1.10→1.43 with a mid-dip) and CHF currency
  - 17-settings-backup-pin: verify_pin_page_seed_backup with i18n-driven
    seed-backup subtitle (uses existing VerifyPinView.description ctor
    param, no lib refactor needed)
  - 26-terms: legal_document_page_terms_loaded with inlined head of
    terms_of_use_de.md (minimal lib refactor — adds @VisibleForTesting
    initialMarkdownContent ctor param, production unchanged)

The temporary golden-bootstrap workflow re-introduces baseline generation
on dfx01 for this branch; it is removed once the baselines are committed.
@TaprootFreak TaprootFreak marked this pull request as ready for review May 23, 2026 22:29
@TaprootFreak
test(goldens): commit 6 handbook-gap baselines from dfx01
9bca06b 
Outputs of golden-bootstrap.yaml run 26344998789. All 6 new PNGs
visually verified to match their handbook ground truth:

  - welcome_page_second_step (gap 03)
  - setup_pin_page_confirming (gap 09)
  - biometric_prompt_sheet_default (gap 10)
  - home_page_loaded (gap 11)
  - verify_pin_page_seed_backup (gap 17)
  - legal_document_page_terms_loaded (gap 26)

Removes .github/workflows/golden-bootstrap.yaml — done.
@TaprootFreak TaprootFreak mentioned this pull request May 23, 2026
Merged
4 tasks
@TaprootFreak
feat(handbook): assemble screenshots from Goldens, not Maestro (stack…
e482d4a 
…ed on #568) (#570)

## Summary

Stacked on #568. Phase 1 of the Maestro → Goldens handbook unification
plan: the 26 PNGs at `handbook.realunit.app/screenshots/` now come from
the same Golden baselines the visual-regression CI verifies on every PR.

## Why

- **Single source of truth**: one pixel-checked baseline per handbook
page. A UI regression that breaks a Golden also breaks the handbook
image before either ships.
- **Determinism**: dfx01's headless Skia/Open Sans rendering is
byte-stable across CI runs. Maestro's iOS-simulator screenshots drifted
on Apple Silicon + iOS 26 driver hangs (mobile-dev-inc/Maestro#3137).
- **Cycle time**: the handbook image rebuilds when Goldens change, in
seconds — no 30-minute Maestro suite to refresh a page.

## What changed

- **`scripts/assemble-handbook-screenshots.sh`** — explicit
`handbook-name → golden-path` mapping for all 26 pages (see PR #568 for
the audit that built this table). Copies and renames Goldens into a flat
output directory the Dockerfile then consumes. Fails loudly if a Golden
goes missing.
- **`Dockerfile.handbook`** — multi-stage build. Stage 1 (alpine + bash)
runs the assembly script. Stage 2 (nginx) copies the assembled
screenshots over the legacy `docs/handbook/screenshots/` tree,
preserving output paths so the handbook HTML `<img>` links work
unchanged.
- **`.github/workflows/handbook-build-check.yaml`** — new PR-only
validation. Runs the assembly script independently (cheap check before
docker spins up), builds the image, starts the container, hits
`/healthz`, verifies the auth gate returns 401 on `/de/`, and probes
`/screenshots/{01,11,26}.png` to prove the assembled files land on disk.
Does **not** push to Docker Hub and does **not** deploy. The
develop-push `handbook-deploy.yaml` retains sole ownership of DEV → PRD
rollout.

## Out of scope

- The legacy `docs/handbook/screenshots/` tree is left in place; the
COPY in Dockerfile.handbook overlays it with the assembled output, so
the legacy PNGs are harmless. **Phase 2** (Maestro pipeline retirement)
decides whether to delete the directory + flow YAMLs.
- DEV deploy verification — happens after merge to develop (the deploy
pipeline does that automatically).

## Test plan

- [ ] `Handbook Build Check` CI green on the PR (docker build +
container smoke)
- [ ] `Visual Regression` job green on the PR (parallel — no Golden
changes here, should pass trivially)
- [ ] After merge to develop: `handbook-deploy.yaml` builds the image,
DEV-deploy succeeds, `https://dev-handbook.realunit.app/de/` shows the
Goldens-sourced screenshots (spot-check 3 pages: welcome, dashboard,
terms)
- [ ] PRD deploy follows DEV-green, same spot-check on prod URL

## Pre-conditions for merge

The stack below this PR must merge in order: #541 → #562 → #568 → this.
Each subsequent merge re-targets the PR base automatically.
@TaprootFreak TaprootFreak merged commit 746961a into feat/visual-regression-handbook-gaps May 25, 2026
3 of 4 checks passed
@TaprootFreak TaprootFreak deleted the feat/handbook-gaps-2 branch May 25, 2026 08:44
@TaprootFreak TaprootFreak mentioned this pull request May 25, 2026
Merged
4 tasks
TaprootFreak added a commit that referenced this pull request May 25, 2026
@TaprootFreak
chore(handbook): drop legacy Maestro screenshots, source from goldens (…
b311bf3 
…#573)

## Summary

Macht die visual-regression Goldens zur **alleinigen** Quelle der
`handbook.realunit.app`-Screenshots und räumt die
Maestro-Tier-3-Pipeline so auf, dass keine zwei Pipelines mehr auf
denselben Pfad schreiben. Bestandsaufnahme vor diesem PR:

- [#541](#541) hat die 57
Goldens eingeführt
- [#568](#568) hat
`scripts/assemble-handbook-screenshots.sh` + `handbook-build-check.yaml`
etabliert; `Dockerfile.handbook` baut die 26 Slots seither aus
`test/goldens/screens/` zusammen
- **Aber:** die 26 alten Maestro-PNGs lagen weiter unter
`docs/handbook/screenshots/`, hatten gegenüber den Goldens drifted (md5
mismatch auf jedem File), und Tier-3 schrieb seine Captures in genau
dieses Verzeichnis — also zwei Pipelines, ein Pfad, mehrdeutige
Bedeutung

## Was ändert sich

**Commit 1 — `git rm` der 26 duplizierten Maestro-PNGs unter
`docs/handbook/screenshots/`**

**Commit 2 — Goldens als kanonische Quelle verdrahten**
- `.gitignore`: `docs/handbook/screenshots/` (jetzt reines
Local-Preview-Verzeichnis, vom Assemble-Script befüllt)
- `.github/workflows/handbook-deploy.yaml`: Path-Trigger um
`test/goldens/screens/**` + `scripts/assemble-handbook-screenshots.sh`
erweitert — Golden-Bumps lösen jetzt einen Deploy aus (das war eine
latente Lücke: vorher hat ein Golden-Update den Handbook-Deploy nicht
erreicht)
- `Dockerfile.handbook` Header bereinigt
- `docs/handbook/README.md`: Top-Statement, "Lokal lesen", "Screenshots
regenerieren" und "Einen neuen Handbook-Eintrag hinzufügen" auf den
golden-driven Workflow umgeschrieben

**Commit 3 — Konsistenz-Audit (nach User-Feedback "ich will alles
professionell und konsistent")**
- Tier-3-Diagnostik-Captures wandern nach `build/handbook-captures/`
(klar getrennt vom Local-Preview-Pfad):
- `scripts/run-handbook-flows.sh` Header neu geschrieben + `SCREENS_DIR`
→ `CAPTURES_DIR`
  - `.github/workflows/tier3-handbook.yaml` Artifact-Pfad nachgezogen
- `.maestro/handbook/*.yaml` (26 Flows): falsche `# Captures
docs/handbook/screenshots/NN-*.png`-Header gegen "Tier-3 navigation
smoke; handbook screenshot is the Golden mapped in ..."-Header getauscht
- `README.md` Workflow-Tabelle: `tier3-handbook.yaml` als
"navigation/tap-routing smoke" beschrieben mit expliziter Note dass
Pixel-Drift Sache von `Visual Regression` ist
- `docs/testing.md` Tier-3-Abschnitt: alte Aussage "uploads
docs/handbook/screenshots/ as a build artifact so reviewers can spot
visual drift" rausgenommen — Drift gehört den Goldens, Tier-3 catched
Tap-Routing/Navigation/Locale/iOS-Build
- `docs/screens.md`: "Handbook"-Spalte erklärt jetzt das Golden-Mapping;
"Handbook numbering"-Note führt mit Goldens und nennt Tier 3 als
parallelen Smoke
- `docs/handbook/de/index.html`: Hero-Lede + `#architecture`-Section
komplett umgeschrieben, Pipeline-Diagramm jetzt page-edit →
`--update-goldens` auf dfx01 → commit → `handbook-deploy.yaml`;
Meta-Description nachgezogen
- `scripts/assemble-handbook-screenshots.sh` Header: Wort "legacy" raus
— `docs/handbook/screenshots/` ist jetzt explizit das Local-Preview-Ziel

**Commit 4 — Subagent-Code-Review-Findings adressiert**
- `docs/handbook/de/index.html`: Typo "committee Repo" → "eingecheckte
Repo" (rendert auf `handbook.realunit.app`)
- `.github/workflows/tier3-handbook.yaml`: "Why no pixel-diff" + "Why
iPhone 17" Header-Kommentare neu gefasst — Pixel-Drift jetzt explizit
Visual-Regression-Job zugeordnet; iPhone-17-Begründung jetzt über
Tap-Koordinaten + Safe-Area-Assertions statt der stalen
Screenshot-Capture-Begründung
- `scripts/assemble-handbook-screenshots.sh`: Beispiel-Pfad in Header
präzisiert (`../screenshots/...` statt `screenshots/...`)
- `docs/handbook/README.md`: Anglizismus "Page-Render" →
"Seitenrendering"

## Bewusst akzeptiertes Trade-off

Der `handbook-deploy.yaml`-Path-Trigger feuert bei jeder Änderung unter
`test/goldens/screens/**` — also auch bei den 9 Feature-Verzeichnissen,
die NICHT in die 26 Handbook-Slots mappen (`buy/`, `sell/`, `kyc/`,
`hardware_connect_bitbox/`, `receive/`, `sell_bitbox/`, `debug_auth/`,
…). Über-Trigger statt fehlende Trigger ist hier bewusst gewählt: die
Mapping-Tabelle in `assemble-handbook-screenshots.sh` zu duplizieren
wäre Wartungslast bei jeder Handbook-Erweiterung; ein `concurrency:
handbook-deploy, cancel-in-progress: false`-Block serialisiert die
Deploys, und das Image-Build ist <10 s + ~30 s Rollout. Worst case sind
1-2 unnötige Deploys pro Sprint — billiger als ein stiller Drift wenn
jemand das Mapping erweitert und den Trigger vergisst.

## Verifiziert

- `bash scripts/assemble-handbook-screenshots.sh /tmp/x` → 26/26 PNGs
sauber aus den committeten Goldens assembliert (keine fehlenden Sources)
- `python3 yaml.safe_load(...)` auf alle berührten Workflows → OK
- `bash -n` auf beide Scripts → OK
- `grep "Captures docs/handbook/screenshots"` → null Treffer im ganzen
Repo
- Mapping ↔ Goldens-Existenz: 26/26 ✅ · Mapping ↔ Maestro-Flow-Existenz:
26/26 ✅ · Semantischer Cross-Check Maestro-Assertion ↔ Golden-Inhalt:
matched bei allen 26
- Subagent-Code-Review (`general-purpose` Agent, model: opus): keine
BLOCKER/MAJOR; 3 MINORs + 2 NITs gefunden, alle in Commit 4 adressiert
- CI auf Commits 2 + 3: Analyze & Test ✅ · Coverage Floor Gate ✅ ·
Visual Regression ✅ · BitBox quirks audit ✅ · Build handbook image +
container smoke ✅
- Tier-3 (Maestro handbook flows): Run
[26397845244](https://github.com/DFXswiss/realunit-app/actions/runs/26397845244)
auf Commit 3 läuft (Label `tier3:full` gesetzt); Commit 4 triggert eine
neue Runde

## Out of scope

- Reduktion oder Abschaffung von `tier3-handbook.yaml` selbst — Tier 3
behält unique Wert als Navigation/Tap-Routing-Smoke +
iOS-Build-/Install-Smoke + `de_CH`-Locale-Check. Separater Follow-up,
sobald die Screenshot-Entkoppelung einen Release-Zyklus in PRD läuft.

## Manual test plan

- [ ] Nach Merge: `handbook-deploy.yaml` triggert auf den Merge-Commit
- [ ] Auf `dev-handbook.realunit.app`: 3 repräsentative Screenshots
(`01-welcome.png`, `12-settings.png`, `26-terms.png`) sind die
Golden-Versionen, nicht die alten Maestro-Captures
- [ ] Nach einem späteren Golden-Bump auf eine der gemappten Pages:
Handbook-Deploy feuert tatsächlich (war vorher silent)
- [ ] Nach einem späteren Tier-3-Run: Artifact `handbook-captures`
enthält PNGs aus `build/handbook-captures/`, nicht aus
`docs/handbook/screenshots/`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@TaprootFreak