Skip to content

feat(cli-app): auto-resolve screenshot dir + WARN on no-addr injection (R1+R3)#2263

Merged
Sriram567 merged 1 commit into
feat/self-hosted-maestro-percy-v1from
feat/cli-app-auto-screenshot-dir
Jun 2, 2026
Merged

feat(cli-app): auto-resolve screenshot dir + WARN on no-addr injection (R1+R3)#2263
Sriram567 merged 1 commit into
feat/self-hosted-maestro-percy-v1from
feat/cli-app-auto-screenshot-dir

Conversation

@Sriram567
Copy link
Copy Markdown
Contributor

Summary

Stacks on top of #2261 — the consolidated self-hosted Maestro+Percy V1. Completes the customer-side UX by removing the last manual-env-var step.

After cli#2261 merges + this PR merges, the canonical self-hosted Maestro+Percy command goes from:

export PERCY_MAESTRO_SCREENSHOT_DIR=$PWD/.percy-out
percy app:exec -- maestro test --test-output-dir $PERCY_MAESTRO_SCREENSHOT_DIR flow.yaml

to just:

percy app:exec -- maestro test flow.yaml

Customer overrides (either via PERCY_MAESTRO_SCREENSHOT_DIR env var or --test-output-dir in maestro args) always win — same precedence pattern as cli#2261's -e PERCY_SERVER auto-inject.

What changes

packages/cli-app/src/exec.js — adds maybeInjectScreenshotDir(ctx, log) next to the existing maybeInjectMaestroServer. The wrapped exec callback calls both helpers before yielding to cli-exec's spawn.

Resolution order for the screenshot dir:

Customer state Behavior
Sets both PERCY_MAESTRO_SCREENSHOT_DIR env AND --test-output-dir flag Fully passive — trust customer, no mutation
Sets env only Use env value, inject matching --test-output-dir <env> flag
Sets flag only Use flag value, mirror to PERCY_MAESTRO_SCREENSHOT_DIR env so the SDK + CLI relay see the same path
Neither set Try ${CWD}/.percy-out via mkdirSync({ recursive: true }). On mkdir failure (EACCES, EROFS, EEXIST as a file), fall back to ${TMPDIR}/percy-maestro-<pid> with a WARN log

The env var and the flag are always aligned to the same path. The SDK reads the env var; Maestro reads the flag — both pointing at the same dir is the contract.

Plus a diagnostic WARN log (R3) in maybeInjectMaestroServer when percy.address() is falsy (percy disabled, start failed). Previously this skip was silent — customer's maestro test would run, no snapshots upload, Percy build would finalize empty with no log hint. The WARN now tells the customer what won't happen and how to fix it. Customer-supplied -e PERCY_SERVER override skip does NOT warn (intentional flow control).

Origin

Brainstorm → plan:

  • Requirements: percy-maestro/docs/brainstorms/2026-06-02-selfhosted-followup-bundle-requirements.md (R1 + R3)
  • Plan: cli/docs/plans/2026-06-02-001-feat-auto-screenshot-dir-plan.md

Testing

  • cli-app: 30/30 specs pass. 14 new scenarios for maybeInjectScreenshotDir (happy path; env override; flag override; both set; EACCES / EROFS / EEXIST fallbacks; hierarchy / npx / python / short-args skips; basename match on absolute paths). 2 new scenarios for the maybeInjectMaestroServer WARN log (fires on no-addr; does NOT fire on customer-override).
  • Lint: clean.
  • End-to-end (deferred — real-device): customer can run the canonical Android quickstart from example-percy-maestro-selfhosted without the export PERCY_MAESTRO_SCREENSHOT_DIR=... step and reach the same green Percy build. Validation runbook + example repo README will be updated in follow-up doc PRs after this lands.

Stacking

Post-Deploy Monitoring & Validation

  • What to monitor
    • Customer reports of Snapshot command was not called after a flow that ran without explicit PERCY_MAESTRO_SCREENSHOT_DIR — would indicate the auto-resolved dir doesn't match what the SDK / Maestro use.
    • WARN-log fallback-rate: how often customers land on ${TMPDIR}/percy-maestro-<pid> instead of ${CWD}/.percy-out. High rate (>5%) would indicate widespread read-only CWDs.
    • The new no-addr WARN log appearing in customer percy logs — indicates a misconfigured PERCY_TOKEN.
  • Validation checks
    • On the example-percy-maestro-selfhosted Android quickstart, run percy app:exec -- maestro test flows/regions.yaml with no env exports → expect green Percy build with snapshots in ${CWD}/.percy-out/screenshots/.
    • With chmod -w . on the CWD, repeat → expect green Percy build using TMPDIR fallback + a WARN line in stdout.
    • Run percy app:exec --disable -- maestro test flows/regions.yaml → expect the no-addr WARN in stdout.
  • Expected healthy signals
    • No new Snapshot command was not called errors on builds where the customer didn't change anything.
    • WARN log fallback rate near 0% in normal customer environments.
  • Failure signals / rollback trigger
    • Customer reports where the screenshot dir is auto-resolved to one path but the SDK or Maestro writes to a different one → revert immediately (signal that the alignment contract is broken).
    • Any BS App Automate Maestro build regression — should be impossible (this PR only affects app:exec, which BS doesn't wrap maestro through), but watch for one cycle.
  • Validation window & owner

🤖 Generated with Claude Opus 4.7 via Claude Code + Compound Engineering v2.54.0

…put-dir; WARN on no-addr injection skip

Removes the last piece of customer-side bookkeeping for self-hosted
Maestro+Percy. Today customers must export PERCY_MAESTRO_SCREENSHOT_DIR
AND pass --test-output-dir <same path> to maestro test. After this PR,
`percy app:exec` does both automatically.

New helper `maybeInjectScreenshotDir(ctx, log)` next to the existing
`maybeInjectMaestroServer`. Resolution order:

  1. Customer set BOTH env + --test-output-dir flag → trust them.
  2. Customer set env only → use env value, inject matching flag.
  3. Customer set flag only → use flag value, mirror to env.
  4. Neither set → try ${CWD}/.percy-out. On mkdir failure (EACCES,
     EROFS, EEXIST), fall back to ${TMPDIR}/percy-maestro-<pid> with
     a WARN log explaining why.

The env var and the flag are always kept aligned to the same path.
The SDK reads the env var; Maestro reads the flag — both pointing at
the same dir is the contract.

Also adds a WARN log in `maybeInjectMaestroServer` when `percy.address()`
is falsy (percy disabled, start failed). Previously this skip was
silent — customer's maestro test would run, no snapshots upload, Percy
build would finalize empty with no log hint. The WARN now tells the
customer what won't happen and how to fix it (set PERCY_TOKEN).
Customer-supplied `-e PERCY_SERVER` override skip does NOT warn —
that's legitimate flow control, not a problem.

Tests:
- 14 new scenarios for maybeInjectScreenshotDir (happy path, env
  override, flag override, both, EACCES/EROFS/EEXIST fallbacks,
  hierarchy/npx/python/short-args skip, basename match on full path).
- 2 new scenarios for maybeInjectMaestroServer's WARN log.
- 30 of 30 cli-app specs pass.
@Sriram567 Sriram567 requested a review from a team as a code owner June 2, 2026 11:33
@Sriram567 Sriram567 requested review from amandeepsingh333 and pankaj443 and removed request for a team June 2, 2026 11:33
@Sriram567 Sriram567 merged commit d9d508d into feat/self-hosted-maestro-percy-v1 Jun 2, 2026
38 of 40 checks passed
@Sriram567 Sriram567 deleted the feat/cli-app-auto-screenshot-dir branch June 2, 2026 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant