feat(cli): auto-inject PERCY_SERVER for maestro test + PERCY_CLI_API alias#2254
Closed
Sriram567 wants to merge 2 commits into
Closed
feat(cli): auto-inject PERCY_SERVER for maestro test + PERCY_CLI_API alias#2254Sriram567 wants to merge 2 commits into
Sriram567 wants to merge 2 commits into
Conversation
Aligns the env contract `percy exec` / `percy app:exec` propagate to
child processes with what `percy-appium-python` reads. The Python SDK
reads `PERCY_CLI_API`; today it only works by coincidence because both
`app:exec` and the SDK default to port 5338. With `--port` overrides
(or any future port shift), the SDK silently points at the wrong CLI
unless the customer exports `PERCY_CLI_API` themselves.
Setting it next to the existing `PERCY_SERVER_ADDRESS` removes that
footgun without changing any other behavior. Matches the unconditional-
override semantics already in place for `PERCY_SERVER_ADDRESS`.
Also adds `PERCY_CLI_API` to cli-doctor's CLEAN_PERCY_ENV test fixture
so env-audit tests stay hermetic against shells that have the var set.
The runtime env-audit code filters on `k.startsWith('PERCY_')` and
needs no change.
Maestro's GraalJS sandbox does not inherit the parent process's env, so `PERCY_SERVER_ADDRESS` exported by app:exec is invisible to the SDK self-hosted. Every customer hits this: their `maestro test` flow falls through to the BS-safe `http://percy.cli:5338` default, the healthcheck fails, and the build finalizes with "Snapshot command was not called". The workaround today is for the customer to thread `-e PERCY_SERVER=http://localhost:<port>` through every Maestro invocation themselves, pairing ports manually when running multi- device. `app:exec` already knows the resolved port via `percy.address()`. Detecting `maestro test` in argv and prepending one `-e` pair removes the footgun without changing any other behavior. The detector is conservative: basename match on argv[0], `test` subcommand, and a scan for any pre-existing `-e PERCY_SERVER=...` (customer override wins). `npx maestro` and other shim invocations correctly fall through to the explicit-`-e` pattern. BrowserStack path is unaffected — BS doesn't wrap maestro with `app:exec`, so this code path is unreachable on BS.
Contributor
Author
|
Consolidated into #2261 (single self-hosted Maestro+Percy V1 PR — all 6 commits, same content, same UX outcomes). Closing this stacked PR in favor of the unified review surface. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two small UX-completion changes on top of #2248. Both remove the port-pairing footgun every native-SDK customer hits self-hosted, with no behavior change on the BrowserStack path.
Unit 1 —
PERCY_CLI_APIalias in@percy/cli-execpercy app:execalready exportsPERCY_SERVER_ADDRESSinto the child process. The Appium Python SDK readsPERCY_CLI_APIinstead. Today this works only by coincidence — both default to port 5338. With--portoverrides (multi-device parallelism), the SDK silently points at the wrong CLI. This addsenv.PERCY_CLI_API = percy?.address()alongside the existing exports so any SDK readingPERCY_CLI_APIauto-aligns to whatever portapp:execpicked.Unit 2 — auto-inject
-e PERCY_SERVERformaestro testin@percy/cli-appMaestro's GraalJS sandbox does not inherit the parent process's env, so
PERCY_SERVER_ADDRESSis invisible to the SDK. Customer's only channel is Maestro-eflags. Today every self-hosted customer threads-e PERCY_SERVER=http://localhost:<port>through every Maestro invocation themselves, pairing ports manually for multi-device.app:execalready knows the resolved port viapercy.address(). The detector is conservative — basename match on argv[0],testsubcommand, and a scan for any pre-existing-e PERCY_SERVER=…(customer override wins per R3).npx maestroand other shim invocations fall through to the explicit--epattern.Stacks on #2248
Base branch:
feat/self-hosted-maestro-percy. Rebase target after #2248 merges:master. Both units are functionally independent of #2248's content but ship in the same release window as the customer-facing completion of self-hosted Maestro support.Why this is safe on BrowserStack
maestro testwithapp:exec.maestro_runner.rbruns maestro directly while the CLI runs inapp exec:startmode separately, so cli-app's wrapped callback is unreachable on BS.runFlow.env > -e > environment, so BS'srunFlow.envsetting would still win.PERCY_CLI_APIexport is unconditional but harmless: BS doesn't run the Appium Python SDK against anapp:exec-spawned child either.Env-var conflict audit
Both names verified against the cli monorepo + every other Percy repo on disk:
PERCY_CLI_API— exactly one existing reader (percy-appium-python/percy/lib/cli_wrapper.py:10), zero existing writers. No CLI internal use. Our write semantically matches the reader's existing default. cli-doctor'senv-audit.jsfilters withk.startsWith('PERCY_')so the new var auto-surfaces; only the test fixture (CLEAN_PERCY_ENV) needs a one-line addition for hermeticity.PERCY_SERVER— read only inside Maestro's GraalJS, never by the CLI process itself. Four writer layers after this PR (runFlow.env> customer-e> our auto-inject > SDK defaulthttp://percy.cli:5338) layer cleanly with no overlap. R3 detector skip prevents double-injection.Testing
maybeInjectMaestroServer(happy path, customer override at multiple positions, basename match on absolute paths,npx/python/appiumskip,maestro hierarchy/list-devicesskip, percy-disabled skip, multi-device port isolation).PERCY_CLI_APIenv-export test mirrors the existingPERCY_SERVER_ADDRESSandPERCY_BUILD_URLtest patterns. Asserts the child seesPERCY_CLI_API === PERCY_SERVER_ADDRESS === \"http://localhost:4567\"when--port=4567.CLEAN_PERCY_ENVfixture updated; 507/508 specs pass (the single failure is a pre-existing proxy-credentials network test inconnectivity.test.js, unrelated).Post-Deploy Monitoring & Validation
/percy/maestro-screenshotPOSTs in self-hosted runs) — should land withoutsessionIdin the request body for self-hosted, and continue to land WITHsessionIdfor BS App Automate.[percy] Percy CLI healthcheck passed.line should now appear without the customer settingPERCY_SERVER.percy-maestro/docs/solutions/best-practices/2026-05-27-self-hosted-maestro-validation.mdwith the explicit-e PERCY_SERVER=…line removed from themaestro testcommand. Build ⬆️ Bump @oclif/config from 1.16.0 to 1.17.0 #14-equivalent (9×9=81) should still produce 3 Unchanged snapshots against an approved baseline.bs-nixpkgscli pin; confirm snapshots produce identically to pre-deploy and thatsessionId=…is present in every CLI relay log line.percy app:exec -- maestro test flow.yamlwith no-e PERCY_SERVER, no manual env aliasing; SDK healthcheck passes; snapshots upload.Snapshot command was not calledpost-deploy → revert. (Should not be possible given the unreachable-on-BS analysis, but watch the first BS run.)app:exec --portinvocations show the same injectedPERCY_SERVER=in their maestro logs → investigate multi-device isolation.🤖 Generated with Claude Opus 4.7 via Claude Code + Compound Engineering v2.54.0