You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
Physical-device fixture and script (phase 2 of the Studio physical-device plan): the Flutter fixture app advertises the agent with a bonsoirProbeAdvertiser when built with PROBE_WIFI=true, and its Info.plist has the NSLocalNetworkUsageDescription and NSBonjourServices entries. scripts/physical-ios-fixture.sh --ip <phone-ip> prints the exact build, install, check and connect steps (CLI and Studio) for the connected iPhone; --check tests ping, the agent's status endpoint and the Bonjour service. It changes nothing.
Opt-in Studio E2E against a physical phone (phase 4 of the physical-device plan): scripts/studio-e2e-physical.sh --host <phone-ip> --token <token> runs four browser-driven tests against the fixture app on a real phone (test connection, connect with the live screen and widget tree, a probe file runs, a wrong token is refused without echoing it). It refuses to run in CI and without the phone's address and token, and checks that an agent answers before starting. studio/e2e-web/PHYSICAL.md is the ten-minute manual checklist with the expected result of each step.
Studio's device picker lists physical iPhones that are paired over the network (phase 3 of the physical-device plan): xcrun devicectl list devices is read next to libimobiledevice, so a phone that is not on a cable no longer silently disappears. Entries show the phone's name and model and how it can be reached (USB, or WiFi only). Selecting a physical device shows a line saying what it needs (the app running in the foreground with the agent: a profile build made with PROBE_AGENT=true, and PROBE_WIFI=true for WiFi); a WiFi-only phone has Connect disabled with a pointer to Connect manually, and an unauthorized or offline Android phone says what to do. probe device list gets the phone's real name too.
--format junit works with flaky-test and test-analytics services (checked against their public docs only, nothing uploaded): suites are sorted, with file, timestamp, a real time and errors; paths are relative with forward slashes (no home directory); <error> marks infrastructure problems (agent connection, setup hooks) and <failure> assertions; <properties> carry probe version, platform, OS version, app id and version, device name and class; data-driven rows get a row property and a re-run test an attempts property. New flags: --junit-attempts (one <testcase> per attempt of a re-run test, off by default because the extra failed cases change the counts of a run that finally passed), --junit-suffix (appended to classname) and --junit-label (run_label property). The JSON report's metadata gains device_class. New page: Results in other tools.
Studio: manual WiFi entry (phase 1 of the physical-device plan, docs/proposals/studio-physical-devices.md): the WiFi overlay has host, port (default 48686) and token fields with validation (IPv4, IPv6 or hostname; port range; token characters), a Test connection step that dials and pings without connecting, and error messages that say what to check (same network, app in the foreground and unlocked, profile build with PROBE_AGENT=true and PROBE_WIFI=true, port, fresh token). The automation endpoint has test_wifi next to connect_wifi (port defaults to 48686), and the MCP tool studio_connect accepts host, port, token (and test_only). Headless browser tests and a native E2E test cover it.
perf.frame_budget_ms in probe.yaml sets the build+raster time above which a frame counts as slow (default 16.7 ms), so slow_frames is meaningful on 120 Hz displays and debug builds (#354). Sent as an optional frameBudgetMs parameter of probe.perf_start (agent 0.25+; older agents keep 16.7 ms and the CLI prints a note); the budget used is reported as frame_budget_ms, and slow_frames is not compared between windows measured with different budgets. Grammar unchanged.
Probe Cloud (preview): opt-in report upload.probe test --upload-report (or hosted.upload_report in probe.yaml) sends a scrubbed copy of the JSON report plus a small envelope (Probe version, device class, retry outcomes, GitHub Actions run info) to a hosted service, with a license that has the probe_cloud plan. The scrubber drops artifacts, device ids and environment, makes paths relative, redacts credential-shaped strings and caps the size. --upload-dry-run prints the exact payload and sends nothing. One attempt with a 5 s timeout; every failure is a warning, never a failed or delayed run. The hosted: block is separate from the device-farm cloud: block. The default endpoint is a placeholder that is not live. New page: Probe Cloud (preview). In CI, supply the license key with the PROBE_LICENSE_KEY environment variable (read at run time, never printed or written). Redirects are never followed and server text is sanitized before it is shown. The CI facts follow the hosted service contract: the pull request source branch (GITHUB_HEAD_REF) is the branch and the target branch is base_branch, ci.job (GITHUB_JOB) and run_label (--upload-label, hosted.label; needed when one CI job uploads more than once), 40-character commit hashes, attempts capped at 20; a real upload needs a GitHub Actions run and a report with a platform, otherwise a clear warning says why nothing was sent.
Changed
A timed-out wait for system dialog appears explains itself on Android (#367): it adds the focused window, the grant state of the permission the dialog is about (or the app's denied runtime permissions) and a hint such as "no system dialog was requested: the app is in the foreground and the permission is denied", so it is clear at once when the app never asked. Also in probe system-dialog wait.
On emulators and simulators the baseline comparison gates memory, cpu and data by default; frame_time and slow_frames gate only on physical devices (they flapped on same-build reruns on virtual devices, #356). --perf-metrics still names any set explicitly. In compare mode windows that span an app restart (or have no memory/frame data) are reported as skipped, counted once, and no longer also as "no entry". This relaxes the default of an existing flag and does not change any grammar, probe.yaml key or protocol meaning; restore the previous set with --perf-metrics all or an explicit list such as --perf-metrics memory,cpu,slow_frames,frame_time,data.
Canary releases attach probe-ios-driver.zip (a non-blocking ios-driver job in release-canary.yml, mirroring the stable workflow), so probe ios-driver install works against a canary. Applies from the next canary; documented in the install guide and the release pipeline notes.
--format junit identities and error kinds changed (migration note).classname, name and file are now relative to the working directory with forward slashes (they were the path as typed; a file outside the directory keeps only its name): consumers keyed on the old identities see new test identities once, which only matters if you passed absolute or ./-prefixed paths. Infrastructure problems (agent connection, before all/before each hooks) are <error> instead of <failure>: consumers must count failures and errors. probe test still exits non-zero for both. Both are valid JUnit; see Results in other tools.
Security
Studio's manual WiFi entry only dials local targets (follow-up to the manual entry): loopback, link-local, RFC 1918 private IPv4, IPv6 unique-local and link-local, .local and single-label hostnames. Hostnames are resolved and every resolved address is checked, so a name that resolves to a public address is refused. A public address needs the owner of the Studio process to set PROBE_STUDIO_ALLOW_PUBLIC_TARGETS=1; automation and MCP callers cannot enable it. Attempts are limited to 6 per minute (TestWiFi and ConnectWiFi share the limit) and errors no longer contain the agent token (dial errors could carry the URL with the token in its query).
Fixed
The ProbeAdvertiser example used an outdated bonsoir API (await broadcast.ready, bonsoir: ^5.1.10); it now uses initialize() and bonsoir 6.0.2, and the docs say why 7.1.x is avoided for now (its Android plugin did not compile against the installed Android SDK platform 36).
The suite time attribute of --format junit was always empty (time=""); it is now the sum of its cases.
A wedged iOS XCUITest driver is reported directly (#357): when the runner log shows "Timed out waiting for AX loaded notification" (xcodebuild exit 65), starting the driver or a later driver call now says the simulator's accessibility service is not responding and how to restart only the test simulator, instead of surfacing as unrelated step timeouts. A driver process that stopped mid-run is reported as stopped.
Docs
iOS guide: "Physical iPhone over WiFi (CLI and Studio): what must be true": build mode and flavor, PROBE_WIFI, the port, the token, network reachability, how the Mac finds the phone (mDNS only with a ProbeAdvertiser; Studio's WiFi overlay has no manual host/port field, the CLI --host/--token path always works), and a three-command check.