You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
Studio's device pane header says what the device view can do: "view only" when the agent has no input calls (older flutter_probe_agent) and "no element select" when it cannot hit-test, with the reason in the tooltip. Before, a read-only view looked like a broken one.
Studio: "+ tap" in the device pane header writes a tap step for the element selected with Alt+click at the editor cursor (shared device view, preview). The step uses the same escaped selector as the clipboard copy; a selection with control characters is refused. Browser test 36 covers it.
probe lint warns about count and zero-state text checks (warning only; the exit status is unchanged). A see or wait until ... appears on text such as "0 items", "3 items" or "No items", with no matching pattern, also passes on longer text that contains it ("10 items"). The warning suggests the anchored form, matching "^0 items$"; a matching pattern anchored at both ends (^...$) silences it, and any other pattern is still flagged. The text-matching docs and the CLI reference say the same.
probe doctor checks that a first run can work and says how to fix what is missing. It looks at probe.yaml, the tests/ folder, the Flutter SDK, the platform tools (adb for Android, xcrun for iOS) and the devices they can see. It changes nothing, and it exits non-zero only when probe.yaml or the tests/ folder is missing; warnings alone do not fail it.
probe doctor --format json prints the same checks as a JSON report (schema probe.doctor/1, with ok and each check's fix), and the doctor MCP tool returns it, so an AI agent can check a machine without running tests. A required check that fails is a normal result, not an MCP error. The tool count is 35 in the development version.
Studio can show the shared device view (preview, off by default; set PROBE_STUDIO_DEVICE_VIEW=1). Studio starts a loopback hub for the connected device in process (internal/deviceview) and the device pane draws it with the shared component: tap, swipe, typing, hover outline and Alt+click selection (the selector goes to the clipboard and the inspector search; text with control characters is refused). The old capture loop stays the default and is not started when the view is on. In Studio's browser test mode the hub also accepts the test page's own origin (deviceview.Options.ExtraOrigins, exact matches only). Browser test 36 covers it; tests 10 and 11 still pass with the flag off.
probe device view: an interactive device view over a local WebSocket (preview, Flutter layer only, from the on-device agent). The command starts a hub on 127.0.0.1 with a one-time token, prints one JSON line (url, token, device) and streams frames to one client at a time; the client can tap, long press, swipe, type and ask which widget is under a point (docs/studio-device-protocol.md). The hub checks the token and the Host and Origin headers, limits message size and rate, keeps only the latest frame per client and writes no files. An agent without the new methods keeps working as a view-only mirror.
Agent methods for the device view (probe_agent): probe.tap_at, probe.long_press_at, probe.swipe_at, probe.type_text, probe.hit_test, probe.dump_tree with bounds:true (nodes with logical-pixel bounds), and scale on probe.screenshot (the reply also carries width, height, logicalWidth, logicalHeight and dpr). All are additive; an older CLI never sends them. JPEG is not offered yet because the agent has no encoder without a new dependency.
A shared TypeScript device view component (studio/frontend/src/device-view/, built to dist-device-view/): letterboxed canvas, gestures to taps, swipes and long presses, hover bounds, Alt+click to produce a selector, reconnect, pause when hidden. It runs in a browser and in a webview; a demo page and a mock hub are included.
VS Code: "FlutterProbe: Open Device View" opens the device in an editor tab through probe device view, with a strict content security policy limited to the hub's origin, and inserts a tap step for an Alt+clicked element. Not available in remote windows yet.
Two read-only MCP tools, studio_widget_tree and studio_ui_state (33 tools in all): the widget tree of the connected device as plain text, and the connection and run state Studio shows, in one call. Both go through the Studio automation endpoint, so no second connection to the agent is made. Every doc surface and the extension manifest list them.
The Studio UI shows fps, average and slowest capture time over the last second (it showed the last frame's time only).
CI integrations, as sources to copy: actions/probe-test (a composite GitHub Action that installs the CLI from a release, runs probe test on a device the job provides, attaches the JUnit report, and can pass --upload-report), bitrise-step (a Bitrise Step used with a git:: reference), and a Codemagic recipe page (/ci-cd/codemagic/, iOS simulator). Both directories are MIT-licensed and self-contained: copy one to its own repository to publish it. They were written from each vendor's documentation and tested locally (asset mapping, argument assembly) but have not run on those services.
probe_version in probe.yaml pins the FlutterProbe CLI release a project is tested with, as an exact release (0.25.1) or a release line (0.25.x). probe test stops before any device is touched or any test starts when the installed CLI does not meet the pin, and names the installed version, the required one and how to install it. A malformed value fails when probe.yaml loads. Without the field nothing changes.
probe validate checks probe.yaml and the probe_version pin without connecting to a device. It reports each check as ok, warn or fail, with the fix for a failure, and exits non-zero when a check fails. --format json prints the same report with schema probe.validate/1. Run it in CI before probe test.
validate_project MCP tool runs probe validate and returns its JSON report, so an AI agent can check the setup and the pin without running tests. A failed check is returned as a normal result. The tool count is 34 in the development version.
Run line and report version.probe test prints probe CLI X / agent A on stderr once the first agent answers the handshake, so stdout (JSON, JUnit) is unchanged. The JSON report metadata and the JUnit suite properties gain agent_version, omitted when the agent does not report a version.
Changed
studio/README.md has a manual checklist for the native Studio window and the VS Code device view (what the browser tests cannot cover).
The Studio E2E workflow runs only when started by hand (gh workflow run "Studio E2E"), no longer on version tags. With no self-hosted runner registered, every tag run queued for 24 hours and was cancelled; and a public repository should not hand a self-hosted runner to anything but a deliberate dispatch.
probe test says what to do when no device is usable. The message names probe doctor, and a failed device listing shows its error instead of only "no connected devices found".
Cloud device farm status is stated honestly. The README table, the new Cloud device farms page, the VS Code README, probe.yaml.example, --cloud-provider help, llms.txt and the comparison pages now say the five providers are implemented but not verified on a live account, that cloud runs need a relay service (not shipped, and the hosted service is not live), and that Maestro Cloud runs Maestro flows only. The page lists what was compared with each vendor's documentation on 2026-10-14, with mismatches.
--cloud-provider firebase prints a deprecation notice. Google's page says Firebase Test Lab "is deprecated and will be fully shut down on September 30, 2027" and points to the Developer Device Platform; FlutterProbe has no provider for it yet. The provider still works until then.
Shorter path to the Probe Cloud offer and to a first test on the site: the landing page has Tutorials and Pricing in its header and footer and a design-partners band (free beta for the first 10 teams, with a link to the sample dashboard), the pricing Team card's button now applies as a design partner, the waitlist is a one-click prefilled email, and the install page opens with a four-command fast path.
The site no longer calls the whole project open source. The CLI is source-available under BSL 1.1 and the Dart agent is MIT, so the landing page, its meta tags, the social image, the contact, privacy and terms pages and one comparison page now say "free to use, source-available CLI (BSL 1.1), MIT agent", and the landing page and pricing state what the BSL's use grant allows (any purpose, including production, except offering a competing hosted testing service).
The landing page's performance numbers are the measured ones. "<50ms command round-trip", "<0.5% flake rate" and "<100ms CLI cold start" were design targets, not measurements, and are gone from the landing page; it now shows the replicated Maestro benchmark (about 2x faster median suite time, 0 flaky FlutterProbe runs of 20, with a link to the data).
The pricing page labels the planned Team price ("Planned price, not final. Free for design partners.").
The landing page's install command is Homebrew (brew install AlphaWaveSystems/tap/probe) instead of cloning the repository and running make build, the install step no longer shows a Linux-only download, and the page links to the first tutorial.
Security
The GitHub Action and the Bitrise Step verify the downloaded CLI. The install scripts compare the download with the SHA-256 digest that GitHub records for the release asset (the digest field of the Releases API; the releases publish no checksum file), install nothing on a mismatch or when no digest is published, and have an off-by-default escape hatch (skip-verify / skip_verify). The Action pins actions/upload-artifact by commit SHA (v7.0.2), takes a github-token for the API call, and has an artifact-name input so matrix legs do not clash. Tested offline with a good, a tampered and an unpublished download (actions/probe-test/scripts/test-install.sh, bitrise-step/test-step.sh). The Codemagic recipe checks the digest too.
Fixed
The device view says why a connection is refused right after another window took over. The hub answers a connection inside its 3 s takeover cool-down with HTTP 429, but a browser WebSocket does not expose the status of a refused handshake, so the VS Code panel and Studio showed only "reconnecting". A connection that fails before it opens, within 4 s of the page being taken over, now reads "another window just took over this device, retrying" and still retries with backoff. Studio runs the hub in process, so it needs nothing like VS Code's --exit-on-stdin-close (that flag only matters for a hub the editor starts as a subprocess).
probe init wrote a sample test that waited for "Welcome", which most apps do not show, so the first probe test of a new project timed out. The sample now only opens the app, so it passes on any app that starts with the agent enabled once project.app in probe.yaml is set. A test checks that the sample parses, lints clean and checks no text. The init output, the README and the install and quick-start guides now describe the same first run.
Two Maestro claims on the Flutter testing frameworks comparison are now sourced and worded to what Maestro documents: the iOS docs describe simulators and do not describe local physical iOS devices (it was stated as "no physical iOS device support"), and the Cloud price is quoted from Maestro's pricing page with the date read. The CI paragraph no longer says built-in reporting requires Maestro Cloud.
AWS Device Farm sessions did not install the uploaded app: CreateRemoteAccessSession was sent without appArn (and with remoteDebugEnabled, which the API reference does not list). The upload ARN is now sent as appArn and the undocumented field is dropped. Not yet verified on a live account.
Studio's live view no longer writes every frame to reports/screenshots/, and no longer fills the device with screenshots. The screenshot request returned the image inline and the CLI also saved it, once per frame, in the working directory of the Studio process (several files a second, without bound); the agent also kept a copy of each frame in the device's temporary directory. The live view and probe studio now ask for the image inline (probe.screenshot with inline: true): nothing is written on the host and the agent deletes its copy. probe test screenshots (take a screenshot, failure screenshots) are unchanged. Compatible both ways: an older agent ignores inline and answers as before (the view still works and its device copies stay), and an older CLI never sends it. The agent change reaches pub.dev with the next release.
probe studio's /api/exec answered ok without running the step, and /api/tap ignored the click's coordinates./api/exec now answers HTTP 501 "not implemented". A tap by text still works; a tap at a screen position answers 501 (the agent has no coordinate tap) and an empty request answers 400.
Studio's recorder wrote type "hi" into the "hi" field for a typing event that named no target: the typed text was used as the selector. It now records type "hi".
Docs
The GitHub Actions page and the tutorial install the released CLI instead of building it from source, enable KVM for the Android emulator action (which its documentation requires on Linux runners), and use the current actions/checkout major version.
README and pricing clarity: the version badge shows the latest release (the tag badge could show a canary tag), the license badges say which part is BSL (CLI) and which is MIT (agent), the table of contents lists "Get started", and the pricing header says what is free and that Probe Cloud is a preview that is not live yet.
The unmeasured "sub-50ms" claim is removed everywhere. "Sub-50ms command round-trips" (and the "<0.5% flake", "<100ms cold start" and "under 90 seconds" goals, and the "~100-200ms" and "10-40ms" figures for other tools and for the WebSocket) were never measured. The README, docs index, comparison and blog pages, architecture page, social image, probe --help text and the agent README now describe direct in-app control and cite the one measurement we have: the replicated benchmark against Maestro (about 2x faster median suite time, 0 flaky runs of 20). The flutter_probe_agent pubspec description changed too; pub.dev shows it from the next release.
A Probe Cloud design-partner guide (/tools/design-partner-guide/): what a partner installs, how to inspect an upload with --upload-dry-run before the hosted service is live, what a GitHub Actions upload needs, and how to send feedback.
The sample dashboard pages at /demo/ are refreshed from their source (static pages with no form or sign-out button).
A "Why FlutterProbe" page comparing FlutterProbe with Maestro, Patrol and the Appium Flutter driver. Every statement about another tool is taken from that tool's own documentation and linked; costs of choosing FlutterProbe and when each tool fits are listed.
Probe Cloud design-partner offer and a Probe Cloud preview page on the docs site: pricing now has a design-partners section (free beta for the first 10 teams, 50% off for life when paid plans launch, apply by email or a public GitHub issue), and /probe-cloud/ states what the preview does today (--upload-report, --upload-dry-run, the scrubber and its caps) and what is only planned.