Skip to content

feat: add agent-device tooling for AI simulator control#30302

Merged
NicolasMassart merged 9 commits into
mainfrom
MCWP-450-ai-simulator-control
May 25, 2026
Merged

feat: add agent-device tooling for AI simulator control#30302
NicolasMassart merged 9 commits into
mainfrom
MCWP-450-ai-simulator-control

Conversation

@NicolasMassart
Copy link
Copy Markdown
Contributor

@NicolasMassart NicolasMassart commented May 18, 2026

Description

Adds agent-device as a dev dependency, exposing it as a CLI for AI agents to control iOS/Android simulators during development.

Device control (opening the app, navigating screens, taking snapshots, interacting with UI elements, capturing visual evidence) runs through:

yarn agent-device <command> --json

The package is installed locally so no global install is required. Version is pinned to an exact version (0.14.8, no semver range) as recommended by the security team — combined with Yarn's lockfile checksum, this prevents undetected same-tag re-deployments.

Ideally to be used with the simulator-control skill

Changelog

CHANGELOG entry: null

Related issues

Fixes: https://consensyssoftware.atlassian.net/browse/MCWP-450

Manual testing steps

Feature: agent-device CLI availability

  Background:
    Given the branch is checked out and `yarn install` has been run
    And the simulator-control skill is installed via `yarn skills --domain testing`
    And an iOS simulator is booted
    And Metro is running via `yarn watch:clean`

  Scenario: AI agent controls the iOS simulator via CLI
    Given the Cursor IDE is open on this project
    When the user sends the following prompt to the Cursor agent:
      """
      Open the MetaMask app on the iOS simulator and take a screenshot of the home screen
      """
    And the agent runs `yarn agent-device devices --platform ios` to list booted simulators
    And the agent runs `yarn agent-device open io.metamask.MetaMask --platform ios`
    And the MetaMask app opens on the simulator
    And the agent runs `yarn agent-device screenshot` and returns the image

Screenshots/Recordings

Before

N/A

After

image

Pre-merge author checklist

Performance checks (if applicable)

  • I've tested on Android
  • I've tested with a power user scenario
  • I've instrumented key operations with Sentry traces for production performance metrics

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Low Risk
Low risk: adds a pinned dev-only CLI dependency plus lockfile updates and documentation, with no runtime/app logic changes; main concern is install friction due to agent-device's Node engine declaration.

Overview
Adds the agent-device package (pinned to 0.14.8) as a dev dependency to provide a local yarn agent-device CLI for controlling iOS/Android simulators.

Updates yarn.lock for the new dependency and related transitive bumps (notably fast-xml-parser/fast-xml-builder), adds agent-device to .depcheckrc.yml ignores since it’s CLI-only, and documents usage in docs/readme/agent-device.md including the Node engine note.

Reviewed by Cursor Bugbot for commit 7c9637f. Bugbot is set up for automated code reviews on this repo. Configure here.

- Added `agent-device` package to `package.json` with version `0.14.8`.
- Updated `yarn.lock` to include the new dependency and its metadata.
- Created configuration files for `agent-device` in `.claude/settings.json`, `.codex/config.toml`, and `.cursor/mcp.json` to define command and arguments for MCP servers.

This change enables the integration of the `agent-device` functionality into the project.
@NicolasMassart NicolasMassart self-assigned this May 18, 2026
@github-actions
Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@socket-security
Copy link
Copy Markdown

socket-security Bot commented May 18, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​agent-device@​0.14.87810010096100

View full report

@socket-security
Copy link
Copy Markdown

socket-security Bot commented May 18, 2026

All alerts resolved. Learn more about Socket for GitHub.

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

Ignoring alerts on:

  • npm/agent-device@0.14.8

View full report

@NicolasMassart NicolasMassart marked this pull request as ready for review May 19, 2026 08:57
@NicolasMassart NicolasMassart added Code Impact - Low Minor code change that can safely applied to the codebase area-devex Issues and PRs focused on developer experience team-mobile-platform Mobile Platform team labels May 19, 2026
@NicolasMassart NicolasMassart requested a review from a team May 19, 2026 09:02
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cec2d37c24

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread package.json Outdated
@NicolasMassart NicolasMassart requested review from mmenoxx and x86NOP May 19, 2026 09:43
@NicolasMassart
Copy link
Copy Markdown
Contributor Author

@SocketSecurity ignore npm/agent-device@0.14.8

Discussed internally with the security team (Craig Scheets, Michal Ogrodniczak). Key findings:

  • Package is in devDependencies — risk is limited to developer devices, not users or the production app
  • No evidence of malicious behavior (no exfiltration, no persistence, no obfuscation)
  • The Android APK download + checksum flow is by design for device automation
  • Version has been pinned to 0.14.8 (no semver range) so any content change under the same tag would be caught by Yarn's lockfile checksum verification

Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 2d86efd. Configure here.

Comment thread yarn.lock
@NicolasMassart NicolasMassart marked this pull request as draft May 19, 2026 12:55
…entation, update depcheck to ignore agent-device
@NicolasMassart NicolasMassart marked this pull request as ready for review May 19, 2026 13:45
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 06449054cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread package.json
@NicolasMassart NicolasMassart changed the title feat: add agent-device MCP tooling for AI simulator control feat: add agent-device tooling for AI simulator control May 19, 2026
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.03%. Comparing base (5624ab0) to head (0644905).
⚠️ Report is 114 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #30302      +/-   ##
==========================================
+ Coverage   81.98%   82.03%   +0.05%     
==========================================
  Files        5447     5444       -3     
  Lines      145537   145287     -250     
  Branches    33248    33212      -36     
==========================================
- Hits       119312   119187     -125     
+ Misses      18094    17974     -120     
+ Partials     8131     8126       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@NicolasMassart NicolasMassart enabled auto-merge May 22, 2026 06:07
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Smart E2E Test Selection

  • Selected E2E tags: None (no tests recommended)
  • Selected Performance tags: None (no tests recommended)
  • Risk Level: low
  • AI Confidence: 97%
click to see 🤖 AI reasoning details

E2E Test Selection:
The PR adds agent-device (v0.14.8) as a new dev dependency used exclusively as a CLI tool for device control (screenshots, taps, scrolls, etc.). The changes are:

  1. package.json: Adds agent-device to devDependencies only — no app runtime code is affected.
  2. .depcheckrc.yml: Ignores agent-device from depcheck since it's a CLI tool, not an imported module.
  3. docs/readme/agent-device.md: New documentation file only — no code changes.
  4. yarn.lock: Lock file update for the new dependency.

No app source code, controllers, UI components, navigation, E2E test infrastructure, or test configurations were modified. This is a purely additive developer tooling change with zero impact on app behavior, user flows, or test execution. No E2E tests need to run to validate this change.

Performance Test Selection:
No app code, UI components, state management, or performance-sensitive paths were changed. Adding a CLI dev dependency has no impact on app performance metrics.

View GitHub Actions results

@sonarqubecloud
Copy link
Copy Markdown

@github-project-automation github-project-automation Bot moved this from Needs dev review to Review finalised - Ready to be merged in PR review queue May 25, 2026
@NicolasMassart NicolasMassart added this pull request to the merge queue May 25, 2026
Merged via the queue into main with commit ac04e1f May 25, 2026
201 of 205 checks passed
@NicolasMassart NicolasMassart deleted the MCWP-450-ai-simulator-control branch May 25, 2026 14:10
@github-project-automation github-project-automation Bot moved this from Review finalised - Ready to be merged to Merged, Closed or Archived in PR review queue May 25, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators May 25, 2026
@metamaskbotv2 metamaskbotv2 Bot added the release-7.80.0 Issue or pull request that will be included in release 7.80.0 label May 25, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-devex Issues and PRs focused on developer experience Code Impact - Low Minor code change that can safely applied to the codebase release-7.80.0 Issue or pull request that will be included in release 7.80.0 size-S team-mobile-platform Mobile Platform team

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants