AutoCua 0.1.9: Windows HTTPS security fix, WebDriverAgent and Chrome no longer downloaded

Security fix for Windows: please upgrade

On Windows, AutoCua could switch HTTPS certificate checks off by itself when a check failed. This happened in two places. One was the check that runs before an AI provider is used, which then covered every connection the app made. The other was the Telegram bot. An attacker on the same network, such as public Wi-Fi or a hostile proxy, causes exactly that failure. They could then read API keys, the bot token, prompts and screenshots, or answer as the AI provider.

• 0.1.9 never turns certificate checks off.
• When the usual certificate bundle fails, AutoCua checks against the Windows certificate store. An antivirus or company proxy whose root Windows trusts, such as Norton or Defender, keeps working. Gemini now uses the Windows store too.
• When even Windows rejects the certificate, the run stops with a message that says what to turn off. You can also point SSL_CERT_FILE at the root certificate.
• The Telegram bot uses the same check, so a firewall block no longer counts as a certificate problem there either. If the bot cannot verify Telegram, it does not start.
• macOS and Linux never had this fallback and are unchanged.

Affected: Windows, versions 0.1.0 to 0.1.8. Fixed in 0.1.9. Advisory: GHSA-qm8g-cw9j-w2gj

WebDriverAgent and Chrome are never downloaded

• iOS: WebDriverAgent now lives in AutoCua_data/WebDriverAgent, and AutoCua never downloads it. While it is missing, iOS automation is paused. The terminal and Settings, Connect Device, iPhone show the one git clone command with the full path.
• Web: Google Chrome is never downloaded or installed. When it is missing, the terminal says so once, the app shows a popup when it starts, and a web run stops cleanly.
• Android: adb still asks y/N on the terminal. Settings, Connect Device, Android now has an Install adb button. After Yes the app closes, and the next start from the terminal asks.

Install or upgrade

pip install -U autocua

Wheels for macOS (Apple Silicon), Windows x64 and Linux x86_64.

Full Changelog: v0.1.8...v0.1.9

AutoCua 0.1.8: Daisy chain mode and parallel sub-agents

What's new

Daisy chain mode

• Computer use now has two modes: This PC and Daisy chain.
• This PC works on this computer only. Phones are offline there.
• Daisy chain adds your iPhone and Android phone as sub-agents. The phones pair as soon as you pick Daisy chain, and the agent sees them ready from its first step.
• New chats and Telegram tasks start in This PC.

Sub-agents in fixed slots

• The agent can now run up to 4 coder agents and 4 browser agents at the same time, plus one Android and one iPhone agent.
• Every slot has its own id: coder agents 1-4, browser agents 5-8, phones 9 and 10.
• A new sub-agent takes the first free slot. If every slot of that type is busy, nothing starts, and the agent is told to wait for one to finish.

Browser agents in parallel

• Each browser agent works in its own background tab of your Chrome.
• The first one opens Chrome and the others join it, so they never fight over the browser.
• While two or more are working, the front tab shows "Multiple agents running" with the number of agents.

Other changes

• macOS: when the screen scan misses an element or puts it in the wrong place, the agent can click by position through System Events, without moving your mouse.
• Chat: a long message you send is shown shortened, with a round arrow button in the corner. Click it to see the whole message, and again to fold it.
• Windows: the TLS check now only contacts the provider you picked. A firewall block is no longer mistaken for HTTPS interception, which used to switch certificate checks off.
• README and docs updates.

Install or upgrade

pip install -U autocua

Wheels for macOS (Apple Silicon), Windows x64 and Linux x86_64.

Full Changelog: v0.1.7...v0.1.8

AutoCua v0.1.7: faster phones in Computer use, iPhone apps over Wi-Fi

AutoCua v0.1.7

This release makes the iPhone and Android sub-agents start much faster in Computer use, fixes iPhone app launching over Wi-Fi, and makes Android over Wi-Fi more reliable.

Faster phones in Computer use

• Sub-agents are ready before the task comes. When a phone comes online, its agent starts and gets ready in the background. When the main agent hands it a task, it begins at once: about 0.05 s on Android, where a busy run used to take 43 s.
• The iPhone stays connected between requests. After a request, the iPhone session stays warm for 10 minutes, so the next request gets the phone in about 0.2 s instead of starting again.
• Faster iPhone start over Wi-Fi. WebDriverAgent is built once and reused, so a start takes about 4 to 5 s instead of 6 to 8 s. If the saved build stops working (for example, an expired free signing profile), it rebuilds by itself.
• Lighter iOS agent. The iOS agent now loads only the AI provider it uses: 351 modules at start instead of 3,587.
• A stuck iOS start recovers on its own. If the iOS agent has not started its run after 20 seconds, it is restarted once, and the details are saved for debugging.

Fixes

• iPhone over Wi-Fi: opening apps works. open_app failed with "Expecting value: line 1 column 1 (char 0)" when the iPhone was on Wi-Fi. The installed app list now comes from Xcode's own tool, so apps open on Wi-Fi as they do on the cable.
• iPhone connect no longer hangs on Xcode. Xcode's devicectl script could start an Xcode component install before every command and hang for minutes. AutoCua now calls the real devicectl directly.
• Android over Wi-Fi after the Mac sleeps. A phone on Wi-Fi could show as "not on Wi-Fi" after the Mac had slept, because adb stopped hearing phones. AutoCua now restarts adb once when nothing is connected, then looks again.
• Claude prompts. The memory field in the screenshot agents' Claude prompts now says clearly what it is for: the screenshot is removed after each step, and this field carries the context forward.

Install

pip install -U autocua

Prebuilt wheels (Python 3.10+):

Platform	Wheel
macOS, Apple silicon	macosx_11_0_arm64
Windows x64	win_amd64
Linux x86_64 (glibc 2.17+)	manylinux_2_17_x86_64

Other platforms build from source and need Rust.

Full changelog: v0.1.6...v0.1.7

## AutoCua v0.1.6: Android is here

This release brings Android to AutoCua: a prebuilt Android agent in every wheel, phones in the desktop app, a phone sub-agent for computer use, and an emulator for terminal runs. Phones can also connect over Wi-Fi in the app, and Telegram tasks now show up as normal chat messages.

Highlights

• Android agent prebuilt in every wheel. No Rust and no scrcpy needed. The phone helper takes the screenshots itself.
• Android in the desktop app. Connect a phone in Settings > Connect Device > Android, then pick Mobile use > Android in the chat box.
• android_agent sub-agent. The computer-use agent can hand phone work to an Android sub-agent on macOS, Linux and Windows, shown live in its own box in the chat.
• Phones over Wi-Fi in the app. Android phones you connected once over USB (Android 11+, Wireless debugging on). iPhones paired with Xcode on the same Wi-Fi.
• Android emulator for terminal runs. device="simulation" is now the default for Android. Android 11 to 17, phone or tablet, your .apk installed, several copies in parallel.
• Telegram tasks as chat messages. A task sent from Telegram runs like a typed one, with a small "sent via: Telegram" pill.

Desktop app

• New Mobile via: Wi-Fi / Wired toggle in Mobile use and Computer use. Wired is the default, and there is no silent fallback either way.
• Phone pairing in the chat box: the box is read only while a phone pairs, a circle turns beside the mode picker, and if pairing fails the error is shown with a red arrow to try again.
• The phone window (simulator and emulator view) is frameless on Windows and Linux, with the video under the window buttons. Resize, snap and maximize all work.
• The phone window now opens on Linux, and you can drag, resize and maximize it on Wayland.
• The Android emulator shows in the same window as iOS Simulator runs, several side by side.
• iOS options show only on a Mac.
• The Rust agents build and adb is offered in the terminal before any window opens.
• Terminal runs use the API key saved in Settings when .env has none.

Agents and tools

• Android tools: web, vault (logins in the "android" section, PIN pads typed for you), back (system Back), video_player (streaming, pause, play) and ground_scroll.
• ground_scroll on iOS and Android: swipe from any point of the screenshot, for lists with no element id.
• iPhone over Wi-Fi: WebDriverAgent starts through Xcode. When several iPhones are paired, the unlocked one goes first.
• Taps, typing and scrolls on Android no longer block the app while adb works.

Setup and install

• Nothing is downloaded without a yes. One question on the terminal before adb, WebDriverAgent, Chrome, the OCR models, the emulator and similar downloads, each from its maker's own server. Set AutoCua_AUTO_DOWNLOAD=1 for CI.
• An adb you installed yourself (Android Studio, Homebrew, apt) is used as it is.
• iOS setup moved into setup_mac.sh behind a timed question. The iPhone disk image comes from Xcode's own copy.
• THIRD_PARTY_LICENSES.txt ships in the wheel. Rust is pinned to 1.97.1.

Fixes

• Every print in the app could fail with I/O operation on closed file. Fixed.
• Telegram runs were saved but showed nothing live in the window. Fixed.
• A run without save_conversation=True wiped the last saved conversation. Now only debug/ is cleared on every run (all six agents).
• The iPhone pick took the newest paired phone even when another one was plugged in. It now takes the connected one.
• A dozing Android phone on Wi-Fi is found again (it used to drop out of adb's list).
• "Device connected" no longer stays as the chat box hint after a run.
• A late answer from an old iOS pick no longer flips the picker.
• Closing the phone window by its button no longer crashes at exit.
• A run that fails while starting keeps the reopened chat's saved steps.
• Parallel web and emulator runs read their output in UTF-8, so emoji lines no longer break on Windows.
• The Settings cog no longer jiggles on a click.

Install

pip install -U autocua

Prebuilt wheels (Python 3.10+):

Platform	Wheel
macOS, Apple silicon	macosx_11_0_arm64
Windows x64	win_amd64
Linux x86_64 (glibc 2.17+)	manylinux_2_17_x86_64

Other platforms build from source and need Rust.

Full changelog: 9205ff9...v0.1.6

AutoCua v0.1.5: Browser vault, faster scans and steadier coders

A vault the web agent can type from, page scans that no longer stall on ad frames, coders that act in the same turn as their notes, and an iPhone link that survives a USB re-plug.

What's new

Vault

• Logins for the web agent: put a site's login in the vault and the agent fills it with the new vault tool. AutoCua types the saved value itself, so the model only ever sees key names like password, and later steps show the typed value as [vault].
• Fills only where it should: a saved site covers its subdomains. The vault types only on https (or http on localhost), checks the page is still on that site right before typing, never types into a frame, and puts a password only into a password field.
• iPhone vault works the same way: it fills the exact key the agent names instead of guessing from the field label, for the app in front by its exact name, and checks the focused field and the app right before typing. It now batches with other input; only a keypad PIN still goes alone.

Browser

• Scans stop stalling on ad frames: a frame inside a same-origin wrapper (common with ad and tag-manager scripts) now answers at once, so reading a page like itv.com takes 20–30 ms instead of 2–3 s. Some frames that were missing from the page tree now show up in the right place.
• Headless runs get past bot walls: headless Chrome now sends the normal Chrome user agent instead of "HeadlessChrome", so sites like itv.com load and sign in. Normal runs are unchanged.
• Extension Test button works with the app open: the popup's Test scan is saved whenever the desktop app is running, during a run or between runs, instead of asking you to run test.command.
• A page can't fake its scan: the page's result is taken only from the top frame Chrome itself labels, so a page lying to one of its frames can no longer swap that frame's content in for the whole page.

Computer use

• Coders act in the same turn as their notes: the coder is steered to make its notes call and its actions together, so a step no longer costs two model calls. macOS, Windows and Linux.
• Notes pause: if a model writes notes twice in a row without acting, the notes tool steps aside for five steps, leaving only the tools that do the work.
• 100 iterations on macOS: the Mac coder (was 50) and minion (was 30) now get 100, as on Windows and Linux, so a minion no longer stops with partial findings a third of the way in.
• Longer replies on GPT-6 Astra, Luna and Sol: coder and minion replies can run to 8,000 tokens (was 2,000), so a whole-file write or a plan is no longer cut off.
• Refusals fall back: when Claude refuses a coder or minion step, the fallback model (if one is set) answers that step instead of the same request being retried three times. An empty reply from Claude now says why it was empty.

iOS

• Survives a USB re-plug: when QuickTime screen mirroring or a cable glitch re-plugs the iPhone, the phone session restarts itself in place, up to 5 times in 10 minutes. Before, every tap failed with "Failed to create WDA session".
• ground_click shows the tapping finger: in the iOS run's tool chain and in the iOS agent's box on the desktops, the same as click.

Telegram

• Comes back on its own: if Telegram is unreachable when the bot starts, it retries after 5, 10, 20 and 40 s, then every minute, instead of staying off for the session. Flood limits (429) are retried too; a bad token still fails loudly once.
• Token stays out of logs: retry lines show the error with the token replaced.
• Windows keeps certificate checks on: an outage at startup no longer leaves the bot running with TLS verification off.

App

• Rebuilt in React + TSX: the page, the intro and its parts are React components now and work as before. pip installs still need no Node: the built UI ships in the package.
• New logo, right-sized Dock icon: the macOS Dock icon now sits at the same size as other apps.

Upgrading

• Vault file format: AutoCua_data/vault/credentials.json now has one section per device: {"ios": {...}, "web": {...}}. The old flat file is no longer read, so move your existing iPhone entries under "ios".
• Restart Chrome once: vault typing in extension mode needs this release's extension, which Chrome loads when it starts. An older one refuses it.
• Agent folders renamed: agent/main_driver is now agent/computer_use on macOS, Windows and Linux, and agent/iphone_use on iOS. This only matters if you import or edit them.

Docs

• New computer use demo video and banner in the README.
• Bug reports and feature requests now have their own forms on GitHub.

Get started

pip install -U autocua

Merged PRs: #22, #25 to #35

Full changelog: 0ded162...v0.1.5

AutoCua v0.1.4: Sharper iOS taps and browser fixes

Sharper iPhone control, a browser window that stays out of your way, and a livelier Shell use mascot.

What's new

iOS

• Sees more controls: the element tree now keeps picker wheels, text boxes (like Google Calendar's "Add title"), scroll lists, and elements that hold a value, like Google Calendar's time wheels and its Event / Task / Birthday row.
• Wide switches work: tapping a switch wider than 100 pt now hits its knob, so whole-row toggles like Google Calendar's "All day" respond.
• New ground_click tool: taps a raw point (x, y on a 0–1000 scale of the screenshot) for targets with no [id], like one chip inside a wide element. click stays the main way to tap.
• Clean shutdown: closing the app now releases the iPhone, and WDA runners left behind by a crashed or force-quit app are cleared on connect. Runners with a live parent are never touched.

Browser

• Chrome is never the whole screen: in extension mode, a maximised or full-screen window goes back to a normal window when a run starts (or 76% × 96% of the screen, centred). A window you sized yourself is left as it is.
• Cursor bubble shimmers again: the light sweeps across the words every 1.2 s.

Shell use

• Round mascot head with moving eyes: the eyes glance around at rest, wake with a quick look when the agent starts, and glide slowly while it works. Motion runs at a steady 60 steps a second, so 120 Hz screens don't speed it up.

Get started

pip install -U autocua

Merged PRs: #18, #19

Full changelog: V0.1.3...v0.1.4

AutoCua v0.1.3: iPhone, browser ,coding and computer use at once

What's new

New chat UI

• The main area works like a chat. Each turn has its own recording, its own bot and a reply box.
• The reply glides to its last line, and the agent strip shows a thinking orb.

Computer use

• Browser, iOS and coder sub-agents each get their own box.
• Screens keep one shape, and browser screenshots sit in the centre.

Shell use

• Shell use is now a chat with its bots.
• The terminal box is the run's tool box.

Intro and modes

• New splash with all seven bots.
• A picked mode's bot comes in big, then becomes the run's bot.

Left bar

• Flat and collapsible, with a pure white app background.
• The chat list runs down to the footer and reads clearer.

Recording

• Every run is recorded and compressed.
• On Linux, recording picks a GPU encoder that works, or falls back to CPU.

Web scrape

• more continues from exactly where the last read stopped.

Security

• The app's API needs a per-run secret token.
• The simulator's WebDriverAgent listens only on this Mac.
• Upgraded pyo3 to 0.29.
• Upgraded Pillow, requests, python-dotenv, Flask and Werkzeug.

Docs

• Updated agent_capabilities.md.

Merged PRs: #8 to #16

AutoCua v0.1.2: iPhone, browser and computer at once

The Mac agent can now hand work to a plugged-in iPhone, computer use, the iPhone and the browser run at the same time, and the desktop app opens in its own Chromium window, pip installs included.

What's new

• iPhone from the Mac agent: plug in a paired iPhone or iPad and the Mac computer-use agent connects it in the background and delegates tasks to it (ios_agent), with the live phone screen in its own window.
• Everything at once: computer use, the iPhone and the browser now work at the same time from the app.
• Desktop app in Chromium: the app opens in its own Electron window on macOS, Windows and Linux, on its own port (27321) instead of 5000. pip installs no longer need Node.js: the first ui=True launch asks before fetching Electron (checked against a pinned SHA-256). Ubuntu 24.04+ is supported.
• Web scrape mode: a new index tool stores what the agent keeps from each chunk as numbered JSON under AutoCua_data/scrapes, and the indexes survive memory compression and resumed chats.
• Leaner agent notes: Claude agents' agent_memory is now just memory + next_goal, and memory compression is on.
• UI: a new splash with the logo and wordmark, the memory gauge as a ring beside the Skills icon, and Shell runs that show their done message and the chat's earlier runs.

Fixes

• Screen resolution set to 1080p across all OSes.
• Mac open_app no longer moves or resizes the app's window.
• Browser scans wait for the page to load, fixing "no answer from AutoCuaBridge in 40s".
• The first click after typing in the chat box now registers.
• iOS connection fixes.

Get started

pip install -U autocua

This release is autocua 0.1.2 on PyPI. The Quick start in the README covers the rest.

Full changelog: v1.0.0...v0.1.2

AutoCua v1.0.0: Automatic Computer Use Agent

Automatic Computer Use Agent

AutoCua is an agent that drives your computer for you. Describe a task and it looks at the screen, decides what to do, and does it with the mouse and keyboard. It also takes control of iOS, on an iPhone, an iPad, or the iOS Simulator.

What it can drive

• Your computer: macOS, Windows and Linux (computer use)
• iPhone and iPad: a paired device or the iOS Simulator (needs a Mac)
• Your browser: Chrome, through the AutoCua extension or headless
• A terminal: a coding agent working in a real shell
• From your phone: send tasks over Telegram

Get started

pip install autocua

Then call run_agent(...) and pick a surface with mode=. The Quick start in the README walks through it, including the extra steps for Linux and iOS.

Works with multiple model providers. Linux computer use needs a GNOME desktop.