Skip to content

[build-tools] Install ffmpeg for argent screen recording when missing - #4110

Merged
szdziedzic merged 7 commits into
mainfrom
szdziedzic-claude/argent-eas-simulator-recordings-1b6e3b
Jul 30, 2026
Merged

[build-tools] Install ffmpeg for argent screen recording when missing#4110
szdziedzic merged 7 commits into
mainfrom
szdziedzic-claude/argent-eas-simulator-recordings-1b6e3b

Conversation

@szdziedzic

@szdziedzic szdziedzic commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Why

Argent screen recording does not work on EAS Simulator sessions. The failure is a missing ffmpeg binary on the worker VMs.

Verified live on session 019fadda-2063-7fbd-a6c6-7176b0de2681 (iOS, --type argent):

POST /tools/screen-recording-start  →
{"error":"[Tool:screen-recording-start] `ffmpeg` was not found on PATH.
 Install it (e.g. `brew install ffmpeg`) to record the screen."}

ffmpeg is argent's encoder — it pipes simulator frames into ffmpeg's stdin to produce the mp4. Argent resolves it from PATH, then /opt/homebrew/bin, /usr/local/bin, /usr/bin. The worker images ship none of them.

Nothing else is wrong. Argent's screen-recording-start checks, in order:

  1. platform / tvOS
  2. simulator-server frame stream → SCREEN_RECORDING_STREAM_UNAVAILABLE
  3. startCaptureresolveFfmpeg()SCREEN_RECORDING_FFMPEG_NOT_FOUND

We hit 3, which proves 2 passed: the frame stream already works on the VM. ffmpeg is the only blocker.

Also worth noting: with --type argent the tool-server runs on the EAS VM, and list-devices returned a booted iPhone 17 with no remote: prefix. So argent's "remote simulators are unsupported" gate does not apply. This is a missing binary, not an architecture problem.

Why this lives in eas-cli and not only in the image

There is a companion infra PR that adds ffmpeg to the worker images: https://github.com/expo/turtle-v2/pull/2598. That change applies to images built from there on. It deliberately does not update or backfill existing images.

So this PR is not a stopgap for a short rollout window. It is what makes recording work on every worker running a current image, for as long as those images stay in rotation. Once a worker boots an image that already carries ffmpeg, the check finds it and this does nothing.

How

ensureFfmpegInstalledAsync in remoteDeviceRunSession.ts, called from eas/start_argent_remote_session.

  • Detection matches argent's own resolutionPATH plus the same three Homebrew prefixes — so we never reinstall a binary argent can already find, and the step is a no-op on images that ship it.
  • Both platforms. Homebrew on macOS (iOS simulators), apt on Linux (Android emulators). Android is covered because argent's capability is android: { emulator: true, device: true, unknown: true }, simulator-server has an android subcommand, and nothing on that path bypasses resolveFfmpeg(). The Linux workers have passwordless sudo (%sudo ALL=(ALL) NOPASSWD: ALL).
  • apt index refreshed first. The worker's index can be older than the image it booted from, which makes the install 404 on a moved package. A failed refresh is not fatal, since the existing index may still resolve.
  • Runs in the background. brew install ffmpeg pulls a large dependency tree and takes minutes. Awaiting it would delay session readiness by that much. Fired with void instead, so the session comes up at its usual speed and only a recording started in the first moments misses ffmpeg. This is the main design decision worth pushback if you disagree.
  • Best-effort, never throws. Screen recording is one optional argent tool. A failed install is logged and reported to Sentry, and the session continues without it. Never rejecting is also what makes the void safe.

Test Plan

Unit tests in remoteDeviceRunSession.test.ts cover six paths: already on a fallback path, already on PATH, installs with Homebrew on darwin, installs with apt on linux, still installs when the apt refresh fails, and warns without throwing when the install fails.

Test Suites: 99 passed, 99 total
Tests:       931 passed, 931 total

yarn typecheck (13 projects), yarn lint (0 errors), and yarn fmt:check all pass.

Not yet verified end-to-end on a live session — this is a draft. To verify, start an argent session on a current image, confirm the build log shows the install, then call screen-recording-start and expect a started recording instead of the error above.

@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.27%. Comparing base (f145ed7) to head (f0756ec).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4110      +/-   ##
==========================================
+ Coverage   62.25%   62.27%   +0.03%     
==========================================
  Files         994      994              
  Lines       44820    44847      +27     
  Branches     9425     9430       +5     
==========================================
+ Hits        27897    27924      +27     
  Misses      15474    15474              
  Partials     1449     1449              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@szdziedzic
szdziedzic marked this pull request as ready for review July 29, 2026 16:34
@szdziedzic
szdziedzic requested a review from sjchmiela July 29, 2026 16:34

@sjchmiela sjchmiela left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The detection thing is very weird to me

// Argent encodes screen recordings by piping simulator frames into `ffmpeg`,
// which it resolves from PATH and then from these prefixes. Keep the list in
// sync with argent so we never reinstall a binary it can already find.
const FFMPEG_FALLBACK_PATHS = [

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a reason to do it like this. I think our list should only check our expected places for ffmpeg and our expected list is just $PATH?

Does argent really not adhere to $PATH?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I simplified it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they also read these paths, but we don't have to

// large Homebrew dependency tree, so do not block session readiness on it:
// the session comes up at its usual speed and only a recording started in
// the first moments misses ffmpeg. Never rejects, so `void` is safe.
void ensureFfmpegInstalledAsync({ runtimePlatform, env, logger });

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably good for now but I wish we had easier time:

  • logging stuff in phases at will
  • run steps in parallel

Then we wouldn't mix so much in this one function and one log group…

Going to wait for hooks and custom fns to land and see if adding parallel: true would be so hard

@sjchmiela sjchmiela left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tyty!

// which it resolves from PATH. It inherits this step's env, so PATH here is the
// PATH argent will search — no need to probe install prefixes ourselves.
async function isFfmpegAvailableAsync(env: BuildStepEnv): Promise<boolean> {
return (await asyncResult(spawn('sh', ['-c', 'command -v ffmpeg'], { env }))).ok;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

honest question -- do we need sh -c?

const ccachePath = spawnSync('command -v ccache', { env, shell: true });

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should be better now

szdziedzic and others added 5 commits July 30, 2026 16:51
Argent encodes screen recordings by piping simulator frames into ffmpeg.
The macOS worker image does not ship it, so `screen-recording-start` fails
with "`ffmpeg` was not found on PATH" on every EAS Simulator argent session.

Installing ffmpeg pulls a large Homebrew dependency tree, so this runs in
the background rather than delaying session readiness. It is best-effort:
screen recording is one optional argent tool, so a failure is logged and
the session continues without it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The orchestration test replaces the whole remoteDeviceRunSession module with
an explicit factory, so a new export has to be listed there too. Without it
the step under test threw "ensureFfmpegInstalledAsync is not a function".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Argent screen recording supports Android emulators and encodes through the
same ffmpeg path, so Android sessions hit the same failure as iOS. The Linux
workers have passwordless sudo, so install with apt there instead of warning
that the platform is unsupported.

The package index can be older than the image the worker booted from, so
refresh it first. A failed refresh is not fatal — the existing index may
still resolve the package.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Argent resolves ffmpeg from PATH first and only probes install prefixes when
that lookup fails, which is a workaround for hosts with a sanitized PATH. The
tool-server inherits this step's env, so PATH here is the PATH argent will
search. Probing the prefixes ourselves added nothing and coupled us to
argent's internal fallback list.

If ffmpeg is ever installed off PATH, the package manager no-ops on the
redundant install.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@szdziedzic
szdziedzic force-pushed the szdziedzic-claude/argent-eas-simulator-recordings-1b6e3b branch from 16bbcbd to 6c81c10 Compare July 30, 2026 14:53
szdziedzic and others added 2 commits July 30, 2026 17:05
`command -v` is a shell builtin, which is the only reason `sh -c` was there.
Spawning ffmpeg directly does the PATH lookup in the spawn call itself: it
rejects with ENOENT when the binary is absent. Running it also proves the
binary actually works, which a path lookup does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The caller runs ensureFfmpegInstalledAsync in the background with `void`, and
there is no unhandledRejection handler in the worker, so any rejection here
would crash the process and take the live session down with it.

`spawn` is not an async function and can throw synchronously. `asyncResult`
only wraps an already-created promise, so it never saw that throw: it escaped
through the unguarded availability check and out of the `void`. Wrap the whole
body instead of trusting each callee, matching fetchServeSimTurnArgsAsync.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

✅ Thank you for adding the changelog entry!

@szdziedzic
szdziedzic merged commit 5754073 into main Jul 30, 2026
11 checks passed
@szdziedzic
szdziedzic deleted the szdziedzic-claude/argent-eas-simulator-recordings-1b6e3b branch July 30, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants