agent-device test cannot make its results directory and stops before the first flow

[benjaminkomen]

Summary

agent-device test stops before it runs the first flow. It cannot make its own results
directory. The error is ENOENT from mkdir.

This happens with --artifacts-dir. It also happens with the default directory. So it is not a
configuration mistake.

agent-device replay runs the same files in the same session correctly. Only test fails.

Version

agent-device 0.20.10. The CLI runs on an ubuntu-latest GitHub Actions runner. The daemon is a
remote EAS Simulator session. The daemon also reports 0.20.10. The target is an iOS simulator.

Steps to repeat, with an explicit directory

The caller makes the directory first. The tool then fails inside it.

mkdir -p remote-device-artifacts/ad-test

agent-device test 'qa-flows/open-*.ad' \
  --platform ios \
  --state-dir "$(agent-device session state-dir)" \
  --artifacts-dir remote-device-artifacts/ad-test \
  --reporter default \
  --reporter junit:remote-device-artifacts/ad-test/junit.xml

Result:

Error (UNKNOWN): ENOENT: no such file or directory, mkdir
'/home/runner/work/installer-app/installer-app/remote-device-artifacts/ad-test/cd5f9c01feec8d70/qa-flows__open-grid-settings.ad/attempt-1'
Hint: Unexpected internal error. Retry with --debug and report the diagnostics log if it persists.
Diagnostic ID: mtkb2nu0-e698d27f

Steps to repeat, with the default directory

Remove --artifacts-dir. The tool then uses the documented default location.

agent-device test 'qa-flows/open-*.ad' \
  --platform ios \
  --state-dir "$(agent-device session state-dir)" \
  --reporter default

Result:

Error (UNKNOWN): ENOENT: no such file or directory, mkdir
'/home/runner/work/installer-app/installer-app/.agent-device/test-artifacts/7c8ca96c9eb63bfc/qa-flows__open-grid-settings.ad/attempt-1'
Diagnostic ID: mtkboyfp-5709fff6

Expected result

test runs each flow it finds. It writes replay.ad and result.txt for each attempt. The
replay-e2e page describes this behaviour.

Probable cause

The path that fails is three levels below a directory that exists. In the first example, the
caller made remote-device-artifacts/ad-test. The tool then failed on
ad-test/<run-id>/<flow>/attempt-1.

A mkdir call without the recursive option gives this error. A recursive call makes all three
levels. This would correct both examples.

Effect on users

This defect blocks more than the results files.

context retries= applies to test only. replay does not read it. So a caller cannot use
per-file retries while test fails. The caller must add retries around replay instead.

Other information

Both runs used the same repository and the same session type. They ran a few minutes apart. The
journey that reaches the screens under test passed in both runs.

[thymikee]

Triage: this is a valid remote-daemon bug, but the failing boundary is not a non-recursive mkdir. v0.20.10 already calls mkdirSync(attemptArtifactsDir, { recursive: true }).

The suite request carries the caller's cwd, then the remote daemon resolves and writes the artifact path against that caller-only location. In this report that is /home/runner/work/..., which does not exist in the remote EAS daemon filesystem. replay avoids the failure because it does not create the per-suite artifact tree.

The remaining design decision is artifact ownership for remote daemons: either materialize suite artifacts on the caller, or write remotely and transfer them through the existing artifact transport. Acceptance should cover both the default directory and a relative --artifacts-dir with a remote daemon, while preserving local-daemon behavior and retries.