Skip to content

xcodebuild test with parallel testing buffers all stdout when writing to a pipe #273

Description

@justine-acorns

When running tuist test with -parallel-testing-enabled YES and the test action, xcodebuild
internally buffers all test result output and only flushes to stdout on process exit. This means no output is produced
during the entire test execution phase, which can be 15-20+ minutes for some larger UI test suites. CI systems like CircleCI kill
the job for exceeding their no-output timeout.

This does NOT affect test-without-building - that action streams results correctly even through a pipe.

Ran some experiments to verify:

  1. Pipe (file redirect): Monitored file size every 10 seconds while xcodebuild test with parallel UI tests wrote to a file. File grew during build, then was completely frozen for 3.5 minutes (zero bytes written), then all 28 test results appeared in a single burst when the process exited.
  2. PTY (pseudo-terminal): Ran the identical command through a pty via Python's pty.fork(). 142 test results streamed continuously throughout execution (from 75s to 605s).
  3. Direct terminal: Running the same command directly in a terminal also streams results in real time.

The root cause is that xcodebuild checks isatty(stdout) and switches to application-level buffering for parallel test
output when stdout is not a TTY. Since Tuist's CommandRunner uses Pipe() for the subprocess, xcodebuild sees a pipe
and buffers.

Workaround:
Splitting into tuist test --build-only followed by tuist test --without-building works because test-without-building
streams output correctly. But this requires users to know about the issue and restructure their CI.

Started putting together some ideas (e.g., using a pty instead of Pipe() in the Command package), but wanted to raise here and discuss first, as this seems like a fairly major change.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions