Screenshot backend: scrcpy virtual display capture

[Coldaine]

Goal

Implement the _capture_scrcpy() backend in adb_vision/screenshot.py.

scrcpy can capture the Android screen by decoding the device's video stream. Unlike adb shell screencap, it uses the device's MediaCodec encoder over ADB, bypassing the Linux framebuffer — which is critical because adb shell screencap returns blank images on MEmu/VirtualBox.

Interface

async def _capture_scrcpy(*, adb_run: AdbRunFn, serial: str, adb_exe: str) -> str:
    """Return base64-encoded PNG screenshot via scrcpy single-frame capture."""

Implementation Plan

There are two approaches — pick the simpler one:

Option A: scrcpy-server single-frame (preferred)

The scrcpy-server.jar is already in this repo at alas_wrapped/bin/scrcpy/ (check exact name).

1. Push scrcpy-server to device: adb push scrcpy-server /data/local/tmp/scrcpy-server.jar
2. Start server requesting a single frame: adb shell CLASSPATH=/data/local/tmp/scrcpy-server.jar app_process / com.genymobile.scrcpy.Server 2.1 max_size=0 max_fps=1 ...
3. Connect to the device socket, read the H.264/MJPEG frame header + one frame
4. Decode the frame to PNG (use av or PIL)
5. Return base64-encoded PNG

Option B: scrcpy CLI --record with --max-fps=1 + immediate kill

1. Run scrcpy -s <serial> --no-window --record=temp.mp4 --max-fps=1 --codec=h264
2. Wait ~1 second, kill the process
3. Extract first frame from temp.mp4 using av (PyAV) or ffmpeg
4. Return base64-encoded PNG

Option C: scrcpy --v4l2-sink or --video-source=display

Explore if scrcpy has a single-screenshot mode we can use.

Testing (TDD)

• Mock test: Mock adb_run calls. Provide a pre-recorded raw H.264 frame or just a fake PNG as the decoded output. Verify the pipeline calls the right adb commands.
• Failure test: scrcpy-server not found on device → clear error.
• Decode test: Given a known raw frame, verify PNG output is valid.

Tests must pass without a device — all subprocess/ADB calls mocked.

Files to Modify

• adb_vision/screenshot.py — replace the _capture_scrcpy stub
• New adb_vision/test_scrcpy.py — tests

Dependencies

May need: av (PyAV) for video frame decoding. Already in agent_orchestrator/pyproject.toml as av>=16.1.0. Add to adb_vision/pyproject.toml if used.

Acceptance Criteria

• _capture_scrcpy() returns valid base64 PNG
• All existing tests still pass
• New tests cover: happy path, server not found, decode failure
• No ALAS imports anywhere

[google-labs-jules]

Jules is on it. When finished, you will see another comment and be able to review a PR.

[kilo-code-bot]

This issue appears to be a duplicate of #39.

Screenshot backend: DroidCast APK HTTP stream (#39)

Similarity score: 90%

This comment was generated by Kilo Auto-Triage.

[google-labs-jules]

Ready for a review! A PR has been created.