feat: clip.py

[greatgitsby]

description

resolves: #34885

clip openpilot routes!

uses xvfb under the hood, so only supporting linux for now

usage: clip.py [-h] [--demo] [-d DATA_DIR] [-e END] [-f FILE_SIZE] [-o OUTPUT] [-p PREFIX] [-q {low,high}] [-s START] [route]

clip your openpilot route.

positional arguments:
  route                 The route (e.g. a2a0ccea32023010/2023-07-27--13-01-19 or a2a0ccea32023010/2023-07-27--13-01-19/20/30)

options:
  -h, --help            show this help message and exit
  --demo                use the demo route
  -d DATA_DIR, --data-dir DATA_DIR
                        local directory where route data is stored
  -e END, --end END     stop clipping at <end> seconds
  -f FILE_SIZE, --file-size FILE_SIZE
                        target file size (Discord/GitHub support max 10MB, default is 9.5MB)
  -o OUTPUT, --output OUTPUT
                        output clip to (.mp4)
  -p PREFIX, --prefix PREFIX
                        openpilot prefix
  -q {low,high}, --quality {low,high}
                        quality of camera (low = qcam, high = hevc)
  -s START, --start START
                        start clipping at <start> seconds

comma.ai

dev call (4/30/25)

• run tool anywhere
• better demo route (start=90, end=105 looks good, no calibrating alert + complete lane change)

requirements

• using qcam from replay runs too fast and UI needs to wait a second before starting recording
  • added a "buffer" to start the replay 1 second early and wait 1 second before starting the recording. this 1 second can be throwaway as it is not part of the user's request
• kill clip if any subprocess fails to run (ffmpeg out of storage/error, xvfb errors, replay errors, UI errors)
• validate all input cases (demo, route w/o start/end, route w/ start/end, invalid inputs)
• route private
  • exception thrown when route is invalid or inaccessible
• validate if qcam is used by replay
  • replay uses qcams if the hevc's aren't uploaded. replay loads very very quickly when using qcam, though quality is not great.
• read from FS for logs/video on device
• all processes should close on exit, no dangling procs!

next steps

• works on Mac using docker! a follow up PR will come for Mac support via the CLI, see a preview below:

❯ BUILD=1 tools/op.sh clip --demo
Checking system → [✔]
[+] Building 0.2s (21/21) FINISHED                                                                                                                                                                                        docker:desktop-linux
 => [internal] load build definition from Dockerfile.openpilot_base                                                                                                                                                                       0.0s
 => => transferring dockerfile: 3.27kB                                                                                                                                                                                                    0.0s
 => [internal] load metadata for docker.io/library/ubuntu:24.04                                                                                                                                                                           0.2s
 => [internal] load .dockerignore                                                                                                                                                                                                         0.0s
 => => transferring context: 442B                                                                                                                                                                                                         0.0s
 => importing cache manifest from ghcr.io/commaai/openpilot-base:latest                                                                                                                                                                   0.0s
 => [ 1/15] FROM docker.io/library/ubuntu:24.04@sha256:1e622c5f073b4f6bfad6632f2616c7f59ef256e96fe78bf6a595d1dc4376ac02                                                                                                                   0.0s
 => [internal] load build context                                                                                                                                                                                                         0.0s
 => => transferring context: 386B                                                                                                                                                                                                         0.0s
 => CACHED [ 2/15] RUN apt-get update &&     apt-get install -y --no-install-recommends sudo tzdata locales ssh pulseaudio xvfb x11-xserver-utils gnome-screenshot python3-tk python3-dev &&     rm -rf /var/lib/apt/lists/*              0.0s
 => CACHED [ 3/15] RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && locale-gen                                                                                                                                 0.0s
 => CACHED [ 4/15] COPY tools/install_ubuntu_dependencies.sh /tmp/tools/                                                                                                                                                                  0.0s
 => CACHED [ 5/15] RUN /tmp/tools/install_ubuntu_dependencies.sh &&     rm -rf /var/lib/apt/lists/* /tmp/* &&     cd /usr/lib/gcc/arm-none-eabi/* &&     rm -rf arm/ thumb/nofp thumb/v6* thumb/v8* thumb/v7+fp thumb/v7-r+fp.sp          0.0s
 => CACHED [ 6/15] RUN apt-get update && apt-get install -y --no-install-recommends     apt-utils     alien     unzip     tar     curl     xz-utils     dbus     gcc-arm-none-eabi     tmux     vim     libx11-6     wget   && rm -rf /v  0.0s
 => CACHED [ 7/15] RUN mkdir -p /tmp/opencl-driver-intel &&     cd /tmp/opencl-driver-intel &&     wget https://github.com/intel/llvm/releases/download/2024-WW14/oclcpuexp-2024.17.3.0.09_rel.tar.gz &&     wget https://github.com/one  0.0s
 => CACHED [ 8/15] RUN dbus-uuidgen > /etc/machine-id                                                                                                                                                                                     0.0s
 => CACHED [ 9/15] RUN useradd -m -s /bin/bash -u 1001 batman                                                                                                                                                                             0.0s
 => CACHED [10/15] RUN usermod -aG sudo batman                                                                                                                                                                                            0.0s
 => CACHED [11/15] RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers                                                                                                                                                                0.0s
 => CACHED [12/15] COPY --chown=batman pyproject.toml uv.lock /home/batman                                                                                                                                                                0.0s
 => CACHED [13/15] COPY --chown=batman tools/install_python_dependencies.sh /home/batman/tools/                                                                                                                                           0.0s
 => CACHED [14/15] RUN cd /home/batman &&     tools/install_python_dependencies.sh &&     rm -rf tools/ pyproject.toml uv.lock .cache                                                                                                     0.0s
 => CACHED [15/15] RUN sudo git config --global --add safe.directory /tmp/openpilot                                                                                                                                                       0.0s
 => exporting to image                                                                                                                                                                                                                    0.0s
 => => exporting layers                                                                                                                                                                                                                   0.0s
 => => writing image sha256:b28e5d17a5f87c412d774b71b3b9fa9d1b785945ed9f3da451de247d92b9d170                                                                                                                                              0.0s
 => => naming to ghcr.io/commaai/openpilot-base:latest                                                                                                                                                                                    0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview 
Checking system → [✔]
Running command → tools/clip/run.py --demo │
───────────────────────────────────────────┘

2025-04-30 06:06:56,058 clip.py INFO    clipping route a2a0ccea32023010/2023-07-27--13-01-19, start=20 end=30 quality=high target_filesize=9.5MB
2025-04-30 06:06:56,064 clip.py INFO    waiting for replay to begin (loading segments, may take a while)...
2025-04-30 06:07:02,156 clip.py INFO    recording in progress (10s)...
2025-04-30 06:07:12,813 clip.py INFO    recording complete: /Users/trey/dev/openpilot/output.mp4

demos

tools/op.sh clip --demo

demo.mp4

tools/op.sh clip d3cff3c1d3af097b/00000006--0df6ded53a -s 120 -e 140 -q high -o ~/Desktop/write2.mp4

write2.mp4

[adeebshihadeh]

Worked first try for me, this is a great start!

[greatgitsby]

converting back to draft. not happy with the performance on some machines (in my linux VM on Mac, it stutters with some settings)

also the bitrate estimate does not reliably restrict file size. using ffmpeg 2 pass is a better approach to get consistent file size as clips will be primarily uploaded to discord or other media platforms

we want lowest latency first pass to avoid stutter with second pass optimizing for size

EDIT: spoke to adeeb offline; don't want to over-optimize on this use case, these improvements can be a follow-up