Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions .github/workflows/test_linux_cuda.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@

name: Unit-tests on Linux GPU

on:
pull_request:
push:
branches:
- nightly
- main
- release/*
workflow_dispatch:

jobs:
tests:
strategy:
matrix:
python_version: ["3.9"]
# TODO: Add more cuda versions.
cuda_arch_version: ["12.4"]
fail-fast: false
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
with:
runner: linux.g5.4xlarge.nvidia.gpu
repository: pytorch/torchcodec
gpu-arch-type: cuda
gpu-arch-version: ${{ matrix.cuda_arch_version }}
timeout: 120

script: |
nvidia-smi
conda create --yes --name test
conda activate test
conda install --quiet --yes pip cmake pkg-config nasm

pip install --quiet --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu124
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Torchvision is required for the gpu benchmark which uses functional transforms from vision. See gpu_benchmark.py for details.

https://github.com/pytorch/torchcodec/blob/f4065f1b477148cfb0ef94167fb0bf3a63803e55/benchmarks/decoders/gpu_benchmark.py#L8

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but we don't need it to build, right? So we might prefer to install it later. This is OK though, since this job is just a temporary setup I guess.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. Should I move it below after the build step so it doesn't accidentally leak as a build-time dep? WDYT?

conda install --quiet --yes nvidia::libnpp

# Build and install FFMPEG from source with CUDA enabled.
# The one on conda doesn't have CUDA enabled.
# Sub-step: install nvidia headers. Reference this link for details:
# https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/ffmpeg-with-nvidia-gpu/index.html
git clone --quiet https://git.videolan.org/git/ffmpeg/nv-codec-headers.git

pushd nv-codec-headers
make --silent PREFIX=$CONDA_PREFIX -j install
popd

# Now build FFMPEG from source with CUDA enabled.
git clone --quiet https://git.ffmpeg.org/ffmpeg.git ffmpeg/
pushd ffmpeg
git checkout origin/release/6.1
which pkg-config
pkg-config --list-all
./configure --prefix=$CONDA_PREFIX --enable-nonfree --enable-cuda-nvcc --disable-static --enable-shared --optflags=-fno-omit-frame-pointer --disable-stripping --enable-cuvid
make --silent -j install
popd

CMAKE_BUILD_PARALLEL_LEVEL=8 CXXFLAGS="" LDFLAGS="-Wl,--allow-shlib-undefined -Wl,-rpath,$CONDA_PREFIX/lib -Wl,-rpath-link,$CONDA_PREFIX/lib -L$CONDA_PREFIX/lib" CMAKE_BUILD_TYPE=Release ENABLE_CUDA=1 ENABLE_NVTX=1 pip install -e ".[dev]" --no-build-isolation -vv --debug

# We skip certain tests because they are not relevant to GPU decoding and they always fail with
# a custom FFMPEG build.
pytest -k "not (test_get_metadata or get_ffmpeg_version)"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, fun fact: it's currently hard to tell whether the GPU tests are being ran at all, because they're protected within an "if" block. I'll share pointers on how to address this offline

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is accurate and a real problem. Should I address this in a different diff?

python benchmarks/decoders/gpu_benchmark.py
conda deactivate
2 changes: 1 addition & 1 deletion src/torchcodec/decoders/_core/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ function(make_torchcodec_library library_name ffmpeg_target)

set(NEEDED_LIBRARIES ${ffmpeg_target} ${TORCH_LIBRARIES} ${Python3_LIBRARIES})
if(ENABLE_CUDA)
list(APPEND NEEDED_LIBRARIES ${CUDA_CUDA_LIBRARY})
list(APPEND NEEDED_LIBRARIES ${CUDA_CUDA_LIBRARY} ${CUDA_nppi_LIBRARY} ${CUDA_nppicc_LIBRARY} )
endif()
if(ENABLE_NVTX)
list(APPEND NEEDED_LIBRARIES nvtx3-cpp)
Expand Down