Add CI for linux cuda #187
Conversation
facebook-github-bot
added
the
CLA Signed
| conda create --yes --name test | ||
| conda activate test | ||
| conda install --yes pip cmake pkg-config nasm |
There was a problem hiding this comment.
Also use --quiet where possible in every conda call, otherwise the logs get really long
| # This fails because conda reactivate fails inside this env | ||
| # conda install --yes conda-forge::compilers | ||
|
|
||
| pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu124 |
There was a problem hiding this comment.
We shouldn't need to install torchaudio, and we shouldn't need to install torchvision at this point as well
There was a problem hiding this comment.
Torchvision is needed by gpu_benchmark.py
|
|
||
| # We skip certain tests because they are not relevant to GPU decoding and they always fail with | ||
| # a custom FFMPEG build. | ||
| pytest -k "not (test_get_metadata or get_ffmpeg_version)" |
There was a problem hiding this comment.
So, fun fact: it's currently hard to tell whether the GPU tests are being ran at all, because they're protected within an "if" block. I'll share pointers on how to address this offline
There was a problem hiding this comment.
This is accurate and a real problem. Should I address this in a different diff?
|
|
||
| # We skip certain tests because they are not relevant to GPU decoding and they always fail with | ||
| # a custom FFMPEG build. | ||
| pytest -k "not (test_get_metadata or get_ffmpeg_version)" |
There was a problem hiding this comment.
This is accurate and a real problem. Should I address this in a different diff?
| conda activate test | ||
| conda install --quiet --yes pip cmake pkg-config nasm | ||
|
|
||
| pip install --quiet --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu124 |
There was a problem hiding this comment.
Torchvision is required for the gpu benchmark which uses functional transforms from vision. See gpu_benchmark.py for details.
There was a problem hiding this comment.
but we don't need it to build, right? So we might prefer to install it later. This is OK though, since this job is just a temporary setup I guess.
There was a problem hiding this comment.
That's a good point. Should I move it below after the build step so it doesn't accidentally leak as a build-time dep? WDYT?
There was a problem hiding this comment.
Thanks @ahmadsharif1 . Let's follow-up about https://github.com/pytorch/torchcodec/pull/187/files#r1723560554 in a separate PR.
Eventually we'll want to make sure wheels can be built (and tested) and integrate that with our existing wheel.yml workflows, but this is a great first step to test GPU-related changes.
This reverts commit cafea81.
3 participants
This diff adds GPU CI to TorchCodec.
It installs the NVFFMPEG headers, builds FFMPEG from source, builds TorchCodec and then runs test on the GPU tests.
This will be useful to catch GPU recording regressions.
I also added an explicit dependency on NPP image processing library because it was failing to build inside the container environment.