Skip to content

Commit

Permalink
test(python): enable building PT OP (#3787)
Browse files Browse the repository at this point in the history
This enables building PyTorch customized OP in the Python tests for
future usage. PyTorch provides undocumented cxx11 CPU-only wheels at
https://download.pytorch.org/whl/torch_stable.html.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Chores**
- Updated package installation commands for `mpich` and `torch` in the
testing workflow.
  - Set `PYTORCH_ROOT` environment variable.
  - Added `DP_ENABLE_PYTORCH` environment variable.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
  • Loading branch information
njzjz committed May 16, 2024
1 parent b2b126b commit d0d596b
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion .github/workflows/test_python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,18 @@ jobs:
with:
python-version: ${{ matrix.python }}
- run: python -m pip install -U uv
- run: uv pip install --system --only-binary=horovod -e .[cpu,test,torch] horovod[tensorflow-cpu] mpi4py mpich
- run: |
uv pip install --system mpich
uv pip install --system "torch==2.3.0+cpu.cxx11.abi" -i https://download.pytorch.org/whl/
export PYTORCH_ROOT=$(python -c 'import torch;print(torch.__path__[0])')
uv pip install --system --only-binary=horovod -e .[cpu,test] horovod[tensorflow-cpu] mpi4py
env:
# Please note that uv has some issues with finding
# existing TensorFlow package. Currently, it uses
# TensorFlow in the build dependency, but if it
# changes, setting `TENSORFLOW_ROOT`.
TENSORFLOW_VERSION: ${{ matrix.python == '3.8' && '2.13.1' || '2.16.1' }}
DP_ENABLE_PYTORCH: 1
DP_BUILD_TESTING: 1
UV_EXTRA_INDEX_URL: "https://pypi.anaconda.org/njzjz/simple https://pypi.anaconda.org/mpi4py/simple"
- run: dp --version
Expand Down

0 comments on commit d0d596b

Please sign in to comment.