Skip to content

Llama distributed example #3831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: abose/trt_llm_installation_dist
Choose a base branch
from
Open

Llama distributed example #3831

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8914503
TRT-LLM installation tool
apbose Feb 13, 2025
9255c4a
Changes for CUDA13
apbose Sep 23, 2025
d09103a
Changes to TRT-LLM download tool for multigpu distributed case
apbose Aug 19, 2025
809c7ee
Distributed utils package, separating out env for single GPU and mult…
apbose Sep 25, 2025
b428e6b
Distributed utils package, separating out env for single GPU and mult…
apbose Sep 25, 2025
f00d349
Distributed llama3 example
apbose Aug 19, 2025
dd88017
Llama torchTRT lib and env initialization reorg
apbose Sep 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/workflows/build-test-linux-aarch64.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,41 @@ jobs:
python -m pytest -ra -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_core_test_results.xml .
popd

tests-py-distributed:
name: Test dynamo distributed [Python]
needs: [filter-matrix, build]
if: false
strategy:
fail-fast: false
matrix:
include:
- repository: pytorch/tensorrt
package-name: torch_tensorrt
pre-script: packaging/pre_build_script.sh
post-script: packaging/post_build_script.sh
smoke-test-script: packaging/smoke_test_script.sh
uses: ./.github/workflows/linux-test.yml
with:
job-name: tests-py-dynamo-distributed
repository: "pytorch/tensorrt"
ref: ""
test-infra-repository: pytorch/test-infra
test-infra-ref: main
build-matrix: ${{ needs.filter-matrix.outputs.matrix }}
pre-script: ${{ matrix.pre-script }}
script: |
set -euo pipefail
export USE_HOST_DEPS=1
export CI_BUILD=1
export USE_TRTLLM_PLUGINS=1
dnf install -y mpich mpich-devel openmpi openmpi-devel
pushd .
cd tests/py
cd dynamo
python -m pytest -ra --junitxml=${RUNNER_TEST_RESULTS_DIR}/dynamo_distributed_test_results.xml distributed/test_nccl_ops.py
popd


concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref_name }}-${{ inputs.repository }}-${{ github.event_name == 'workflow_dispatch' }}-${{ inputs.job-name }}
cancel-in-progress: true
33 changes: 33 additions & 0 deletions .github/workflows/build-test-linux-x86_64.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,39 @@ jobs:
python -m pytest -ra -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_core_test_results.xml .
popd

tests-py-distributed:
name: Test dynamo distributed [Python]
needs: [filter-matrix, build]
strategy:
fail-fast: false
matrix:
include:
- repository: pytorch/tensorrt
package-name: torch_tensorrt
pre-script: packaging/pre_build_script.sh
post-script: packaging/post_build_script.sh
smoke-test-script: packaging/smoke_test_script.sh
uses: ./.github/workflows/linux-test.yml
with:
job-name: tests-py-dynamo-distributed
repository: "pytorch/tensorrt"
ref: ""
test-infra-repository: pytorch/test-infra
test-infra-ref: main
build-matrix: ${{ needs.filter-matrix.outputs.matrix }}
pre-script: ${{ matrix.pre-script }}
script: |
set -euo pipefail
export USE_HOST_DEPS=1
export CI_BUILD=1
export USE_TRTLLM_PLUGINS=1
dnf install -y mpich mpich-devel openmpi openmpi-devel
pushd .
cd tests/py
cd dynamo
python -m pytest -ra --junitxml=${RUNNER_TEST_RESULTS_DIR}/dynamo_distributed_test_results.xml distributed/test_nccl_ops.py
popd

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref_name }}-tensorrt-${{ inputs.repository }}-${{ github.event_name == 'workflow_dispatch' }}-${{ inputs.job-name }}
cancel-in-progress: true
1 change: 1 addition & 0 deletions dev_dep_versions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
__cuda_version__: "12.8"
__tensorrt_version__: "10.12.0"
__tensorrt_rtx_version__: "1.0.0"
__tensorrt_llm_version__: "0.17.0.post1"
Loading
Loading