Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
b73cb81
[chore] Use sccache in builds
mdboom Oct 17, 2025
b4c2849
Set compiler env vars
mdboom Oct 17, 2025
7f6c222
Add the other special envvar
mdboom Oct 17, 2025
2d8d2c8
Try adding vars to cibuildwheel
mdboom Oct 17, 2025
577fa3d
Try a different way to pass vars fo cibuildwheel
mdboom Oct 17, 2025
5605da0
Try another approach
mdboom Oct 17, 2025
41f1359
Adjust SCCACHE_DIR inside the container
mdboom Oct 17, 2025
4d3b284
Specify a cache dir
mdboom Oct 17, 2025
5b15052
Try something different
mdboom Oct 17, 2025
a0e0603
Remove explicit SCCACHE_DIR manipulation
mdboom Oct 17, 2025
303e91d
Add envvar
mdboom Oct 17, 2025
4bb10e5
Explicitly start the host
mdboom Oct 17, 2025
8732bca
Pass along GHA cache url
mdboom Oct 17, 2025
a542dc5
Expose the necessary tokens
mdboom Oct 17, 2025
a032a0a
Try a different approach
mdboom Oct 17, 2025
3db7f80
Is the token enough?
mdboom Oct 17, 2025
d824864
Does this work?
mdboom Oct 17, 2025
bec6624
ACTIONS_RESULTS_URL
mdboom Oct 17, 2025
eaf3f4a
Get some debug
mdboom Oct 17, 2025
b5b0af4
fix syntax
leofang Oct 20, 2025
b608357
try to get accurate cache hit/miss rates
leofang Oct 20, 2025
ee75abe
debug
leofang Oct 20, 2025
f7a986f
fix windows
leofang Oct 20, 2025
9b0d041
Try to fix cache writing
leofang Oct 20, 2025
3ad8ea7
restore SCCACHE_GHA_ENABLED
leofang Oct 20, 2025
e2ec4df
set up env vars in build stage
leofang Oct 20, 2025
319d289
pass env vars to cibuildwheel
leofang Oct 20, 2025
dc19abe
fix
leofang Oct 20, 2025
d0757ee
try this
leofang Oct 20, 2025
35df9ff
Merge branch 'main' into use-sccache
leofang Oct 20, 2025
4d31536
try this
leofang Oct 20, 2025
93b92b7
start server on host and make container aware of the ports
leofang Oct 20, 2025
a56abde
Revert "start server on host and make container aware of the ports"
leofang Oct 20, 2025
cf6a476
try local build approach with all env vars passed to container
leofang Oct 20, 2025
d164f8a
Merge branch 'main' into use-sccache
leofang Oct 21, 2025
c10dc60
clean up & apply sccache to cuda.core builds
leofang Oct 21, 2025
3ad2235
Add steps to find cl.exe and generate wrapper script
leofang Oct 23, 2025
7fd19cd
Merge branch 'main' into use-sccache
leofang Oct 23, 2025
d5243c1
fix
leofang Oct 23, 2025
20e0ce3
defer format normalization
leofang Oct 23, 2025
ce6c03d
debug
leofang Oct 23, 2025
814fb82
debug
leofang Oct 23, 2025
b9cc2b3
debug
leofang Oct 23, 2025
d1d804f
debug
leofang Oct 23, 2025
7de169b
debug
leofang Oct 23, 2025
541a9b2
debug
leofang Oct 23, 2025
098feeb
debug
leofang Oct 23, 2025
8f6f008
debug
leofang Oct 23, 2025
65beac0
use big gun
leofang Oct 23, 2025
7cac30d
fix
leofang Oct 23, 2025
6be8df7
debug
leofang Oct 23, 2025
1eb6a1b
fix sccache dir
leofang Oct 23, 2025
6cfe70d
Remove windows experimentations
mdboom Oct 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .github/workflows/build-wheel.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,19 @@ jobs:
with:
fetch-depth: 0

# The env vars ACTIONS_CACHE_SERVICE_V2, ACTIONS_RESULTS_URL, and ACTIONS_RUNTIME_TOKEN
# are exposed by this action.
- name: Enable sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # 0.0.9

# xref: https://github.com/orgs/community/discussions/42856#discussioncomment-7678867
- name: Adding addtional GHA cache-related env vars
uses: actions/github-script@v7
with:
script: |
core.exportVariable('ACTIONS_CACHE_URL', process.env['ACTIONS_CACHE_URL'])
core.exportVariable('ACTIONS_RUNTIME_URL', process.env['ACTIONS_RUNTIME_URL'])

- name: Setup proxy cache
uses: nv-gha-runners/setup-proxy-cache@main
continue-on-error: true
Expand Down Expand Up @@ -129,9 +142,27 @@ jobs:
CIBW_ENVIRONMENT_LINUX: >
CUDA_PATH=/host/${{ env.CUDA_PATH }}
CUDA_PYTHON_PARALLEL_LEVEL=${{ env.CUDA_PYTHON_PARALLEL_LEVEL }}
CC="/host/${{ env.SCCACHE_PATH }} cc"
CXX="/host/${{ env.SCCACHE_PATH }} c++"
SCCACHE_GHA_ENABLED=true
ACTIONS_RUNTIME_TOKEN=${{ env.ACTIONS_RUNTIME_TOKEN }}
ACTIONS_RUNTIME_URL=${{ env.ACTIONS_RUNTIME_URL }}
ACTIONS_RESULTS_URL=${{ env.ACTIONS_RESULTS_URL }}
ACTIONS_CACHE_URL=${{ env.ACTIONS_CACHE_URL }}
ACTIONS_CACHE_SERVICE_V2=${{ env.ACTIONS_CACHE_SERVICE_V2 }}
SCCACHE_DIR=/host/${{ env.SCCACHE_DIR }}
SCCACHE_CACHE_SIZE=${{ env.SCCACHE_CACHE_SIZE }}
CIBW_ENVIRONMENT_WINDOWS: >
CUDA_PATH="$(cygpath -w ${{ env.CUDA_PATH }})"
CUDA_PYTHON_PARALLEL_LEVEL=${{ env.CUDA_PYTHON_PARALLEL_LEVEL }}
# check cache stats before leaving cibuildwheel
CIBW_BEFORE_TEST_LINUX: >
"/host/${{ env.SCCACHE_PATH }}" --show-stats
# force the test stage to be run (so that before-test is not skipped)
# TODO: we might want to think twice on adding this, it does a lot of
# things before reaching this command.
CIBW_TEST_COMMAND: >
echo "ok!"

- name: List the cuda.bindings artifacts directory
run: |
Expand Down Expand Up @@ -167,11 +198,29 @@ jobs:
CUDA_PYTHON_PARALLEL_LEVEL=${{ env.CUDA_PYTHON_PARALLEL_LEVEL }}
CUDA_CORE_BUILD_MAJOR=${{ env.BUILD_CUDA_MAJOR }}
PIP_FIND_LINKS=/host/${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}
CC="/host/${{ env.SCCACHE_PATH }} cc"
CXX="/host/${{ env.SCCACHE_PATH }} c++"
SCCACHE_GHA_ENABLED=true
ACTIONS_RUNTIME_TOKEN=${{ env.ACTIONS_RUNTIME_TOKEN }}
ACTIONS_RUNTIME_URL=${{ env.ACTIONS_RUNTIME_URL }}
ACTIONS_RESULTS_URL=${{ env.ACTIONS_RESULTS_URL }}
ACTIONS_CACHE_URL=${{ env.ACTIONS_CACHE_URL }}
ACTIONS_CACHE_SERVICE_V2=${{ env.ACTIONS_CACHE_SERVICE_V2 }}
SCCACHE_DIR=/host/${{ env.SCCACHE_DIR }}
SCCACHE_CACHE_SIZE=${{ env.SCCACHE_CACHE_SIZE }}
CIBW_ENVIRONMENT_WINDOWS: >
CUDA_PATH="$(cygpath -w ${{ env.CUDA_PATH }})"
CUDA_PYTHON_PARALLEL_LEVEL=${{ env.CUDA_PYTHON_PARALLEL_LEVEL }}
CUDA_CORE_BUILD_MAJOR=${{ env.BUILD_CUDA_MAJOR }}
PIP_FIND_LINKS="$(cygpath -w ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }})"
# check cache stats before leaving cibuildwheel
CIBW_BEFORE_TEST_LINUX: >
"/host${{ env.SCCACHE_PATH }}" --show-stats
# force the test stage to be run (so that before-test is not skipped)
# TODO: we might want to think twice on adding this, it does a lot of
# things before reaching this command.
CIBW_TEST_COMMAND: >
echo "ok!"

- name: List the cuda.core artifacts directory and rename
run: |
Expand Down Expand Up @@ -330,11 +379,29 @@ jobs:
CUDA_PYTHON_PARALLEL_LEVEL=${{ env.CUDA_PYTHON_PARALLEL_LEVEL }}
CUDA_CORE_BUILD_MAJOR=${{ env.BUILD_PREV_CUDA_MAJOR }}
PIP_FIND_LINKS=/host/${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}
CC="/host/${{ env.SCCACHE_PATH }} cc"
CXX="/host/${{ env.SCCACHE_PATH }} c++"
SCCACHE_GHA_ENABLED=true
ACTIONS_RUNTIME_TOKEN=${{ env.ACTIONS_RUNTIME_TOKEN }}
ACTIONS_RUNTIME_URL=${{ env.ACTIONS_RUNTIME_URL }}
ACTIONS_RESULTS_URL=${{ env.ACTIONS_RESULTS_URL }}
ACTIONS_CACHE_URL=${{ env.ACTIONS_CACHE_URL }}
ACTIONS_CACHE_SERVICE_V2=${{ env.ACTIONS_CACHE_SERVICE_V2 }}
SCCACHE_DIR=/host/${{ env.SCCACHE_DIR }}
SCCACHE_CACHE_SIZE=${{ env.SCCACHE_CACHE_SIZE }}
CIBW_ENVIRONMENT_WINDOWS: >
CUDA_PATH="$(cygpath -w ${{ env.CUDA_PATH }})"
CUDA_PYTHON_PARALLEL_LEVEL=${{ env.CUDA_PYTHON_PARALLEL_LEVEL }}
CUDA_CORE_BUILD_MAJOR=${{ env.BUILD_PREV_CUDA_MAJOR }}
PIP_FIND_LINKS="$(cygpath -w ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }})"
# check cache stats before leaving cibuildwheel
CIBW_BEFORE_TEST_LINUX: >
"/host${{ env.SCCACHE_PATH }}" --show-stats
# force the test stage to be run (so that before-test is not skipped)
# TODO: we might want to think twice on adding this, it does a lot of
# things before reaching this command.
CIBW_TEST_COMMAND: >
echo "ok!"

- name: List the cuda.core artifacts directory and rename
run: |
Expand Down
3 changes: 3 additions & 0 deletions ci/tools/env-vars
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ if [[ "${1}" == "build" ]]; then
echo "BUILD_CUDA_MAJOR=${BUILD_CUDA_MAJOR}" >> $GITHUB_ENV
echo "BUILD_PREV_CUDA_MAJOR=$((${BUILD_CUDA_MAJOR} - 1))" >> $GITHUB_ENV
CUDA_BINDINGS_ARTIFACT_BASENAME="cuda-bindings-python${PYTHON_VERSION_FORMATTED}-cuda${CUDA_VER}-${HOST_PLATFORM}"
# Enforce an explicit cache dir so that we can reuse this path later
echo "SCCACHE_DIR=${HOME}/.cache/sccache" >> $GITHUB_ENV
echo "SCCACHE_CACHE_SIZE=1G" >> $GITHUB_ENV
elif [[ "${1}" == "test" ]]; then
BUILD_CUDA_MAJOR="$(cut -d '.' -f 1 <<< ${BUILD_CUDA_VER})"
TEST_CUDA_MAJOR="$(cut -d '.' -f 1 <<< ${CUDA_VER})"
Expand Down
Loading