refactor(build): build pure Python wheel and `noarch: python` conda by gforsyth · Pull Request #738 · rapidsai/cuxfilter

gforsyth · 2025-11-06T17:31:04Z

xref rapidsai/build-planning#43

waiting on rapidsai/shared-workflows#441

…cs build

copy-pr-bot · 2025-11-07T14:10:37Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

jakirkham

Thanks Gil! 🙏

Had a couple minor comments and a question below

jakirkham · 2025-11-13T23:54:16Z

ci/build_docs.sh


 rapids-logger "Downloading artifacts from previous jobs"
-PYTHON_CHANNEL=$(rapids-download-conda-from-github python)
+PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name "conda_python" cuxfilter --pure)")


I don't think shell lets us nest quotes like this. If we do need to nest quotes, the usual way is through concatenation of strings, which involves alternating between single and double quotes

That said, given other commands similar to this one don't use quotes, maybe we can just drop them

Suggested change

PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name "conda_python" cuxfilter --pure)")

PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name conda_python cuxfilter --pure)")

bash doesn't have quoting rules that I like, but they do allow for nesting quotes -- and alternating single and double quotes is dangerous in bash as they have different expansion semantics (specifically that single quotes prevent variable expansion)

jakirkham · 2025-11-13T23:54:38Z

ci/build_wheel.sh


 ../ci/validate_wheel.sh "${RAPIDS_WHEEL_BLD_OUTPUT_DIR}"
+
+RAPIDS_PACKAGE_NAME="$(rapids-package-name wheel_python cuxfilter --pure --cuda "$RAPIDS_CUDA_VERSION")"


Suggested change

RAPIDS_PACKAGE_NAME="$(rapids-package-name wheel_python cuxfilter --pure --cuda "$RAPIDS_CUDA_VERSION")"

RAPIDS_PACKAGE_NAME="$(rapids-package-name wheel_python cuxfilter --pure --cuda ${RAPIDS_CUDA_VERSION})"

I think that's a shellcheck smell -- you always quote around variable expansions to prevent unexpected word-splitting or globbing

jakirkham · 2025-11-13T23:55:21Z

ci/test_notebooks.sh


 rapids-logger "Downloading artifacts from previous jobs"
-PYTHON_CHANNEL=$(rapids-download-conda-from-github python)
+PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name "conda_python" cuxfilter --pure)")


Suggested change

PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name "conda_python" cuxfilter --pure)")

PYTHON_CHANNEL=$(rapids-download-from-github $(rapids-package-name conda_python cuxfilter --pure)")

jakirkham · 2025-11-13T23:55:48Z

ci/test_python.sh


 rapids-logger "Downloading artifacts from previous jobs"
-PYTHON_CHANNEL=$(rapids-download-conda-from-github python)
+PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name "conda_python" cuxfilter --pure)")


Suggested change

PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name "conda_python" cuxfilter --pure)")

PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name conda_python cuxfilter --pure)")

jakirkham · 2025-11-13T23:56:17Z

ci/test_wheel.sh


-RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")"
-CUXFILTER_WHEELHOUSE=$(RAPIDS_PY_WHEEL_NAME="cuxfilter_${RAPIDS_PY_CUDA_SUFFIX}" RAPIDS_PY_WHEEL_PURE="1" rapids-download-wheels-from-github python)
+CUXFILTER_WHEELHOUSE=$(rapids-download-from-github "$(rapids-package-name "wheel_python" cuxfilter --pure --cuda "$RAPIDS_CUDA_VERSION")")


Suggested change

CUXFILTER_WHEELHOUSE=$(rapids-download-from-github "$(rapids-package-name "wheel_python" cuxfilter --pure --cuda "$RAPIDS_CUDA_VERSION")")

CUXFILTER_WHEELHOUSE=$(rapids-download-from-github "$(rapids-package-name wheel_python cuxfilter --pure --cuda ${RAPIDS_CUDA_VERSION})")

jakirkham · 2025-11-13T23:57:33Z

.github/workflows/build.yaml

      date: ${{ inputs.date }}
      script: ci/build_python.sh
      sha: ${{ inputs.sha }}
+      pure-conda: true


Do we need something similar in the case of wheels?

There is already a pure-wheel: true input for wheels -- that's been changed to handle matrix filtering upstream.

gforsyth · 2025-11-17T21:45:43Z

This is ready for another look @jakirkham

jameslamb

It does look to me like this is achieving what we want (only 2 conda builds and 2 wheel builds, 1 per CUDA major) and the package metadata looks correct.

But I left a couple of suggestions on the implementation, enough that I think it's worth blocking the PR.

jameslamb · 2025-12-04T17:07:13Z

ci/build_docs.sh


 rapids-logger "Downloading artifacts from previous jobs"
-PYTHON_CHANNEL=$(rapids-download-conda-from-github python)
+PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name conda_python cuxfilter --pure --cuda "${RAPIDS_CUDA_VERSION}")")


I left a suggestion about this on the upstream issue... I think we'll need to update the RAPIDS developer docs to account for this new need to invoke rapids-package-name:

rapidsai/build-planning#43 (comment)

ci/validate_wheel.sh

jameslamb · 2025-12-04T17:13:45Z

ci/build_wheel.sh

+
+RAPIDS_PACKAGE_NAME="$(rapids-package-name wheel_python cuxfilter --pure --cuda "${RAPIDS_CUDA_VERSION}")"
+export RAPIDS_PACKAGE_NAME


Suggested change

RAPIDS_PACKAGE_NAME="$(rapids-package-name wheel_python cuxfilter --pure --cuda "${RAPIDS_CUDA_VERSION}")"

export RAPIDS_PACKAGE_NAME

Was this left over from an earlier iteration?

It's at the end of this script that isn't called by anything else, so I don't think it'll actually affect anything.

No, that's required -- the conda-python-build shared workflow looks for that environment variable and if it is set it uses that as the artifact name when uploading.

This is build_wheel.sh, I don't think it's involved in the conda build, right?

Oops, missed the filename. The same pattern also applies for wheels-build.yaml

Oh I see, it's setting it for the benefit of this:

https://github.com/rapidsai/shared-workflows/blob/32156d9710c4a6994bb8bab8ddf1441aae03c12c/.github/workflows/wheels-build.yaml#L277

To be used as input in the upload step:

https://github.com/rapidsai/shared-workflows/blob/32156d9710c4a6994bb8bab8ddf1441aae03c12c/.github/workflows/wheels-build.yaml#L291

It wasn't obvious to me that this was modifying something in the environment for the next shared-workflows step to pick up. But I understand how that's working now.

ci/validate_wheel.sh

Co-authored-by: James Lamb <jaylamb20@gmail.com>

jameslamb

No other comments from me, very excited we're improving the handling of libraries like this! Thanks for the explanations.

gforsyth · 2025-12-04T18:56:47Z

/merge

refactor(build): build pure Python wheel and noarch: python conda

167b488

gforsyth requested review from a team as code owners November 6, 2025 17:31

gforsyth requested a review from msarahan November 6, 2025 17:31

gforsyth added 2 commits November 6, 2025 14:26

chore(typo): spell the package name correctly

102e2f4

fix(conda): update artifact name generation for notebook tests and do…

19fcbcd

…cs build

gforsyth added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Nov 6, 2025

gforsyth marked this pull request as draft November 7, 2025 14:10

refactor(matrix): move matrix_filter handling upstream

55fc7c9

gforsyth marked this pull request as ready for review November 7, 2025 15:20

Rerun CI

f7b9fad

bdice approved these changes Nov 7, 2025

View reviewed changes

jakirkham reviewed Nov 13, 2025

View reviewed changes

gforsyth and others added 3 commits November 14, 2025 08:28

chore: remove extraneous quotes from rapids-package-name call

bb9e48d

fix(publish): add wheel artifact prefix override

cd59b4c

Merge branch 'main' into pure_python

f542d30

gforsyth changed the base branch from main to release/25.12 November 17, 2025 20:31

feat(pure-install): retag wheels for linux only

d52fdae

gforsyth changed the base branch from release/25.12 to main November 18, 2025 17:23

gforsyth mentioned this pull request Dec 3, 2025

Properly support building pure Python packages rapidsai/build-planning#43

Closed

Merge branch 'main' into pure_python

a16b92a

gforsyth force-pushed the pure_python branch from 14ab8d3 to fe08335 Compare December 4, 2025 15:12

fix(conda): switch to cuda_major builds for pure conda

20e51d3

gforsyth force-pushed the pure_python branch from fe08335 to 20e51d3 Compare December 4, 2025 16:03

fix(conda): use cuda major version for notebook tests and docs build

e58bcd2

jameslamb requested changes Dec 4, 2025

View reviewed changes

Update ci/validate_wheel.sh

105e87e

Co-authored-by: James Lamb <jaylamb20@gmail.com>

refactor: move wheel retagging to build_wheel.sh

8d5efb7

jameslamb approved these changes Dec 4, 2025

View reviewed changes

rapids-bot bot merged commit b9964b1 into rapidsai:main Dec 4, 2025
26 checks passed

gforsyth deleted the pure_python branch December 4, 2025 18:56

	PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name "conda_python" cuxfilter --pure)")
	PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-package-name conda_python cuxfilter --pure)")


		../ci/validate_wheel.sh "${RAPIDS_WHEEL_BLD_OUTPUT_DIR}"

		RAPIDS_PACKAGE_NAME="$(rapids-package-name wheel_python cuxfilter --pure --cuda "$RAPIDS_CUDA_VERSION")"

	CUXFILTER_WHEELHOUSE=$(rapids-download-from-github "$(rapids-package-name "wheel_python" cuxfilter --pure --cuda "$RAPIDS_CUDA_VERSION")")
	CUXFILTER_WHEELHOUSE=$(rapids-download-from-github "$(rapids-package-name wheel_python cuxfilter --pure --cuda ${RAPIDS_CUDA_VERSION})")


		RAPIDS_PACKAGE_NAME="$(rapids-package-name wheel_python cuxfilter --pure --cuda "${RAPIDS_CUDA_VERSION}")"
		export RAPIDS_PACKAGE_NAME

Conversation

gforsyth commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

copy-pr-bot bot commented Nov 7, 2025

Uh oh!

jakirkham left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gforsyth commented Nov 17, 2025

Uh oh!

jameslamb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jameslamb left a comment

Choose a reason for hiding this comment

Uh oh!

gforsyth commented Dec 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gforsyth commented Nov 6, 2025 •

edited

Loading