Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use conda to build python packages during GPU tests #4702

Merged
merged 1 commit into from
May 9, 2022

Conversation

jjacobelli
Copy link
Contributor

@jjacobelli jjacobelli commented Apr 14, 2022

This PR convert the from sources build we are doing in GPU test job to a conda build. This is done for the following reasons:

  • This is required step to improve the Ops CI/CD setup to a more convenient pipeline
  • This is required to start using conda compilers and mamba to build RAPIDS packages
  • This prevent us from manually managing and installing the dependencies in GPU job
  • This ensure the packages can be installed
  • This ensure the tests are running and working against the package content and not the build results. Currently the Python packages are not tested.

This may increase the global pipeline time, but the usage of mamba should resolve this as mamba is faster than conda to build packages

@jjacobelli jjacobelli added 5 - DO NOT MERGE Hold off on merging; see PR for details improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Apr 14, 2022
@jjacobelli jjacobelli self-assigned this Apr 14, 2022
@jjacobelli jjacobelli requested a review from a team as a code owner April 14, 2022 11:36
@jjacobelli jjacobelli added this to PR-WIP in v22.06 Release via automation Apr 14, 2022
@github-actions github-actions bot added the gpuCI gpuCI issue label Apr 14, 2022
@jjacobelli jjacobelli force-pushed the remove-from-sources branch 2 times, most recently from d32f3ed to f8532f1 Compare April 14, 2022 14:58
@jjacobelli jjacobelli marked this pull request as draft April 15, 2022 09:45
@jjacobelli jjacobelli force-pushed the remove-from-sources branch 2 times, most recently from 4d72306 to 143bed8 Compare April 20, 2022 07:31
@github-actions github-actions bot added the Cython / Python Cython or Python issue label Apr 20, 2022
@jjacobelli jjacobelli force-pushed the remove-from-sources branch 5 times, most recently from 9024886 to 26884c8 Compare April 21, 2022 13:18
@codecov-commenter
Copy link

Codecov Report

❗ No coverage uploaded for pull request base (branch-22.06@b3967cf). Click here to learn what that means.
The diff coverage is n/a.

@@               Coverage Diff               @@
##             branch-22.06    #4702   +/-   ##
===============================================
  Coverage                ?   89.25%           
===============================================
  Files                   ?      292           
  Lines                   ?    24990           
  Branches                ?        0           
===============================================
  Hits                    ?    22306           
  Misses                  ?     2684           
  Partials                ?        0           
Flag Coverage Δ
dask 30.87% <0.00%> (?)
non-dask 75.82% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b3967cf...26884c8. Read the comment docs.

@github-actions github-actions bot removed the Cython / Python Cython or Python issue label Apr 22, 2022
@github-actions github-actions bot added the Cython / Python Cython or Python issue label Apr 25, 2022
@jjacobelli jjacobelli removed the 5 - DO NOT MERGE Hold off on merging; see PR for details label Apr 25, 2022
@jjacobelli jjacobelli marked this pull request as ready for review April 25, 2022 10:11
@jjacobelli jjacobelli requested a review from a team as a code owner April 25, 2022 10:11
@jjacobelli jjacobelli force-pushed the remove-from-sources branch 3 times, most recently from a5d7115 to b7ea289 Compare May 5, 2022 08:28
@jjacobelli
Copy link
Contributor Author

rerun tests

Signed-off-by: Jordan Jacobelli <jjacobelli@nvidia.com>
v22.06 Release automation moved this from PR-WIP to PR-Reviewer approved May 9, 2022
ci/gpu/build.sh Show resolved Hide resolved
@jjacobelli
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 3060322 into rapidsai:branch-22.06 May 9, 2022
v22.06 Release automation moved this from PR-Reviewer approved to Done May 9, 2022
vimarsh6739 pushed a commit to vimarsh6739/cuml that referenced this pull request Oct 9, 2023
This PR convert the `from sources` build we are doing in GPU test job to a `conda build`. This is done for the following reasons:
- This is required step to improve the Ops CI/CD setup to a more convenient pipeline
- This is required to start using `conda` compilers and `mamba` to build RAPIDS packages
- This prevent us from manually managing and installing the dependencies in GPU job
- This ensure the packages can be installed
- This ensure the tests are running and working against the package content and not the build results. Currently the Python packages are not tested.

This may increase the global pipeline time, but the usage of `mamba` should resolve this as `mamba` is faster than `conda` to build packages

Authors:
  - Jordan Jacobelli (https://github.com/Ethyling)

Approvers:
  - AJ Schmidt (https://github.com/ajschmidt8)
  - Dante Gama Dessavre (https://github.com/dantegd)

URL: rapidsai#4702
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Cython / Python Cython or Python issue gpuCI gpuCI issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

4 participants