Add supported CPU/GPU operators to API docs and update docstrings #5150

beckernick · 2023-01-23T18:43:27Z

This PR:

Adds a section to the Module Configuration section of the API documentation (next to the section on Verbosity Levels). As device selection capabilities can be configured, this feels like a reasonable location
Updates the relevant estimator docstrings with a reference to device selection (always the final section before Notes) and a link to the documentation to learn more.

This closes #5143 and closes #5144

This PR fixes a minor typo in the cuML intro documentation. Authors: - Christopher Akiki (https://github.com/cakiki) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4695

CMake 3.23.1 contains the bug fixes that we need to make use of CMake 3.23, so now we can update the pinnings to just avoid 3.23.0. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Jordan Jacobelli (https://github.com/Ethyling) - Robert Maynard (https://github.com/robertmaynard) URL: rapidsai#4698

`float64` support in treelite->FIL import and Python layer Authors: - Andy Adinets (https://github.com/canonizer) - Levs Dolgovs (https://github.com/levsnv) Approvers: - Philip Hyunsu Cho (https://github.com/hcho3) - William Hicks (https://github.com/wphicks) URL: rapidsai#4690

Increased test coverage for TreeExplainer, greatly expanding model types tested. New tests take around 4.8s on my machine. Fixes rapidsai#4352 New bugs found: rapidsai#4663 dmlc/treelite#375 rapidsai#4670 Authors: - Rory Mitchell (https://github.com/RAMitchell) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4671

This PR: - [x] Moves all pytest from `test` to `tests` - [x] Converts `test` module to `tests` directory - [x] Creates a `testing` module that contains all utility functions for pytests in `tests` need. Authors: - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Jordan Jacobelli (https://github.com/Ethyling) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4703

@beckernick

In this PR, I'll wrap `dask-glm` models so that it accepts `dask_cudf Dataframes` and behaves like other `cuml.dask` models. `dask-glm` provides three estimators: `LogisticRegression`, `LinearRegression` and `PoissonRegression`. MNMG `LogisticRegression` is requested by @beckernick . @JohnZed for visibility. Thank you all. Authors: - Jiwei Liu (https://github.com/daxiongshu) - Nick Becker (https://github.com/beckernick) - Dante Gama Dessavre (https://github.com/dantegd) Approvers: - Ray Douglass (https://github.com/raydouglass) - AJ Schmidt (https://github.com/ajschmidt8) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#3512

Authors: - Victor Lafargue (https://github.com/viclafargue) - Dante Gama Dessavre (https://github.com/dantegd) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#4706

xref: rapidsai/integration#471 Authors: - https://github.com/jakirkham Approvers: - Dante Gama Dessavre (https://github.com/dantegd) - William Hicks (https://github.com/wphicks) - Ray Douglass (https://github.com/raydouglass) URL: rapidsai#4720

This PR convert the `from sources` build we are doing in GPU test job to a `conda build`. This is done for the following reasons: - This is required step to improve the Ops CI/CD setup to a more convenient pipeline - This is required to start using `conda` compilers and `mamba` to build RAPIDS packages - This prevent us from manually managing and installing the dependencies in GPU job - This ensure the packages can be installed - This ensure the tests are running and working against the package content and not the build results. Currently the Python packages are not tested. This may increase the global pipeline time, but the usage of `mamba` should resolve this as `mamba` is faster than `conda` to build packages Authors: - Jordan Jacobelli (https://github.com/Ethyling) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4702

This PR allows building libcuml++ static libs via CMake option `-DBUILD_SHARED_LIBS=ON|OFF`. ~I was seeing a linker error due to not having `-fPIC` enabled, but I wouldn't have expected this to affect static libs. I'll investigate some more and turn it off if possible, but for now `-fPIC` is enabled.~ Authors: - Paul Taylor (https://github.com/trxcllnt) Approvers: - Robert Maynard (https://github.com/robertmaynard) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4673

Authors: - Victor Lafargue (https://github.com/viclafargue) Approvers: - Corey J. Nolet (https://github.com/cjnolet) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4716

Stacked on rapidsai#4671. - Remove extra redundant class in python layer. - Simplify the interface between C++ and python using variants. - Fix rapidsai#4670 by allowing double precision data - Document TreeExplainer - Add interventional shap method - Add shapley interactions and taylor interactions - Promote from experimental - Support sklearn estimator types from xgb/lgbm (i.e. no need to convert to booster before using TreeExplainer) Authors: - Rory Mitchell (https://github.com/RAMitchell) Approvers: - Philip Hyunsu Cho (https://github.com/hcho3) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4697

Fixes rapidsai#4613 This is basically the same as what scikit-learn does: https://github.com/scikit-learn/scikit-learn/blob/3578a9f9825514d3bfd2f2448bf4e0f95326b938/sklearn/tree/_classes.py#L281-L282 Authors: - Thomas J. Fan (https://github.com/thomasjpfan) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4717

Adjust the name of the option according to the changes proposed in rapidsai/raft#610 Authors: - Artem M. Chirkin (https://github.com/achirkin) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#4718

) Closes rapidsai#4725 rapidsai#3848 removes the usage of `optim_batch_size` in code. This PR removes the parameter from the docstring and in `UMAPParams`. Authors: - Thomas J. Fan (https://github.com/thomasjpfan) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4732

This PR fixes our nightly docs build by removing the `exit` statement at the bottom of this script. In our nightly docs builds, this script is invoked via `source`, so any `exit` statements cause the parent script to exit which prevents completion of the rest of the build. It also removes some unnecessary `pip`/`conda` install commands since these packages exist already in the container. Authors: - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - Sevag Hanssian (https://github.com/sevagh)

Authors: - Corey J. Nolet (https://github.com/cjnolet) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4582

[gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]

This PR enables the usage of conda compilers to build conda packages. This is required to use `mambabuild` Authors: - Jordan Jacobelli (https://github.com/Ethyling) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#4577

[gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]

Simplify recipes using `{{ target_platform }}` for `sysroot` package and `compiler()` function in `ignore_run_exports_from` Authors: - Jordan Jacobelli (https://github.com/Ethyling) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#4749

[gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]

Using mamba should speed up CI Authors: - Jordan Jacobelli (https://github.com/Ethyling) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#4542

Inference for `float64` random forests using FIL. Authors: - Andy Adinets (https://github.com/canonizer) Approvers: - Rory Mitchell (https://github.com/RAMitchell) - William Hicks (https://github.com/wphicks) URL: rapidsai#4739

…sai#4620) This is to resolve issue rapidsai#4589 Authors: - Jiwei Liu (https://github.com/daxiongshu) Approvers: - Micka (https://github.com/lowener) - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#4620

Now that RAFT has a `span` implementation, use that in TreeExplainer. Authors: - Philip Hyunsu Cho (https://github.com/hcho3) - Rory Mitchell (https://github.com/RAMitchell) Approvers: - Jiaming Yuan (https://github.com/trivialfis) - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#4714

…idsai#4509) - Promote `ColumnTransformer` out of experimental - Add `ColumnTransformer` to documentation (answers to rapidsai#4418) - Update preprocessing documentation Authors: - Victor Lafargue (https://github.com/viclafargue) - Dante Gama Dessavre (https://github.com/dantegd) Approvers: - Micka (https://github.com/lowener) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4509

This PR closes issues rapidsai#3123, rapidsai#4704 and rapidsai#4292 Authors: - Victor Lafargue (https://github.com/viclafargue) Approvers: - Corey J. Nolet (https://github.com/cjnolet) - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#4711

Authors: - William Hicks (https://github.com/wphicks) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#5174

Authors: - Carl Simon Adorf (https://github.com/csadorf) - Dante Gama Dessavre (https://github.com/dantegd) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#5168

@ajschmidt8

This PR adds a docs_build process to the PR and Build workflows for this repository. The generated docs are synced to s3 for only the build workflows. cc @ajschmidt8 Authors: - Ajay Thorve (https://github.com/AjayThorve) - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#5155

Authors: - Corey J. Nolet (https://github.com/cjnolet) Approvers: - Micka (https://github.com/lowener) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#5103

@dantegd

The version of `faiss` that's currently used by `libcuml` conflicts with the version used by `raft`. After discussions with @dantegd and @cjnolet, it was determined that `cuml` should really just depend on `raft`'s version of `faiss` anyway. Therefore, this PR removes `faiss` from `libcuml`. Authors: - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - Ray Douglass (https://github.com/raydouglass) URL: rapidsai#5175

viclafargue

LGTM

beckernick · 2023-01-31T18:28:52Z

Docstring updates not being generated as expected in the API docs locally -- TODO before merge

beckernick · 2023-01-31T18:54:03Z

Docstring updates not being generated as expected in the API docs locally -- TODO before merge

Solved. I was not aware until now that the sections names in the docstring are explicitly enforced in the numpydoc code. Based on the recommendations in the numpydoc Style Guide, I'm moving the CPU/GPU lines to the bottom of the Extended Summary rather than Notes.

…code blocks

codecov-commenter · 2023-02-02T11:46:24Z

Codecov Report

Base: 69.26% // Head: 67.12% // Decreases project coverage by -2.15% ⚠️

Coverage data is based on head (642e773) compared to base (de32125).
Patch coverage: 24.44% of modified lines in pull request are covered.

Additional details and impacted files

@@               Coverage Diff                @@
##           branch-23.02    #5150      +/-   ##
================================================
- Coverage         69.26%   67.12%   -2.15%     
================================================
  Files               192      192              
  Lines             12333    12396      +63     
================================================
- Hits               8543     8321     -222     
- Misses             3790     4075     +285

Impacted Files	Coverage Δ
python/cuml/linear_model/lasso.py	`100.00% <ø> (ø)`
python/cuml/testing/strategies.py	`33.76% <19.04%> (-60.31%)`	⬇️
...ing/text/stem/porter_stemmer_utils/suffix_utils.py	`91.11% <100.00%> (+0.20%)`	⬆️
python/cuml/neighbors/kernel_density.py	`52.59% <0.00%> (-35.07%)`	⬇️
python/cuml/metrics/pairwise_kernels.py	`70.07% <0.00%> (-11.03%)`	⬇️
python/cuml/testing/utils.py	`78.63% <0.00%> (-10.39%)`	⬇️
python/cuml/internals/memory_utils.py	`78.18% <0.00%> (-8.19%)`	⬇️
python/cuml/internals/array.py	`81.62% <0.00%> (-6.53%)`	⬇️
python/cuml/feature_extraction/_vectorizers.py	`84.38% <0.00%> (-5.65%)`	⬇️
... and 10 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

beckernick · 2023-02-02T16:52:25Z

Link to @viclafargue 's notebook once #5130 merges

csadorf · 2023-02-03T11:29:28Z

python/cuml/cluster/hdbscan/hdbscan.pyx

+    This estimator supports cuML's experimental device selection capabilities.
+    It can be configured to run on either the CPU or the GPU.
+    To learn more, please see :ref:`device-selection`.
+


Should we maybe automatically add this comment via a common decorator? Something like @device_interop_enabled ? This would also allow us to (maybe at a later stage) set some flag that helps with testing. This could of course be done in a follow-up to not block this PR.

beckernick · 2023-02-17T15:42:03Z

Closing in favor of #5239

) This PR: - Adds a section to the `Module Configuration` section of the API documentation (next to the section on Verbosity Levels). As device selection capabilities can be configured, this feels like a reasonable location - Updates the relevant estimator docstrings with a reference to device selection (always at the bottom of the "Extended Summary" section, based on recommendations in the numpydoc [Style Guide](https://numpydoc.readthedocs.io/en/latest/format.html#sections)) and a link to the documentation to learn more. This closes #5143 and closes #5144 This is a replacement for #5150 Authors: - Nick Becker (https://github.com/beckernick) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: #5239

cakiki and others added 30 commits April 11, 2022 15:24

Change "principals" to "principles" (rapidsai#4695)

dfc1eae

This PR fixes a minor typo in the cuML intro documentation. Authors: - Christopher Akiki (https://github.com/cakiki) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4695

Remove import of scalar_broadcast_to from stemmer (rapidsai#4706)

19d1afc

Authors: - Victor Lafargue (https://github.com/viclafargue) - Dante Gama Dessavre (https://github.com/dantegd) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#4706

Require UCX 1.12.1+ (rapidsai#4720)

7e91505

xref: rapidsai/integration#471 Authors: - https://github.com/jakirkham Approvers: - Dante Gama Dessavre (https://github.com/dantegd) - William Hicks (https://github.com/wphicks) - Ray Douglass (https://github.com/raydouglass) URL: rapidsai#4720

Fix identifier collision (rapidsai#4716)

b487fdc

Authors: - Victor Lafargue (https://github.com/viclafargue) Approvers: - Corey J. Nolet (https://github.com/cjnolet) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4716

Random ball cover algorithm for 3D data (rapidsai#4582)

15aba09

Authors: - Corey J. Nolet (https://github.com/cjnolet) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4582

DOC

9d07811

Merge remote-tracking branch branch-22.04 into branch-22.06

06c3a62

Merge pull request rapidsai#4745 from rapidsai/branch-22.06

c5b517c

[gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]

Use conda compilers (rapidsai#4577)

3e3a1e0

This PR enables the usage of conda compilers to build conda packages. This is required to use `mambabuild` Authors: - Jordan Jacobelli (https://github.com/Ethyling) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#4577

Merge pull request rapidsai#4746 from rapidsai/branch-22.06

d3efc51

[gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]

Merge pull request rapidsai#4750 from rapidsai/branch-22.06

074b98a

[gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]

Build packages using mambabuild (rapidsai#4542)

76c7ffc

Using mamba should speed up CI Authors: - Jordan Jacobelli (https://github.com/Ethyling) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#4542

wphicks and others added 5 commits January 31, 2023 03:05

Increase margin for flaky FIL test (rapidsai#5174)

56d767c

Authors: - William Hicks (https://github.com/wphicks) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#5174

Disable hypothesis tests for 23.02 burndown. (rapidsai#5168)

348a8cb

Authors: - Carl Simon Adorf (https://github.com/csadorf) - Dante Gama Dessavre (https://github.com/dantegd) Approvers: - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#5168

Adding benchmark notebook for hdbscan soft clustering (rapidsai#5103)

fb4bab4

Authors: - Corey J. Nolet (https://github.com/cjnolet) Approvers: - Micka (https://github.com/lowener) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#5103

Merge branch 'branch-23.02' into doc/device-selection-support-matrix

8c53763

beckernick marked this pull request as ready for review January 31, 2023 16:25

beckernick requested a review from a team as a code owner January 31, 2023 16:25

beckernick added doc Documentation non-breaking Non-breaking change labels Jan 31, 2023

beckernick and others added 2 commits January 31, 2023 11:37

flake8 and add to additional estimators

6ecb7ab

viclafargue approved these changes Jan 31, 2023

View reviewed changes

beckernick added 4 commits January 31, 2023 14:01

updated to be compliant with numpydoc conventions

c0320cd

Merge branch 'branch-23.02' into doc/device-selection-support-matrix

5d8d495

fix 3 vs 4 space indent necessary for consistent formatting with RST …

30378e3

…code blocks

fix formatting

642e773

csadorf reviewed Feb 3, 2023

View reviewed changes

ajschmidt8 force-pushed the branch-23.02 branch from 3bc1de0 to e7fd6cc Compare February 13, 2023 18:56

ajschmidt8 requested review from a team as code owners February 13, 2023 18:56

beckernick mentioned this pull request Feb 17, 2023

Add supported CPU/GPU operators to API docs and update docstrings #5239

Merged

beckernick closed this Feb 17, 2023

beckernick deleted the doc/device-selection-support-matrix branch August 9, 2023 16:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add supported CPU/GPU operators to API docs and update docstrings #5150

Add supported CPU/GPU operators to API docs and update docstrings #5150

beckernick commented Jan 23, 2023 •

edited

Loading

viclafargue left a comment

beckernick commented Jan 31, 2023

beckernick commented Jan 31, 2023

codecov-commenter commented Feb 2, 2023

beckernick commented Feb 2, 2023

csadorf Feb 3, 2023 •

edited

Loading

beckernick commented Feb 17, 2023

Add supported CPU/GPU operators to API docs and update docstrings #5150

Add supported CPU/GPU operators to API docs and update docstrings #5150

Conversation

beckernick commented Jan 23, 2023 • edited Loading

viclafargue left a comment

Choose a reason for hiding this comment

beckernick commented Jan 31, 2023

beckernick commented Jan 31, 2023

codecov-commenter commented Feb 2, 2023

Codecov Report

beckernick commented Feb 2, 2023

csadorf Feb 3, 2023 • edited Loading

Choose a reason for hiding this comment

beckernick commented Feb 17, 2023

beckernick commented Jan 23, 2023 •

edited

Loading

csadorf Feb 3, 2023 •

edited

Loading