FIL to import categorical models from treelite #4173

levsnv · 2021-08-23T17:00:24Z

importing forests with categorical nodes from Treelite
python tests for lightgbm forests fit on categorical features and containing categorical nodes as well as numerical nodes in the same forest
above tests include multiclass (GBDT) tests
sklearn internally transforms categorical features into numerical: RandomForestClassifier GradientBoostingClassifier
(cuml RF tests can come separately, if needed)
(xgboost tests can come separately, if needed)

This reverts commit 472575d.

cpu inference supports categorical nodes data generation supports categorical features moved is_categoricals_h generation from GPU to CPU (value flow more obvious) fitted parameters to conventions neater memory management

…raction

…osed to forest construction one

…cal-no-import

…tegorical-treelite

canonizer

Approved, provided that the comments are addressed.

cpp/src/fil/fil.cu

cpp/src/fil/internal.cuh

cpp/test/sg/fil_test.cu

This reverts commit 8ccdb27.

codecov-commenter · 2021-09-25T07:30:49Z

Codecov Report

❗ No coverage uploaded for pull request base (branch-21.10@d0aaafc). Click here to learn what that means.
The diff coverage is n/a.

@@               Coverage Diff               @@
##             branch-21.10    #4173   +/-   ##
===============================================
  Coverage                ?   86.07%           
===============================================
  Files                   ?      231           
  Lines                   ?    18652           
  Branches                ?        0           
===============================================
  Hits                    ?    16055           
  Misses                  ?     2597           
  Partials                ?        0

Flag	Coverage Δ
dask	`47.03% <0.00%> (?)`
non-dask	`78.74% <0.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d0aaafc...fb67202. Read the comment docs.

wphicks

LGTM! Great work!

teju85 · 2021-09-28T08:43:02Z

rerun tests

levsnv · 2021-09-28T21:09:10Z

rerun tests

levsnv · 2021-09-28T21:56:49Z

rerun tests

conda_package_handling.exceptions.InvalidArchiveError: Error with archive /opt/conda/envs/rapids/pkgs/cudatoolkit-11.2.72-h2bc3f7f_0.tar.bz2.  You probably need to delete and re-download or re-create this file.  Message from libarchive 
("14:30:32 EOFError: Compressed file ended before the end-of-stream marker was reached"

dantegd · 2021-09-29T04:17:07Z

@gpucibot merge

dantegd · 2021-09-29T11:08:41Z

rerun tests

dantegd · 2021-09-29T11:27:26Z

@gpucibot merge

dantegd · 2021-09-29T17:29:32Z

@gpucibot merge

* importing forests with categorical nodes from Treelite * python tests for lightgbm forests fit on categorical features and containing categorical nodes as well as numerical nodes in the same forest * above tests include multiclass (GBDT) tests * sklearn internally transforms categorical features into numerical: [RandomForestClassifier](https://scikit-learn.org/0.24/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier.fit) [GradientBoostingClassifier](https://scikit-learn.org/0.24/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html#sklearn.ensemble.GradientBoostingClassifier.fit) * (cuml RF tests can come separately, if needed) * (xgboost tests can come separately, if needed) Authors: - Levs Dolgovs (https://github.com/levsnv) - Dante Gama Dessavre (https://github.com/dantegd) Approvers: - https://github.com/Salonijain27 - Andy Adinets (https://github.com/canonizer) - William Hicks (https://github.com/wphicks) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#4173

levsnv added 30 commits June 1, 2021 14:24

a sketch

4df552a

Merge branch 'branch-21.08' of github.com:rapidsai/cuml into HEAD

44277f6

added inference code

81f8415

Merge branch 'branch-21.08' of github.com:rapidsai/cuml into HEAD

5d3d81e

missed some errors

a70b5ef

now compiles

0a03a3e

stash

472575d

Revert "stash"

efe875c

This reverts commit 472575d.

enums are more printable now

38bb778

fixed some bugs; added debug prints

9cc2549

Merge branch 'branch-21.08' of github.com:rapidsai/cuml into categorical

a14d364

stack trace

f6e89e6

implemented omp reduction for memory pool size estimation

a9c011d

added allocation/copy code

daec42b

implemented filling up the bit pool with category masks

361afaf

fixed a type mismatch, added assert

cf46cc0

drafting categorical node and set generation in test code

c28fba3

fixed compiler but not runtime issues

d89d082

many things

67157b0

cpu inference supports categorical nodes data generation supports categorical features moved is_categoricals_h generation from GPU to CPU (value flow more obvious) fitted parameters to conventions neater memory management

debugging

64cb540

fixed some bugs

5608266

...

11a4c2e

fixed some bugs

d75d620

added FIL categorical inference and test generation, no Treelite inte…

63aa434

…raction

this was used to measure overhead when template parameter is irrelevant

a4ca1c0

reverted benchmark; started on <can_be_categorical> propagation

6be93b5

moved <can_be_categorical> to an inference template parameter, as opp…

3559efc

…osed to forest construction one

fixed <branch_can_be_categorical> dispatching

ef95861

Merge branch 'branch-21.08' of github.com:rapidsai/cuml into categori…

45a642e

…cal-no-import

addressed review comments

3b24ea2

levsnv added 4 commits September 23, 2021 23:32

small review comments

8c5800d

todo

baf7c52

style

29b4f6e

Merge branch 'categorical-treelite' of github.com:levsnv/cuml into ca…

c7d9110

…tegorical-treelite

levsnv added 3 - Ready for Review Ready for review by team and removed 4 - Waiting on Author Waiting for author to respond to review labels Sep 24, 2021

levsnv requested a review from canonizer September 24, 2021 06:38

remove empty category list workaround

8ccdb27

canonizer approved these changes Sep 24, 2021

View reviewed changes

levsnv added 2 commits September 24, 2021 18:37

Revert "remove empty category list workaround"

2ef383f

This reverts commit 8ccdb27.

addressed review comments

fb67202

levsnv requested a review from wphicks September 28, 2021 00:47

size_t, uint32_t, int32_t

8b8ccec

wphicks approved these changes Sep 28, 2021

View reviewed changes

dantegd approved these changes Sep 29, 2021

View reviewed changes

v21.10 Release automation moved this from PR-Needs review to PR-Reviewer approved Sep 29, 2021

levsnv mentioned this pull request Sep 29, 2021

[BUG] within cuml, Treelite might not handle empty categorical nodes #4246

Open

Merge branch 'branch-21.10' into categorical-treelite

36f090e

rapids-bot bot merged commit 003f6fc into rapidsai:branch-21.10 Sep 29, 2021

v21.10 Release automation moved this from PR-Reviewer approved to Done Sep 29, 2021

dantegd mentioned this pull request Oct 11, 2021

[FEA] Forest Inference Library categorical variables #2660

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FIL to import categorical models from treelite #4173

FIL to import categorical models from treelite #4173

levsnv commented Aug 23, 2021 •

edited

canonizer left a comment

codecov-commenter commented Sep 25, 2021

wphicks left a comment

teju85 commented Sep 28, 2021

levsnv commented Sep 28, 2021

levsnv commented Sep 28, 2021

dantegd commented Sep 29, 2021

dantegd commented Sep 29, 2021

dantegd commented Sep 29, 2021

dantegd commented Sep 29, 2021

FIL to import categorical models from treelite #4173

FIL to import categorical models from treelite #4173

Conversation

levsnv commented Aug 23, 2021 • edited

canonizer left a comment

Choose a reason for hiding this comment

codecov-commenter commented Sep 25, 2021

Codecov Report

wphicks left a comment

Choose a reason for hiding this comment

teju85 commented Sep 28, 2021

levsnv commented Sep 28, 2021

levsnv commented Sep 28, 2021

dantegd commented Sep 29, 2021

dantegd commented Sep 29, 2021

dantegd commented Sep 29, 2021

dantegd commented Sep 29, 2021

levsnv commented Aug 23, 2021 •

edited