[RMP] Support Tree Ranking Models (like XGBoost) in Merlin Models and Systems

[viswa-nvidia]

Problem

Gradient-boosted decision trees (GBDTs) are commonly used in the industry as part of the scoring phase of recommender systems. Supporting serving of these models and integrating with the Merlin ecosystem will help facilitate usage of these models in these systems.

The Triton Inference Server has a backend called FIL (Forest Inference Library) to facilitate GPU accelerated serving of these models.

Random forests (RF) and gradient-boosted decision trees (GBDTs) have become workhorse models of applied machine learning. XGBoost and LightGBM, popular packages implementing GBDT models, consistently rank among the most commonly used tools by data scientists on the Kaggle platform. We see similar interest in forest-based models in industry, where they are applied to problems ranging from inventory forecasting, to ad ranking, to medical diagnostics.

RAPIDS Forest Inference Library: Prediction at 100 million rows per second

Goals

• Enable the use of Tree based models (e.g. GBDTs, Random Forests) in a Merlin Systems ensemble.
• Support the training of XGBoost models from a Merlin Dataset.

Constraints

• Use the Triton FIL backend in Merlin systems

Starting Point

Merlin-models (Data Scientist)

• XGBoost Integration models#132
  • First version of API
    • First sketch of XGBoost integration with Merlin Dataset models#433
    • XGBoost - Switch to dask API models#466
  • Example
    • add xgboost example models#522

NVTabular (Data Scientist)

• [NA] Operators for batch prediction with these models
• Note: Batch prediction is not in scope for this development

Merlin-systems (Product Engineer)

• FIL integration in ensemble API.
  Supports the following models: XGBoost, LightGBM, Scikit-Learn (Random Forest), cuML (Random Forest)
  • Add Forest Inference Operator  systems#118
  • Add an Operator for Forest Inference Libary (FIL) Triton backend. systems#110

Examples and Docs (Everyone)

• Example of training tree models ( End to End example )
• Example of serving tree models ( End to End example ) models#575

Aha! Link: https://nvaiinfa.aha.io/features/MERLIN-828

[viswa-nvidia]

@benfred , could you map your XGboost tickets here ?

[karlhigley]

@marcromeyn Re-assigned this to you for the time being since it sounds like you'll be working with our new hire on both the Models and Systems sides. Once he's in our GH org, might make him the lead on this though?

[oliverholworthy]

An update on this issue.

Merlin Models

We have a first version of the XGBoost API in Merlin Models from release 22.06. Once we have an example that part can be considered complete, for the purposes of this issue. With ongoing support for feature improvements via other issues.

Merlin Systems

Operators have been added to Merlin Systems to enable XGBoost, LightGBM, Scikit-Learn (Random Forest), cuML (Random Forest) models being added as part of a serving ensemble in Triton. There are a couple of small issues being addressed (operator outputs) to make this usable from the next release 22.07.

NVTabular

The comment "NVTabular - Operators for batch prediction with these models" in the description I think may be out of scope of this issue. At least, it's unclear to me what the relationship with NVTabular is in this context. In terms of batch prediction for the XGBoost integration in Merlin Models. This is supported as part of the predict method, since we're using the Dask version of XGBoost, which can be used in a distributed setting.

In terms of a common pattern for running evaluation / which will involve batch/distributed prediction. The ongoing work in this area may be covered by #407 or #405

[viswa-nvidia]

@radekosmulski , could you please add the ticket tracking the examples here in the top

[viswa-nvidia]

Closing this ticket. Example is pending and is planned for 22.08