Catboost vs ONNX runtime : very large RAM usage. Expected ? #4959
Replies: 2 comments 2 replies
-
Technically, this would appear to be an |
Beta Was this translation helpful? Give feedback.
-
As @cbourjau said, there is no rule (x in set(...)) in onnx, these rule are unfolded into many rule ==. That would explain the memory increase. This is true for lightgbm. The function doing that is https://github.com/onnx/onnxmltools/blob/main/onnxmltools/convert/lightgbm/operator_converters/LightGbm.py#L673. However, the catboost exporter is implemented by catboost. I did not look what they do in that specific case. onnxruntime can handle DAGs. onnxruntime does not support sparse but unless you run a big batch, it should not explain the huge memory increase. It could be possible to create new features based on nodes using a rule (x in set(...)). This should reduce the memory increase but it could lead to the addition of many features. You may want to use the latest nightly build. I made some modifications last week to reduce the memory footprint by 60% for TreeEnsemble in onnxruntime. |
Beta Was this translation helpful? Give feedback.
-
Hi,
i have a model from catboost. It weights ~ 100MB on disk (dumped by joblib) and around 250MB in RAM when loaded "natively" with catboost. If i save it as onnx format using catboost save_model() function, it weights ~ 500MB on disk and when loaded in an InferenceSession, it takes ~ 5.5GB in ram ! So roughly 22 times the RAM usage of native catboost.
Is it expected ?
Btw, there's also performance degradation using ort : 10k predictions take 32s with catboost, but ~55s with ort
Doing this CPU only, with python 3.11, catboost 1.1.1 and onnxruntime 1.14.1
Am i missing something ?
Thanks for any advice
Beta Was this translation helpful? Give feedback.
All reactions