Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix xgboost converter #373

Merged
merged 10 commits into from
Mar 5, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions .azure-pipelines/linux-conda-CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,24 +13,33 @@ jobs:
vmImage: 'Ubuntu-16.04'
strategy:
matrix:
Python27:
Python27-123-RT040:
python.version: '2.7'
ONNX_PATH: onnx==1.2.3
ONNXRT_PATH: onnxruntime==0.4.0
xgboost.version: ''
# Python35:
# python.version: '3.5'
Python36:
Python36-141-RT050:
python.version: '3.6'
ONNX_PATH: onnx==1.4.1
ONNXRT_PATH: onnxruntime==0.5.0
Python37:
xgboost.version: ''
Python37-150-RT100:
python.version: '3.7'
ONNX_PATH: onnx==1.5.0
ONNXRT_PATH: onnxruntime==1.0.0
Python37-RT100:
xgboost.version: ''
Python37-160-RT111-XGB0:
python.version: '3.7'
ONNX_PATH: onnx==1.6.0
ONNXRT_PATH: onnxruntime==1.0.0
ONNXRT_PATH: onnxruntime==1.1.1
xgboost.version: '<1.0'
Python37-160-RT111:
python.version: '3.7'
ONNX_PATH: onnx==1.6.0
ONNXRT_PATH: onnxruntime==1.1.1
xgboost.version: '>=1.0'
maxParallel: 3

steps:
Expand All @@ -56,6 +65,7 @@ jobs:
python -m pip install git+https://github.com/onnx/keras-onnx
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install xgboost$(xgboost.version)
test '$(python.version)' != '2.7' && pip install $(ONNXRT_PATH)
pip install pytest
git clone --recursive https://github.com/cjlin1/libsvm libsvm
Expand Down
15 changes: 10 additions & 5 deletions .azure-pipelines/win32-conda-CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,29 +13,33 @@ jobs:
vmImage: 'vs2017-win2016'
strategy:
matrix:
Python35:
Python35-141-RT030:
python.version: '3.5'
ONNX_PATH: onnx==1.4.1
ONNXRT_PATH: onnxruntime==0.3.0
COREML_PATH: https://github.com/apple/coremltools/archive/v2.0.zip
sklearn.version: '==0.19.1'

Python36:
Python36-141-RT030:
python.version: '3.6'
ONNX_PATH: onnx==1.4.1
ONNXRT_PATH: onnxruntime==0.3.0
COREML_PATH: git+https://github.com/apple/coremltools@3.1
sklearn.version: ''

Python37:
Python37-150-RT040:
python.version: '3.7'
ONNX_PATH: onnx==1.5.0
ONNXRT_PATH: onnxruntime==0.4.0
COREML_PATH: git+https://github.com/apple/coremltools@3.1
sklearn.version: ''

Python37-RT100:
Python37-160-RT111:
python.version: '3.7'
ONNX_PATH: onnx==1.6.0
ONNXRT_PATH: onnxruntime==1.0.0
ONNXRT_PATH: onnxruntime==1.1.1
COREML_PATH: git+https://github.com/apple/coremltools@3.1
sklearn.version: ''

maxParallel: 3

Expand Down Expand Up @@ -64,6 +68,7 @@ jobs:
python -m pip install -r requirements.txt
python -m pip install -r requirements-dev.txt
python -m pip install %ONNXRT_PATH%
python -m pip install scikit-learn$(sklearn.version)
echo Test onnxruntime installation... && python -c "import onnxruntime"
REM install libsvm from github
git clone --recursive https://github.com/cjlin1/libsvm libsvm
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,14 @@ def save_read_sparkml_model_data(spark, model):
import tempfile
import os
import time
path = os.path.join(tempfile.tempdir, type(model).__name__ + "_" + str(time.time()))
#print(path)
tdir = tempfile.tempdir
if tdir is None:
tdir = spark.util.Utils.createTempDir().getAbsolutePath()
if tdir is None:
raise FileNotFoundError(
"Unable to create a temporary directory for model '{}'"
".".format(type(model).__name__))
path = os.path.join(tdir, type(model).__name__ + "_" + str(time.time()))
model.write().overwrite().save(path)
df = spark.read.parquet(os.path.join(path, 'data'))
#df.show(100)
return df
4 changes: 4 additions & 0 deletions onnxmltools/convert/xgboost/operator_converters/XGBoost.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ def validate(xgb_node):
raise AttributeError('ojective')
except AttributeError as e:
raise RuntimeError('Missing attribute in XGBoost model ' + str(e))
if hasattr(xgb_node, 'missing') and not np.isnan(xgb_node.missing):
raise RuntimeError("Cannot convert a XGBoost model where missing values are not "
"nan but {}.".format(xgb_node.missing))

@staticmethod
def common_members(xgb_node, inputs):
Expand Down Expand Up @@ -277,6 +280,7 @@ def convert_xgboost(scope, operator, container):
cls = XGBClassifierConverter
else:
cls = XGBRegressorConverter
cls.validate(xgb_node)
cls.convert(scope, operator, container)


Expand Down
3 changes: 2 additions & 1 deletion requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
codecov
coremltools
cython
lightgbm
h2o
mleap
Expand All @@ -13,4 +14,4 @@ scikit-learn
scipy
svm
wheel
xgboost<=0.90
xgboost
6 changes: 6 additions & 0 deletions tests/h2o/test_h2o_converters.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"""
import unittest
import os
import sys
import numpy as np
import pandas as pd
from sklearn.datasets import load_diabetes, load_iris, make_classification
Expand Down Expand Up @@ -183,6 +184,7 @@ def test_h2o_regressor(self):
"< StrictVersion('1.3.0')",
)

@unittest.skipIf(sys.version_info[:2] <= (3, 5), reason="not available")
def test_h2o_regressor_cat(self):
y = "IsDepDelayed"
train, test = _prepare_one_hot("airlines.csv", y, exclude_cols=["IsDepDelayed_REC"])
Expand All @@ -207,6 +209,8 @@ def test_h2o_classifier_multi_2class(self):
_convert_mojo(mojo_path)
self.assertRegexpMatches(err.exception.args[0], "not supported")


@unittest.skipIf(sys.version_info[:2] <= (3, 5), reason="not available")
def test_h2o_classifier_bin_cat(self):
y = "IsDepDelayed_REC"
train, test = _prepare_one_hot("airlines.csv", y, exclude_cols=["IsDepDelayed"])
Expand All @@ -224,6 +228,8 @@ def test_h2o_classifier_bin_cat(self):
"< StrictVersion('1.3.0')",
)


@unittest.skipIf(sys.version_info[:2] <= (3, 5), reason="not available")
def test_h2o_classifier_multi_cat(self):
y = "fYear"
train, test = _prepare_one_hot("airlines.csv", y)
Expand Down
Loading