Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for H2O GBM MOJO #358

Merged
merged 1 commit into from
Jan 6, 2020
Merged

Add support for H2O GBM MOJO #358

merged 1 commit into from
Jan 6, 2020

Conversation

honzasterba
Copy link
Contributor

No description provided.

@claassistantio
Copy link

claassistantio commented Dec 12, 2019 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@lgtm-com
Copy link

lgtm-com bot commented Dec 12, 2019

This pull request introduces 6 alerts when merging 48f0eec into 8ce90a3 - view on LGTM.com

new alerts:

  • 2 for Use of the return value of a procedure
  • 2 for Unused import
  • 1 for Unreachable code
  • 1 for 'import *' may pollute namespace

@lgtm-com
Copy link

lgtm-com bot commented Dec 13, 2019

This pull request introduces 5 alerts when merging 035f097 into 8ce90a3 - view on LGTM.com

new alerts:

  • 2 for Use of the return value of a procedure
  • 2 for Unused import
  • 1 for 'import *' may pollute namespace

@lgtm-com
Copy link

lgtm-com bot commented Dec 13, 2019

This pull request introduces 5 alerts when merging 7adb7c4 into 8ce90a3 - view on LGTM.com

new alerts:

  • 2 for Use of the return value of a procedure
  • 2 for Unused import
  • 1 for 'import *' may pollute namespace

@lgtm-com
Copy link

lgtm-com bot commented Dec 13, 2019

This pull request introduces 2 alerts when merging 17da084 into 8ce90a3 - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

@lgtm-com
Copy link

lgtm-com bot commented Dec 13, 2019

This pull request introduces 2 alerts when merging 2c5372e into 8ce90a3 - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

@lgtm-com
Copy link

lgtm-com bot commented Dec 13, 2019

This pull request introduces 2 alerts when merging eead297 into 8ce90a3 - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

@honzasterba
Copy link
Contributor Author

waiting for next release of H2O for the tests to start passing

@lgtm-com
Copy link

lgtm-com bot commented Dec 18, 2019

This pull request introduces 2 alerts when merging dc06f9e into 9b34fc4 - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

@honzasterba honzasterba changed the title [WIP] Add support for H2O GBM MOJO Add support for H2O GBM MOJO Dec 18, 2019
@lgtm-com
Copy link

lgtm-com bot commented Dec 18, 2019

This pull request introduces 2 alerts when merging ca77241 into 9b34fc4 - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

@lgtm-com
Copy link

lgtm-com bot commented Dec 18, 2019

This pull request introduces 2 alerts when merging 06b1689 into 9b34fc4 - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

@lgtm-com
Copy link

lgtm-com bot commented Dec 18, 2019

This pull request introduces 2 alerts when merging 8af5395 into 9b34fc4 - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

@honzasterba
Copy link
Contributor Author

@wenbingl please review

wenbingl
wenbingl reviewed Dec 18, 2019
View reviewed changes
onnxmltools/convert/main.py
@@ -94,3 +94,9 @@ def convert_xgboost(*args, **kwargs):
from .xgboost.convert import convert
return convert(*args, **kwargs)

def convert_h2o(*args, **kwargs):
if not utils.h2o_installed():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given this method was added, please update the requirements.txt for onnxconverter-common version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was not yet released, do you know when the next release is planned?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'll take care of it.

tests/h2o/test_h2o_converters.py

@classmethod
def tearDownClass(cls):
h2o.cluster().shutdown()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is JRE needed to run these cases?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, is that an issue?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, a double confirm.

onnxmltools/convert/h2o/convert.py
name = str(uuid4().hex)

mojo_str = h2o.print_mojo(model_path, format="json")
mojo_model = json.loads(mojo_str)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All other converters are converting the model already loaded in memory, Can this new converter follow the same behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately this is impossible, since h2o core is in java our Python API is just a wrapper over a REST API exposed by the back-end. Furhter more the MOJO model format is meant for final deployment outside of h2o so there is little support for it in python.
the closes thing we could do is to accept the JSON representation of the model (generated by json.loads(h2o.print_mojo(path, "json")) but that would only make this less user-friendly IMO

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point is the model passed to converter are all in-memory. so it is important that we can keep the consistence among the converter APIs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand you concern. Would it be better if instead of a filesystem path the argument was the zip-file loaded into memory?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any type of the in-memory object is ok with me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

onnxmltools/convert/h2o/convert.py Outdated Show resolved Hide resolved
onnxmltools/convert/h2o/operator_converters/h2o.py Show resolved Hide resolved
@wenbingl
Copy link
Member

A dumb question, is it possible that the H2O Mojo model can be converted to xgboost model?

@honzasterba
Copy link
Contributor Author

Maybe for some cases that could be done, but definitely not for the general case.

@lgtm-com
Copy link

lgtm-com bot commented Dec 23, 2019

This pull request introduces 2 alerts when merging 06d7af2 into 37e51ab - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

@lgtm-com
Copy link

lgtm-com bot commented Jan 2, 2020

This pull request introduces 2 alerts when merging 7be8641 into 37e51ab - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

wenbingl
wenbingl approved these changes Jan 2, 2020
View reviewed changes
Copy link
Member

@wenbingl wenbingl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you fix the LGTM warning as well?

@lgtm-com
Copy link

lgtm-com bot commented Jan 2, 2020

This pull request introduces 2 alerts when merging 4a49efe into 37e51ab - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

@honzasterba
Copy link
Contributor Author

Fixed one of the LGTM warnings, the other is a common patter used thoughout the code-base I would much rather stick to the code-style of the project.
Rebased on top of latest master.

@lgtm-com
Copy link

lgtm-com bot commented Jan 3, 2020

This pull request introduces 2 alerts and fixes 1 when merging 1413a71 into 37e51ab - view on LGTM.com

new alerts:

  • 1 for Unused import
  • 1 for 'import *' may pollute namespace

fixed alerts:

  • 1 for 'import *' may pollute namespace

@lgtm-com
Copy link

lgtm-com bot commented Jan 3, 2020

This pull request introduces 1 alert and fixes 1 when merging ef356d1 into 37e51ab - view on LGTM.com

new alerts:

  • 1 for Unused import

fixed alerts:

  • 1 for 'import *' may pollute namespace

@honzasterba
Copy link
Contributor Author

@wenbingl should be ready to merge at your convenience

or perhaps a release of onnxconverter-common is still required?

@wenbingl wenbingl merged commit 362444a into onnx:master Jan 6, 2020
@honzasterba honzasterba deleted the h2o_gbm_mojo branch January 6, 2020 20:15
@exalate-issue-sync exalate-issue-sync bot mentioned this pull request May 12, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wenbingl wenbingl wenbingl approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@honzasterba @claassistantio @wenbingl