Xgboost 4j and Predictor implementations in separate sbt subprojects #1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a diff over combust#645
Main observation
I think the original solution at combust#645 is less disrupting to most people who don’t want to know about Predictor. Everyone will by default keep using xgboost4j. The solution in this PR brings a non-deterministic deserialization behaviour if users import both MLeap subprojects
xgboost-runtime
andxgboost-predictor
.Why? The “xgboost.classifier” key in the BundleRegistry contains either one of the Ops. I think this depends on how maven merges the reference.conf files via the “AppendTransformer”. This might be non-deterministic, similarly to how maven handles dependency conflicts? Secondly it also randomly depends on the scala hash key when filling up the BundleRegistry hash table.
Other observations
mleap-xgboost-predictor
needs a sbt dependency onmleap-xgboost-runtime
because it's using the testing helpers code from there. I am not sure sharing the testing helpers justified creating a third common project.store()
is not implemented) I added static model files in resources that were serialized via theruntime
project.