[SW-449] Support for Sparse Vectors and Upgrade to H2O 3.14.0.2 #334
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jakubhava
force-pushed
the
jh/jira/sw-449
branch
from
4391a0b
to
cf0c508
Compare
|
Still some tests are failing on external cluster. I will fix them tonight or tomorrow at last, but getting to the final state. Benchmarks seems to be in favour of this change |
jakubhava
force-pushed
the
jh/jira/sw-449
branch
2 times, most recently
from
440547c
to
2f40a54
Compare
|
Ready for review, this is final state |
|
@mmalohlava |
jakubhava
force-pushed
the
jh/jira/sw-449
branch
from
cff47f6
to
d8b872a
Compare
jakubhava
force-pushed
the
jh/jira/sw-449
branch
from
d8b872a
to
eb00021
Compare
jakubhava
changed the title
mmalohlava
reviewed
Aug 14, 2017
| ExternalFrameUtils.EXPECTED_STRING | ||
| } else if (clazz == classOf[java.sql.Timestamp]) { | ||
| ExternalFrameUtils.EXPECTED_TIMESTAMP | ||
| } else if (clazz == classOf[org.apache.spark.mllib.linalg.Vector]) { |
There was a problem hiding this comment.
For future: what about about org.apache.spark.ml.linalg.Vector ?
There was a problem hiding this comment.
Oki, found it, you are doing Vectors.toML(..) coversion before.
|
Right now blocked by: |
mmalohlava
force-pushed
the
jh/jira/sw-449
branch
from
a536d95
to
484b135
Compare
Upgrade of H2O to 3.14.0.2
- More tests for RDD[Vector] - Missing header - Add automl into assembly - More tests. - Align test data to given lenght - More testing and testing and testing - Enable assertions for H2O launched for external model cluster tests
mmalohlava
force-pushed
the
jh/jira/sw-449
branch
from
8b5d1f5
to
0a5fbef
Compare
mmalohlava
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SparkDataFrameConverter refactored. Now we first create flat dataframe in spark to simplify handling on sw side.
The refactoring was needed in order to separate handling of vectors.
An improvement of this solution would be to drop the flag whether the vector is sparse or dense and make it part of the expected types
This change requires h2oai/h2o-3#1348