-
Notifications
You must be signed in to change notification settings - Fork 363
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SW-806] Fix Importing/Exporting of pipelines, pipelineModels and MOJ…
- Loading branch information
Showing
17 changed files
with
418 additions
and
103 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,4 +4,5 @@ Deployment | |
.. toctree:: | ||
:maxdepth: 2 | ||
|
||
backends.rst | ||
backends.rst | ||
load_mojo.rst |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
Importing H2O Mojo | ||
------------------ | ||
|
||
H2O Mojo can be imported to Sparkling Water from all data sources Spark supports such as local file, S3 or HDFS and the | ||
semantics of the import is the same as in the Spark API. | ||
|
||
|
||
If HDFS is not available for Spark, then call, in Scala: | ||
|
||
.. code:: scala | ||
import org.apache.spark.ml.h2o.models._ | ||
val model = H2OMOJOModel.createFromMojo("prostate.mojo") | ||
or in Python: | ||
|
||
.. code:: python | ||
from pysparkling.ml import * | ||
model = H2OMOJOModel.create_from_mojo("prostate.mojo") | ||
attempts to load the mojo file with the specified name from the current working directory. | ||
You can also specify the full path such as, in Scala: | ||
|
||
.. code:: scala | ||
import org.apache.spark.ml.h2o.models._ | ||
val model = H2OMOJOModel.createFromMojo("/Users/peter/prostate.mojo") | ||
or in Python: | ||
|
||
.. code:: python | ||
from pysparkling.ml import * | ||
model = H2OMOJOModel.create_from_mojo("/Users/peter/prostate.mojo") | ||
In the case Spark is running on Hadoop and HDFS is available, then call, in Scala: | ||
|
||
.. code:: scala | ||
import org.apache.spark.ml.h2o.models._ | ||
val model = H2OMOJOModel.createFromMojo("prostate.mojo") | ||
or in Python: | ||
|
||
.. code:: python | ||
from pysparkling.ml import * | ||
model = H2OMOJOModel.create_from_mojo("prostate.mojo") | ||
attempts to load the mojo from the HDFS home directory of the current user. | ||
You can also specify the absolute path in this case as, in Scala: | ||
|
||
.. code:: scala | ||
import org.apache.spark.ml.h2o.models._ | ||
val model = H2OMOJOModel.createFromMojo("/user/peter/prostate.mojo") | ||
or in Python: | ||
|
||
.. code:: python | ||
from pysparkling.ml import * | ||
model = H2OMOJOModel.create_from_mojo("/user/peter/prostate.mojo") | ||
Both calls load the mojo file from the following location ``hdfs://{server}:{port}/user/peter/prostate.mojo``, where ``{server}`` and ``{port}`` is automatically filled in by Spark. | ||
|
||
|
||
You can also manually specify the type of data source you need to use, in that case, you need to provide the schema, in Scala: | ||
|
||
.. code:: scala | ||
import org.apache.spark.ml.h2o.models._ | ||
// HDFS | ||
val modelHDFS = H2OMOJOModel.createFromMojo("hdfs:///user/peter/prostate.mojo") | ||
// Local file | ||
val modelLocal = H2OMOJOModel.createFromMojo("file:///Users/peter/prostate.mojo") | ||
or in Python: | ||
|
||
.. code:: python | ||
from pysparkling.ml import * | ||
# HDFS | ||
val model_hdfs = H2OMOJOModel.create_from_mojo("hdfs:///user/peter/prostate.mojo") | ||
# Local file | ||
val model_local = H2OMOJOModel.create_from_mojo("file:///Users/peter/prostate.mojo") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.