Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-23120][PYSPARK][ML] Add basic PMML export support to PySpark #21172

Conversation

holdenk
Copy link
Contributor

@holdenk holdenk commented Apr 26, 2018

What changes were proposed in this pull request?

Adds basic PMML export support for Spark ML stages to PySpark as was previously done in Scala. Includes LinearRegressionModel as the first stage to implement.

How was this patch tested?

Doctest, the main testing work for this is on the Scala side. (TODO holden add the unittest once I finish locally).

…nearRegressionModel as the first implementation
@SparkQA
Copy link

SparkQA commented Apr 26, 2018

Test build #89899 has finished for PR 21172 at commit 8cbef82.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • class LinearRegressionModel(JavaModel, JavaPredictionModel, GeneralJavaMLWritable, JavaMLReadable):
  • class GeneralMLWriter(MLWriter):
  • class GeneralJavaMLWriter(JavaMLWriter):
  • class GeneralJavaMLWritable(JavaMLWritable):

@SparkQA
Copy link

SparkQA commented Apr 26, 2018

Test build #89900 has finished for PR 21172 at commit 9b14a1c.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Fix sc ref
@holdenk holdenk force-pushed the SPARK-23120-add-pmml-export-support-to-pyspark branch from 9b14a1c to 64d6db8 Compare April 26, 2018 19:55
@SparkQA
Copy link

SparkQA commented Apr 26, 2018

Test build #89902 has finished for PR 21172 at commit 64d6db8.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 4, 2018

Test build #90234 has finished for PR 21172 at commit af21aa7.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 5, 2018

Test build #90236 has finished for PR 21172 at commit 9d7d684.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 11, 2018

Test build #90515 has finished for PR 21172 at commit 67b59a7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@holdenk holdenk changed the title [SPARK-23120][PYSPARK][ML][WIP] Add basic PMML export support to PySpark [SPARK-23120][PYSPARK][ML] Add basic PMML export support to PySpark Jun 28, 2018
@holdenk
Copy link
Contributor Author

holdenk commented Jun 28, 2018

Merged to master

@asfgit asfgit closed this in a95a4af Jun 28, 2018
@vruusmann
Copy link
Contributor

Here's a pointer to another PySpark-to-PMML conversion tool: https://github.com/jpmml/pyspark2pmml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants