Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transformer class org.apache.spark.ml.feature.Mytransformer is not supported #77

Closed
GuoYL36 opened this issue Aug 24, 2019 · 4 comments
Closed

Comments

@GuoYL36
Copy link

GuoYL36 commented Aug 24, 2019

When I used the jpmml-spark library to generate a model in PMML format for pipeline model, this error appeared: Transformer class org. apache. spark. ml. feature. Mytransformer is not supported.
Among them, Mytransformer operator is my own extended transformer operator, because org.apache.spark.ml supports developers to extend transformer operator, but jpmml-spark library does not support developer-defined transformer operator. What method can jpmml-spark support developers to customize operator?

@vruusmann
Copy link
Member

Closing as exact duplicate of #72

@GuoYL36
Copy link
Author

GuoYL36 commented Aug 24, 2019

Closing as exact duplicate of #72

Thanks!

@GuoYL36
Copy link
Author

GuoYL36 commented Aug 25, 2019

Closing as exact duplicate of #72
I don't understand you said that register it with the JPMML-SparkML runtime by mentioning it in some META-INF/sparkml2pmml.properties file: https://github.com/jpmml/jpmml-sparkml/blob/master/src/main/resources/META-INF/sparkml2pmml.properties#L1-L28.
Do I only add "org.apache.spark.ml.feature.Mytransformer = org.jpmml.sparkml.feature.MytransformerConverter" in META-INF/sparkml2pmml.properties file, and then run my code?

@vruusmann
Copy link
Member

I don't understand you said that register it with the JPMML-SparkML runtime by mentioning it in some META-INF/sparkml2pmml.properties file:

You should package your custom transformer class as a standalone JAR file, while contains its own META-INF/sparkml2pmml.properties file.

However, for quick and dirty hacking, you could put your custom transformer class into the official JPMML-SparkML codebase, and modify its META-INF/sparkml2pmml.properties file.

Do I only add "org.apache.spark.ml.feature.Mytransformer = org.jpmml.sparkml.feature.MytransformerConverter"

The format is:

<Apache Spark transformer class name> = <JPMML-SparkML transformer converter class name>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants