jpmml-spark fails

[simhan]

jpmml-spark fails from github pull

Downloaded the latest . update the pom to spark 1.6.0

mvn -V = 3.3.1
mvn clean install

[INFO] 3 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] JPMML-Spark ........................................ SUCCESS [ 0.592 s]
[INFO] pmml-spark ......................................... FAILURE [ 2.775 s]
[INFO] pmml-spark-example ................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.492 s
[INFO] Finished at: 2016-06-23T17:55:38-07:00
[INFO] Final Memory: 39M/316M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile) on project pmml-spark: Compilation failure: Compilation failure:
[ERROR] /PMML-SPARK/jpmml-spark-master/pmml-spark/src/main/java/org/jpmml/spark/PMMLPredictionModel.java:[31,28] cannot find symbol
[ERROR] symbol: class DataFrame
[ERROR] location: package org.apache.spark.sql
[ERROR] /PMML-SPARK/jpmml-spark-master/pmml-spark/src/main/java/org/jpmml/spark/PMMLPredictionModel.java:[88,46] cannot find symbol
[ERROR] symbol: class DataFrame
[ERROR] location: class org.jpmml.spark.PMMLPredictionModel
[ERROR] /PMML-SPARK/jpmml-spark-master/pmml-spark/src/main/java/org/jpmml/spark/PMMLPredictionModel.java:[88,16] cannot find symbol
[ERROR] symbol: class DataFrame
[ERROR] location: class org.jpmml.spark.PMMLPredictionModel
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :pmml-spark

[vruusmann]

The class org.apache.spark.sql.DataFrame has been moved away from the org.apache.spark:spark-sql_2.10 library. So, you need to find out its new location, and add that library to your project.

I'm really surprised that Apache Spark decided to break API between 1.X and 2.X versions like that. They are very determined in maintaining backwards compatibility in all other areas.

[Diliprajgor]

any idea on how to resolve this problem?

[Diliprajgor]

Hi Simhan,
the issue is resolved after changing the pom.xml. changed the [1.5.0,) to 1.6.0 at all the places.