NoSuchMethodError BoosterGetEvalNames with LightGBM

[peay]

Hello,

I am trying to upgrade to v0.17, and I keep getting errors such as below when training.

I've also tried building master but I am getting the same error. I see that the last tagged released of LightGBM doesn't seem to have that SWIG interface (https://github.com/microsoft/LightGBM/blob/v2.2.3/swig/lightgbmlib.i).

Any thoughts?

cc @imatiach-msft

java.lang.NoSuchMethodError: com.microsoft.ml.lightgbm.lightgbmlib.LGBM_BoosterGetEvalNamesSWIG(Lcom/microsoft/ml/lightgbm/SWIGTYPE_p_void;I)Lcom/microsoft/ml/lightgbm/SWIGTYPE_p_p_char;
	at com.microsoft.ml.spark.TrainUtils$.getEvalNames(TrainUtils.scala:129)
	at com.microsoft.ml.spark.TrainUtils$.trainCore(TrainUtils.scala:138)
	at com.microsoft.ml.spark.TrainUtils$.translate(TrainUtils.scala:231)
	at com.microsoft.ml.spark.TrainUtils$.trainLightGBM(TrainUtils.scala:352)
	at com.microsoft.ml.spark.LightGBMBase$$anonfun$5.apply(LightGBMBase.scala:68)
	at com.microsoft.ml.spark.LightGBMBase$$anonfun$5.apply(LightGBMBase.scala:68)
	at org.apache.spark.sql.execution.MapPartitionsExec$$anonfun$5.apply(objects.scala:188)
	at org.apache.spark.sql.execution.MapPartitionsExec$$anonfun$5.apply(objects.scala:185)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:121)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

[imatiach-msft]

hi @peay , based on the error message you are using an older version of lightgbm jar with newer mmlspark jar. I think it is related to this fix:
microsoft/LightGBM#2118
For mmlspark v0.17 you need to use the corresponding lightgbm 2.2.300:
https://mvnrepository.com/artifact/Azure/mmlspark/0.17
Specifically this lightgbm jar:
https://mvnrepository.com/artifact/com.microsoft.ml.lightgbm/lightgbmlib/2.2.300
Hope this helps!

[peay]

Does 2.2.300 correspond to v2.2.3 in LightGBM's repository?

[imatiach-msft]

hi @peay yes, 2.2.300 corresponds to v2.2.3 from LightGBM repository