[SPARK-38346][MLLIB] Add cache in MLlib BinaryClassificationMetrics

[waruto210]

What changes were proposed in this pull request?

Cache two RDDs in BinaryClassificationMetrics.scala.

Why are the changes needed?

Two RDDs in BinaryClassificationMetrics.scala were used by diffrenet jobs but not cached when we run some example code as follow(view full description on jira):

  val pipeline: Pipeline = new Pipeline().setStages(Array(assembler, classifier))

  val Array(tr, te) = originalData.randomSplit(Array(0.7, .03), 666)
  val model = pipeline.fit(tr)
  val modelDF = model.transform(te)
  val evaluator = new BinaryClassificationEvaluator().setLabelCol(labelCol).setRawPredictionCol("prediction")
  println(evaluator.evaluate(modelDF))

Does this PR introduce any user-facing change?

NO

How was this patch tested?

origin tests

[waruto210]

cc @zhengruifeng @imatiach-msft

[AmplabJenkins]

Can one of the admins verify this patch?

[srowen]

I'm not sure this is worth it. For one, this cached RDD leaks, and is not unpersisted. It doesn't check if it's already cached. And I don't think it probably helps much as you pay the caching overhead just to avoid 1 recomputation for count(), which may be not a big deal perf-wise.

[zhengruifeng]

you can cache the returned rdd on your own purpose.
We should not cache it here

[zhengruifeng]

this RDD counts seems used twice at most in this method. I think it not worthwhile to cache it.

[waruto210]

The user-level cache requires developers to understand the program characteristics themselves, and the focus here is on the cache of libraries-level (MLlib), like SPARK-16697, SPARK-16880 and SPARK-18356 in MLLib.
@zhengruifeng

[zhengruifeng]

@waruto210 normally, ml algorithm will use the cached dataset maxIters (by default 20~100) times, so we must cache it. In this case, I think it not worthwhile.