lambda encoding breaks JUnit integration when for lambda with 0 parameters (junit.framework.AssertionFailedError: Test method isn't public)

[unkarjedy]

Compiler version

Scala 3.3.3
JUnit 4.13.2

ThisBuild / version := "0.1.0-SNAPSHOT"

ThisBuild / scalaVersion := "3.3.3"

lazy val root = (project in file("."))
  .settings(
    name := "untitled82",
    libraryDependencies ++= Seq(
      "junit" % "junit" % "4.13.2" % Test,
      "com.github.sbt" % "junit-interface" % "0.13.3" % Test
    ),
    testOptions += Tests.Argument(TestFrameworks.JUnit)
  )

Minimized code

I had a module in Scala 2 and some junit tests there.
After I updated the module to Scala 3, tests became broken due to the new Scala 3 lambda encoding scheme.
When the test class contains a lambda with 0 parameters (e.g. for Runnable) JUnit reports test failure

Use this code:

import junit.framework.TestCase
import junit.framework.TestCase.assertEquals

class MyTest1 extends TestCase {
  def testOk(): Unit = {
    assertEquals(1, 1)
  }

  def testOk_WithLambda_Function1(): Unit = {
    (1 to 3).foreach { x =>
      assertEquals(1, 1)
    }
  }

  def testBad_WithLambda_Function0(): Unit = {
    runPerformanceTest { () =>
      assertEquals(1, 1)
    }
  }

  private def runPerformanceTest(runnable: Runnable): Unit = runnable.run()
}

Run sbt test
Observe an error:

[error] Test junit.framework.TestSuite$1.warning failed: junit.framework.AssertionFailedError: Test method isn't public: testBad_WithLambda_Function0$$anonfun$1(MyTest1), took 0.001 sec
[error]     at junit.framework.Assert.fail(Assert.java:57)
[error]     at junit.framework.TestCase.fail(TestCase.java:223)
[error]     at junit.framework.TestSuite$1.runTest(TestSuite.java:96)
[error]     at junit.framework.TestCase.runBare(TestCase.java:142)
[error]     at junit.framework.TestResult$1.protect(TestResult.java:122)
[error]     at junit.framework.TestResult.runProtected(TestResult.java:142)
[error]     at junit.framework.TestResult.run(TestResult.java:125)
[error]     at junit.framework.TestCase.run(TestCase.java:130)
[error]     at junit.framework.TestSuite.runTest(TestSuite.java:241)
[error]     at junit.framework.TestSuite.run(TestSuite.java:236)
[error]     ...

This is because in Scala 3, the last lambda is encoded like this:

  private final static synthetic testBad_WithLambda_Function0$$anonfun$1()V

JUnit uses test prefix to detect test candidates.

I understand that it's not entirely Scala 3 fault, and most likely it should be fixed on the JUnit side.
It just was never an issue with Java/Scala2, so I decided to register the issue here at least to track it.

---

Note, in Scala 2, the lambda was encoded as:

public final static synthetic $anonfun$testBad_WithLambda_Function0$1()V

(it's public and it uses $anonfun$ prefix)

In Java, the similar lambda would be encoded as:

private static synthetic lambda$testBad_WithLambda_Function0$0()V

(uses lambda$ prefix)

Interestingly, Kotlin has the same issue (1.9.23), which is another argument that most likely it should be fixed in JUnit.

import junit.framework.TestCase

class MyTestKotlin : TestCase() {
    fun testOk() { assertEquals(1, 1) }

    fun testBad_WithLambda_Function0() {
        runPerformanceTest {  assertEquals(1, 1)  }
    }

    private fun runPerformanceTest(runnable: Runnable) = runnable.run()
}

[WojciechMazur]

I think there are little changes to fix it on Scala side as it can severely impact binary compatibility. At the same JUnit 3 is already a legacy library, it's 18 years since JUnit 4 was released and 8 years since JUnit 5 first release. Both of these offer much more intuitive integration by usage of @Test annotation for tested methods. Most of the migrations from Junit 3 to 4 can be made using regex search and replace.
Edit. I missed the fact you're using JUnit 4.x.y version. Seems like JUnit 4 can use JUnit 3 naming convention to search for test methods using test prefix and a new approach using@Test annotation.

[sjrd]

Indeed, let's not cater to JUnit 3. That wouldn't make any sense.

[som-snytt]

Surprising that synthetic is not excluded. JUnit 4 does not advertise a minimum JDK. The latest jar is built on JDK 6 for JDK 5 (class file 49).