Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #1256: Add JUnit support #1841

Merged
merged 28 commits into from Jul 9, 2020
Merged

Conversation

errikos
Copy link
Member

@errikos errikos commented Jul 9, 2020

Add JUnit support for Scala Native and its projects. See docs/user/testing.rst for usage instructions.

The feature was mostly ported from Scala.js, with some adaptations for Scala Native.

Notes

  • The existing Scala Native test runner (TestMain) does not support multiple testing frameworks. For that reason, we now spawn one TestMain process per testing framework. This can be addressed in a future PR.
  • There is a runtime bug that is triggered by the (String, Throwable) constructor of org.junit.AssumptionViolatedException. The bug is only present in JUnit self-tests. At runtime, the native test runner exits unexpectedly when the AssumptionViolation is tested. This is why the aforementioned constructor is not available at the moment. Some details:
    • The constructor is always made reachable, no matter if it is used or not. This suggests maybe some bug in the reachability analysis and/or code generation.
    • After some digging around, I found out that the one to blame is the Throwable argument. When the argument is changed to anything else (or removed entirely), then everything works as expected. That is, the constructor is only made reachable when used and the runtime error goes away.

Testing

  • The test-all command alias now also runs the JUnit self-tests.
  • There is an example JUnit test added in unit-tests/src/test/scala/junit/JUnitExampleTest.scala.

Usage

To enable JUnit support for a Scala Native project, add the following lines to your build.sbt file:

libraryDependencies += "org.scala-native" %%% "junit-runtime" % "0.4.0-SNAPSHOT"
addCompilerPlugin("org.scala-native" % "junit-plugin" % "0.4.0-SNAPSHOT" cross CrossVersion.full)

If you want to get more detailed output from the JUnit runtime, also include the following line:

testOptions += Tests.Argument(TestFrameworks.JUnit, "-a", "-s", "-v")

You may then use test or testOnly in the sbt shell to run all or specific tests, respectively.

@errikos errikos requested a review from sjrd July 9, 2020 09:34
errikos and others added 2 commits July 9, 2020 12:28
Co-authored-by: Lorenzo Gabriele <lorenzolespaul@gmail.com>
Copy link
Collaborator

@sjrd sjrd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good for the most part. I have a few comments. Most are minor. The most important comment is about using Future in the Bootstrapper API, even if we don't support async tests yet.

build.sbt Outdated Show resolved Hide resolved
build.sbt Outdated Show resolved Hide resolved
build.sbt Outdated Show resolved Hide resolved
build.sbt Outdated Show resolved Hide resolved
build.sbt Show resolved Hide resolved
unit-tests/src/test/scala/junit/JUnitExampleTest.scala Outdated Show resolved Hide resolved
project/build.sbt Outdated Show resolved Hide resolved
@errikos errikos requested a review from sjrd July 9, 2020 13:54
Copy link
Collaborator

@sjrd sjrd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Sorry I found one more problem in the async package object and how it's used.

@ekrich
Copy link
Member

ekrich commented Jul 9, 2020

I saw a comment about normalized names but I think this takes care of it unless this setting is not used. https://github.com/scala-native/scala-native/blob/master/build.sbt#L23

@errikos
Copy link
Member Author

errikos commented Jul 9, 2020

I saw a comment about normalized names but I think this takes care of it unless this setting is not used. https://github.com/scala-native/scala-native/blob/master/build.sbt#L23

Yep, basically that was the part that introduced the problem. It converts the camelCase name to kebab-case.

So, jUnitPlugin became j-unit-plugin, which is ugly. I renamed to junitPlugin so that it becomes junit-plugin.

@errikos errikos requested a review from sjrd July 9, 2020 14:40
@sjrd sjrd changed the title Add JUnit support Fix #1256: Add JUnit support Jul 9, 2020
@ekrich
Copy link
Member

ekrich commented Jul 9, 2020

@errikos Cool, I saw the code after it changed to junit... so I was behind the happenings.

@errikos errikos merged commit 2050173 into scala-native:master Jul 9, 2020
ekrich pushed a commit to ekrich/scala-native that referenced this pull request May 21, 2021
Add JUnit support for Scala Native and its projects. See `docs/user/testing.rst` for usage instructions.

The feature was mostly ported from Scala.js, with some adaptations for Scala Native.

### Notes
- The existing Scala Native test runner (`TestMain`) does not support multiple testing frameworks. For that reason, we now spawn one `TestMain` process per testing framework. This can be addressed in a future PR.
- There is a runtime bug that is triggered by the `(String, Throwable)` constructor of `org.junit.AssumptionViolatedException`. The bug is only present in JUnit self-tests. At runtime, the native test runner exits unexpectedly when the AssumptionViolation is tested. This is why the aforementioned constructor is not available at the moment. Some details:
  - The constructor is always made reachable, no matter if it is used or not. This suggests maybe some bug in the reachability analysis and/or code generation.
  - After some digging around, I found out that the one to blame is the `Throwable` argument. When the argument is changed to anything else (or removed entirely), then everything works as expected. That is, the constructor is only made reachable when used and the runtime error goes away.

### Testing
- The `test-all` command alias now also runs the JUnit self-tests.
- There is an example JUnit test added in `unit-tests/src/test/scala/junit/JUnitExampleTest.scala`.

### Usage
To enable JUnit support for a Scala Native project, add the following lines to your `build.sbt` file:
```scala
libraryDependencies += "org.scala-native" %%% "junit-runtime" % "0.4.0-SNAPSHOT"
addCompilerPlugin("org.scala-native" % "junit-plugin" % "0.4.0-SNAPSHOT" cross CrossVersion.full)
```
If you want to get more detailed output from the JUnit runtime, also include the following line:
```scala
testOptions += Tests.Argument(TestFrameworks.JUnit, "-a", "-s", "-v")
```
You may then use `test` or `testOnly` in the sbt shell to run all or specific tests, respectively.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants