New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NETBEANS-6] Adding Support for JUnit 5 #892

Merged
merged 4 commits into from Sep 25, 2018

Conversation

Projects
None yet
6 participants
@mcdonnell-john
Copy link
Contributor

mcdonnell-john commented Sep 20, 2018

Main Features include:

  • Added JUnit 5 Library - Using latest 5.3.1 release.
  • Added JUnit 5 Templates
  • New Test classes default to JUnit 5 if JDK >= 8 & JUnit 4 not already used.
  • Test Classes when JUnit 5 is found on the class path(Maven) use JUnit 5.
  • Run/Debug single test method support included. Checks when Using JUnit 5 of the method has an annotation that inherits from @testable
  • Updated JUnit IDE hints for JUnit 5 - i.e. Assertions as opposed to Assert and ordering of arguements.
  • Informational message to users when creating a Test Suite with Junit 5 - Currently not supported with Jupiter, message refers users to switch to the Vintage Engine.

@project-bot project-bot bot added this to new/unassigned/reopened prs in Apache Cordova: project-bot automation testing Sep 20, 2018

@janpio janpio removed this from new/unassigned/reopened prs in Apache Cordova: project-bot automation testing Sep 20, 2018

@sormuras

This comment has been minimized.

Copy link

sormuras commented Sep 20, 2018

Nice to see JUnit 5 support in NB, soon! ;-)

Using Testable as the signal to show users "testable" elements is good.

Just scrolled through the PR and didn't find a single "junit-platform-launcher" snippet. How does NetBeans execute tests? Via the command line? Or Maven/Surefire?

@mcdonnell-john

This comment has been minimized.

Copy link
Contributor Author

mcdonnell-john commented Sep 20, 2018

Hi @sormuras,

So I toy-ed around for a while looking to see if I could implement an EngineDiscoveryRequest but couldn't get it to work, and then went back to the @testable check based on its JavaDocs and left it at that for now.

When using Maven based projects, Netbeans uses Maven/Surefire - I added a quick check for Surefire >2.22.0 for supporting individual test methods.
I don't use Ant for any project, so would need help from someone who does to see what's needed to support changes for Ant.

@sormuras

This comment has been minimized.

Copy link

sormuras commented Sep 20, 2018

I see. So, at the moment, users see something like "Run test..." in the IDE/editor and when click, a Maven run tailored to the action is executed. Correct? If yes, it's good starting point.

You could use the junit-platform-console launcher, like I do here: https://github.com/sormuras/junit-platform-maven-plugin/tree/master/src/main/java/de/sormuras/junit/platform/maven/plugin -- then you are less dependent on how Surefire finds and executes tests.

Regarding Ant and its configuration in NB ... methinks, it would need to include junit-platform-console and use the provided launcher.

Or...

You use the Launcher API like described here: https://junit.org/junit5/docs/current/user-guide/#launcher-api

How can I test-drive this PR? Checkout your "apache/incubator-netbeans" clone, switch to branch feature/JUnit5 and "Press Play on Tape"?

@sormuras

This comment has been minimized.

Copy link

sormuras commented Sep 20, 2018

Answering my last question on my own: http://netbeans.apache.org/download/index.html#source

@mcdonnell-john

This comment has been minimized.

Copy link
Contributor Author

mcdonnell-john commented Sep 20, 2018

Thanks for the comments.

NetBeans is closely tied to maven, and for JUnit surefire - I tried to keep as close to the existing solution that exists for JUnit 4 as I could to keep the changes as simple as possible.

I think the biggest issue with this is the fact that surefire doesn't support the @DisplayName annotation, since the reporting in NetBeans also uses the output from Surefire from what I have seen. - So that's a gap we have with the maven support for JUnit 5 in NetBeans

Yes,
"How can I test-drive this PR? Checkout your "apache/incubator-netbeans" clone, switch to branch feature/JUnit5 and"... - run "ant tryme"

@@ -0,0 +1,24 @@
<#-- FreeMarker template (see http://freemarker.org/) -->

This comment has been minimized.

@junichi11

junichi11 Sep 20, 2018

Member

Please add the template file to the licenseinfo.xml if don't add the license header to template files. e.g. see

<fileset>
<file>src/org/netbeans/modules/xml/resources/templates/emptyDTD.dtd.template</file>
<file>src/org/netbeans/modules/xml/resources/templates/emptyXML.xml.template</file>
<file>src/org/netbeans/modules/xml/resources/templates/emptyXmlSchema.xsd.template</file>
<file>src/org/netbeans/modules/xml/resources/templates/xhtml.xml.template</file>
<file>src/org/netbeans/modules/xml/resources/templates/xml_entity.ent.template</file>
<license ref="Apache-2.0-ASF" />
<comment type="TEMPLATE_MINIMAL_IP"/>
</fileset>

This comment has been minimized.

@mcdonnell-john

mcdonnell-john Sep 20, 2018

Author Contributor

Hi @junichi11 Thanks for spotting this.

Just a note on this point though, the other licenses aren't covered in a licenseinfo.xml file, they are infact covered via the rat exclusions file:

java/junit.ui/src/org/netbeans/modules/junit/ui/resources/*.template
. So my new template file should be covered by this, as opposed to a licenseinfo.xml entry.

I added a new logo to differentiate JUnit 5 and the old JUnit version so will definitely add a reference to that as that was missed and commit that shortly.

This comment has been minimized.

@junichi11

junichi11 Sep 20, 2018

Member

I see. I suppose that we should move them to the licenseinfo.xml if possible because we have to verify rat-exclusions and licenseinfo.xml :)

@@ -0,0 +1,67 @@
<#-- FreeMarker template (see http://freemarker.org/) -->

This comment has been minimized.

@junichi11

junichi11 Sep 20, 2018

Member

Please add the template file to the licenseinfo.xml.

This comment has been minimized.

@mcdonnell-john

mcdonnell-john Sep 20, 2018

Author Contributor

same comment as above.

@sormuras

This comment has been minimized.

Copy link

sormuras commented Sep 21, 2018

Regarding @DisplayName and Surefire: working on it already https://issues.apache.org/jira/projects/SUREFIRE/issues/SUREFIRE-1546

@sormuras

This comment has been minimized.

Copy link

sormuras commented Sep 21, 2018

FYI: there is an issue with Surefire keeps using version 5.2 of JUnit (JUnit Platform 1.2.0) although you override the version in a pom.xml file. See https://issues.apache.org/jira/projects/SUREFIRE/issues/SUREFIRE-1564 for details.

@mcdonnell-john

This comment has been minimized.

Copy link
Contributor Author

mcdonnell-john commented Sep 21, 2018

Regarding @DisplayName and Surefire: working on it already https://issues.apache.org/jira/projects/SUREFIRE/issues/SUREFIRE-1546

Cool, When this comes in, then we may have to look at the Test Results window in NetBeans to show the names explicitly.

@sormuras

This comment has been minimized.

Copy link

sormuras commented Sep 21, 2018

If stars align, it should work out-of-the-box. Just update to the yet to be released Surefire version and the ReportEntry object will report the display names instead of the "legacy reporting names", i.e. the name of the test methods and classes.

@geertjanw

This comment has been minimized.

Copy link
Member

geertjanw commented Sep 21, 2018

Great, if I can do something to try this out, would be great -- maybe you could update this page with the list of features provided for JUnit 5 here: https://cwiki.apache.org/confluence/display/NETBEANS/Miscellaneous

@geertjanw

This comment has been minimized.

Copy link
Member

geertjanw commented Sep 22, 2018

Any reason not to merge this, what still needs to be done, etc?

@mcdonnell-john

This comment has been minimized.

Copy link
Contributor Author

mcdonnell-john commented Sep 22, 2018

Any reason not to merge this, what still needs to be done, etc?

Hi, I don't think there is.

@geertjanw

This comment has been minimized.

Copy link
Member

geertjanw commented Sep 22, 2018

Can you add screenshots to illustrate these features to that Miscellaneous page?

@mcdonnell-john

This comment has been minimized.

Copy link
Contributor Author

mcdonnell-john commented Sep 22, 2018

Will do - should be able to get to that on Sunday...

@Saljack

This comment has been minimized.

Copy link

Saljack commented Sep 24, 2018

Hi, why JUnit BOM is not used in pom.xml for versions and dependencies? It looks better and more clear in my oppinion?

  </dependencies>
    <!--JUNIT 5-->
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-params</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-api</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.junit</groupId>
        <artifactId>junit-bom</artifactId>
        <version>5.3.1</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>
@mcdonnell-john

This comment has been minimized.

Copy link
Contributor Author

mcdonnell-john commented Sep 24, 2018

Hi, why JUnit BOM is not used in pom.xml for versions and dependencies? It looks better and more clear in my oppinion?

Because the current NetBeans implementation of adding libraries to an existing pom doesn't support that, as far as I could see. I tried to limit my changes to re-use a lot of what already worked with JUnit 4.

It's also a matter of taste, and to me its not a big deal.

@geertjanw

This comment has been minimized.

Copy link
Member

geertjanw commented Sep 24, 2018

Great, and thanks for the screenshots (https://cwiki.apache.org/confluence/display/NETBEANS/Miscellaneous). What is left to be done, for this release, and for subsequent releases?

@mcdonnell-john

This comment has been minimized.

Copy link
Contributor Author

mcdonnell-john commented Sep 24, 2018

HI @geertjanw I cant think of anything else, I mostly focused on Maven support and maybe some ant stuff was handle there, but there probably some work there if someone who uses ant wants to take on.

I think for this release we should be okay, and we'll need to update the Test Specs for JUnit testing for NetCat to help test this a little more.

Moving forward, for subsequent releases:
What ever is missing for Ant support,
Better test suite generation - when Jupiter supports it

Apart from that I don't see anything as Surefire should be fixing their display name annotation handling issue, if that doesn't need any additional changes then great, if not we can include that in a future release.

@geertjanw

This comment has been minimized.

Copy link
Member

geertjanw commented Sep 24, 2018

Great. Let's merge if no one objects 24 hours from now.

@mcdonnell-john mcdonnell-john merged commit 073c42c into apache:master Sep 25, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@Tibor17

This comment has been minimized.

Copy link

Tibor17 commented Sep 26, 2018

We want to cut a new Surefire Version 2.22.1 for JUnit5 5.2.0+.
This will support 5.3.0 too of course and Java 11.
We have decided that version 3.0.0-M1 will adopt Display Name.
We won't to postpone the release 2.22.1 too much.

@geertjanw

This comment has been minimized.

Copy link
Member

geertjanw commented Jan 27, 2019

I took a look at JUnit 5 and Ant in NetBeans, some thoughts here:

https://blogs.apache.org/netbeans/entry/junit-5-apache-ant-and

@sormuras

This comment has been minimized.

Copy link

sormuras commented Jan 27, 2019

Probably the best solution for Ant users in Apache NetBeans would be to be able to continue to use JUnit 4, rather than JUnit 5, since the latter is a total nightmare to set up in the context of Ant, as described above. No need to endure nightmares...

Have you tried the all-in-one-fat-jar solution with Ant? Download junit-platform-console-standalone and that's all.

Copied from the User Guide

junit-platform-console-standalone
An executable JAR with all dependencies included is provided at Maven Central under the
junit-platform-console-standalone directory. See Console Launcher for details.

@geertjanw

This comment has been minimized.

Copy link
Member

geertjanw commented Jan 27, 2019

Ah, OK, that's great.

Could we bundle that with Apache NetBeans, in the same way that we're currently bundling jupiter-api, jupiter-params, and jupiter-engine? And could we remove those three and replace it with the fat JAR? Is it licensed in the same way?

@sormuras

This comment has been minimized.

Copy link

sormuras commented Jan 27, 2019

Could we bundle that with Apache NetBeans,...

Perhaps. It's a "all-in" solution. Could Maven projects still refer to the modules and use them while Ant projects get the junit-platform-console-standalone support?

Is it licensed in the same way?

It states:

Open Source Licenses
====================

This product may include a number of subcomponents with separate
copyright notices and license terms. Your use of the source code for
these subcomponents is subject to the terms and conditions of the
subcomponent's license, as noted in the LICENSE-<subcomponent>.md
files.

It contains the following licences:

  • Eclipse Public License - v 1.0 (junit 3 + 4)
  • Eclipse Public License - v 2.0 (junit 5)
  • BSD License (hamcrest)
  • Apache License 2.0 (picocli, univocity-parsers)
@sormuras

This comment has been minimized.

Copy link

sormuras commented Jan 27, 2019

@geertjanw Here is an example of a "patched" Ant installation that allows Ant projects to be configured hassle-free:

https://github.com/junit-team/junit5-samples/tree/master/junit5-jupiter-starter-ant

See https://github.com/junit-team/junit5-samples/blob/master/junit5-jupiter-starter-ant/build.sh#L6-L20 for how to "install" the junit-platform-console-standalone.jar into Ant (as a lib) and here is how you may use the built-in task and the console launcher side-by-side: https://github.com/junit-team/junit5-samples/blob/master/junit5-jupiter-starter-ant/build.xml#L35-L61

@geertjanw

This comment has been minimized.

Copy link
Member

geertjanw commented Jan 27, 2019

Yes, I've been studying this one a lot, it's great and very useful:

https://github.com/junit-team/junit5-samples/blob/master/junit5-jupiter-starter-ant/build.xml

And the org.junit.platform.console.ConsoleLauncher works fine, produces results, etc. The only question is how to hook it into the Results Window in NetBeans, which is done very well for Surefire, which is why JUnit 5 with Maven in NetBeans works so smoothly in this regard.

@sormuras

This comment has been minimized.

Copy link

sormuras commented Jan 27, 2019

[...] which is done very well for Surefire [...]

By a listener that Surefire calls when tests are executed?

Nevertheless and as you have written in your blog, NetBeans should use the JUnit Platform Launcher API to start test runs. That's the sole purpose of it -- no need to depend on a plugin of a specific build tool that tries to "find and execute" tests in a "custom" way. Copied from the beginning of the User Guide:

The JUnit Platform serves as a foundation for launching testing frameworks on the JVM. It also defines the TestEngine API for developing a testing framework that runs on the platform. Furthermore, the platform provides a Console Launcher to launch the platform from the command line and build plugins for Gradle and Maven as well as a JUnit 4 based Runner for running any TestEngine on the platform.

https://junit.org/junit5/docs/current/user-guide/index.html#overview-what-is-junit-5

And every new TestEngine, wrapping an existing test framework (mainrunner, TestNG, Spock, JQWIK, ... https://github.com/junit-team/junit5/wiki/Third-party-Extensions#junit-platform-test-engines) or one that is invented tomorrow (JUnit 6 "aka Saturn") will be integrated in your build tool right-away. Zero effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment