New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Surefire Plugin should evaluate @DisplayName #990

Closed
mab opened this Issue Jul 27, 2017 · 13 comments

Comments

Projects
None yet
8 participants
@mab

mab commented Jul 27, 2017

Overview

Feature request.

Currently the surefire plugin does not evaluate @DisplayName in the test reports. It would be great if Surefire would use @DisplayName where present.

Maven surfire output from the junit5-maven-consumer project:

Running com.example.project.FirstTest
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.115 sec <<< FAILURE! - in com.example.project.FirstTest
myFirstTest(TestInfo)  Time elapsed: 0.035 sec  <<< FAILURE!
org.opentest4j.AssertionFailedError: 1 + 1 should equal 2 ==> expected: <2> but was: <3>
        at com.example.project.FirstTest.myFirstTest(FirstTest.java:27)

In contrast the output from the IntelliJ plugin using the @DisplayName results in more readable test names.

image

Desired out put would be

Running FirstTest
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.115 sec <<< FAILURE! - in FirstTest
My 1st JUnit 5 test! 😎  Time elapsed: 0.035 sec  <<< FAILURE!
org.opentest4j.AssertionFailedError: 1 + 1 should equal 2 ==> expected: <2> but was: <3>
        at com.example.project.FirstTest.myFirstTest(FirstTest.java:27)

For parameterized Tests it would be great to have an output like in IntelliJ in the surefire plugin to. (DisplayName, Iteration and value of the argument causing a failed test)

image

Deliverables

  • ...
@marcphilipp

This comment has been minimized.

Show comment
Hide comment
@marcphilipp

marcphilipp Jul 27, 2017

Member

The rationale behind not using display names is that it would "break" the XML report generated by Surefire. Thus, we need a new reporting format and Surefire needs to adopt it before we can report display names (cf. #373).

Member

marcphilipp commented Jul 27, 2017

The rationale behind not using display names is that it would "break" the XML report generated by Surefire. Thus, we need a new reporting format and Surefire needs to adopt it before we can report display names (cf. #373).

@sbrannen

This comment has been minimized.

Show comment
Hide comment
@sbrannen

sbrannen Jul 27, 2017

Member

+1 to what @marcphilipp said: it's unfortunately not possible to support all JUnit Platform reporting features in reporting systems that only support JUnit 4.

So, the entire world has to wait until there is a new reporting standard which supports custom display names, test sources, etc.

Member

sbrannen commented Jul 27, 2017

+1 to what @marcphilipp said: it's unfortunately not possible to support all JUnit Platform reporting features in reporting systems that only support JUnit 4.

So, the entire world has to wait until there is a new reporting standard which supports custom display names, test sources, etc.

@luolong

This comment has been minimized.

Show comment
Hide comment
@luolong

luolong Oct 20, 2017

The rationale behind not using display names is that it would "break" the XML report generated by Surefire.

So how come Gradle reports are fine?

luolong commented Oct 20, 2017

The rationale behind not using display names is that it would "break" the XML report generated by Surefire.

So how come Gradle reports are fine?

@marcphilipp

This comment has been minimized.

Show comment
Hide comment
@marcphilipp

marcphilipp Oct 20, 2017

Member

So how come Gradle reports are fine?

Because our Gradle plugin uses ConsoleLauncher to write these XML reports.

Member

marcphilipp commented Oct 20, 2017

So how come Gradle reports are fine?

Because our Gradle plugin uses ConsoleLauncher to write these XML reports.

@mibutec

This comment has been minimized.

Show comment
Hide comment
@mibutec

mibutec May 14, 2018

Same for Dynamic Tests

Reporting in CI environment

Error | testFactoryMethodName[1][1]
Error | testFactoryMethodName[1][4]
Error | testFactoryMethodName[1][6]
Error | testFactoryMethodName[1][7]
Error | testFactoryMethodName[1][8]

Could it be a solution to enode display names in a way they are compatible to surefire, i.e. My 1st JUnit 5 test! 😎 => My_1st_JUnit_5_test

mibutec commented May 14, 2018

Same for Dynamic Tests

Reporting in CI environment

Error | testFactoryMethodName[1][1]
Error | testFactoryMethodName[1][4]
Error | testFactoryMethodName[1][6]
Error | testFactoryMethodName[1][7]
Error | testFactoryMethodName[1][8]

Could it be a solution to enode display names in a way they are compatible to surefire, i.e. My 1st JUnit 5 test! 😎 => My_1st_JUnit_5_test

@mibutec

This comment has been minimized.

Show comment
Hide comment
@mibutec

mibutec May 14, 2018

Is there any workaround?

mibutec commented May 14, 2018

Is there any workaround?

@mibutec

This comment has been minimized.

Show comment
Hide comment
@mibutec

mibutec May 22, 2018

@marcphilipp @sbrannen

How about this? For now Dynamic Tests are hardly to use in CI...

mibutec commented May 22, 2018

@marcphilipp @sbrannen

How about this? For now Dynamic Tests are hardly to use in CI...

@sbrannen

This comment has been minimized.

Show comment
Hide comment
@sbrannen

sbrannen Jun 3, 2018

Member

Since JUnit 5's Surefire provider is currently being taken over by the Maven team, the JUnit Team is not actively working on Surefire related issues.

Member

sbrannen commented Jun 3, 2018

Since JUnit 5's Surefire provider is currently being taken over by the Maven team, the JUnit Team is not actively working on Surefire related issues.

@dantran

This comment has been minimized.

Show comment
Hide comment
@dantran

dantran commented Sep 11, 2018

@k1w1m8

This comment has been minimized.

Show comment
Hide comment
@k1w1m8

k1w1m8 Sep 13, 2018

per #1320

I was wondering what was happening in Surefire wrt JUnit5.

It appears there is a single committer on the project. They are obviously doing an heroic job there, but there is a limit to what one person can do. Therefore, tickets for fixing the reporting of new JUnit5 features are not being worked on.

https://issues.apache.org/jira/browse/SUREFIRE-1567
https://issues.apache.org/jira/browse/SUREFIRE-1546

IMHO, having JUnit5 without first class support for new features in Surefire is pretty pointless given the importance of Maven to the Java ecosystem.

Is this something that the JUnit5 team are concerned about? Is there a plan to address this issue?

I've never worked on Surefire, but I'm considering getting involved...

k1w1m8 commented Sep 13, 2018

per #1320

I was wondering what was happening in Surefire wrt JUnit5.

It appears there is a single committer on the project. They are obviously doing an heroic job there, but there is a limit to what one person can do. Therefore, tickets for fixing the reporting of new JUnit5 features are not being worked on.

https://issues.apache.org/jira/browse/SUREFIRE-1567
https://issues.apache.org/jira/browse/SUREFIRE-1546

IMHO, having JUnit5 without first class support for new features in Surefire is pretty pointless given the importance of Maven to the Java ecosystem.

Is this something that the JUnit5 team are concerned about? Is there a plan to address this issue?

I've never worked on Surefire, but I'm considering getting involved...

@sbrannen

This comment has been minimized.

Show comment
Hide comment
@sbrannen

sbrannen Sep 13, 2018

Member

@k1w1m8, please see my response in #1320 (comment).

Member

sbrannen commented Sep 13, 2018

@k1w1m8, please see my response in #1320 (comment).

@k1w1m8

This comment has been minimized.

Show comment
Hide comment
@k1w1m8

k1w1m8 commented Sep 13, 2018

Thanks @sbrannen.

@sormuras

This comment has been minimized.

Show comment
Hide comment
Member

sormuras commented Sep 21, 2018

@sormuras sormuras closed this Sep 21, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment