No tags in reports when updating from serenity version 3.6.12 to 4.0.12

[Totiii]

What happened?

I just updated my serenity version to the 4.0.12 and since then i don't see my tags in the reports. It was working in 3.6.12.
I changed my import from net.thucydides.core.annotations.WithTagValuesOf to net.serenitybdd.annotations.WithTagValuesOf.
I also can't aggregate my report based on the tags, like i did before with this command: mvn serenity:aggregate -Dtags="Tag1"

@ExtendWith(SerenityJUnit5Extension.class)
@WithTagValuesOf({"Tag1", "Tag2"})
public class Scenario {

    @Test
    public void test() {
      // my test
    }
}

What did you expect to happen?

I want to see my tags in my tests and also be able to aggregate my tests with this command: mvn serenity:aggregate -Dtags="myTag"

Serenity BDD version

4.0.12

JDK version

17.0.2

Execution environment

Windows, firefox

How to reproduce the bug.

This is what my features looks like:

@ExtendWith(SerenityJUnit5Extension.class)
@WithTagValuesOf({"Tag1", "Tag2"})
public class Scenario {

    @Test
    public void test() {
      // my test
    }
}

And this is my pom.xml

  <properties>
    <java.version>17</java.version>
    <serenity.version>4.0.12</serenity.version>
    <junit5.version>5.9.2</junit5.version>
    <junit-jupiter.version>5.10.0</junit-jupiter.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <encoding>UTF-8</encoding>
    <tags></tags>
    <parallel.tests>4</parallel.tests>
    <webdriver.base.url></webdriver.base.url>
  </properties>

  <dependencies>
    <!-- Serenity -->
    <dependency>
      <groupId>net.serenity-bdd</groupId>
      <artifactId>serenity-core</artifactId>
      <version>${serenity.version}</version>
    </dependency>
    <dependency>
      <groupId>net.serenity-bdd</groupId>
      <artifactId>serenity-junit5</artifactId>
      <version>${serenity.version}</version>
    </dependency>
    <dependency>
      <groupId>net.serenity-bdd</groupId>
      <artifactId>serenity-screenplay</artifactId>
      <version>${serenity.version}</version>
    </dependency>
    <dependency>
      <groupId>net.serenity-bdd</groupId>
      <artifactId>serenity-ensure</artifactId>
      <version>${serenity.version}</version>
    </dependency>
    <dependency>
      <groupId>net.serenity-bdd</groupId>
      <artifactId>serenity-screenplay-webdriver</artifactId>
      <version>${serenity.version}</version>
    </dependency>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-classic</artifactId>
      <version>1.4.6</version>
    </dependency>
    <!-- JUNIT 5 DEPENDENCY-->
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-api</artifactId>
      <version>${junit-jupiter.version}</version>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>${junit-jupiter.version}</version>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-params</artifactId>
      <version>${junit-jupiter.version}</version>
    </dependency>
    <dependency>
      <groupId>org.assertj</groupId>
      <artifactId>assertj-core</artifactId>
      <version>3.24.2</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>3.0.0-M5</version>
        <configuration>
          <includes>
            <include>**/Scenario*.java</include>
          </includes>
          <systemPropertyVariables>
            <webdriver.base.url>${webdriver.base.url}</webdriver.base.url>
            <junit.jupiter.extensions.autodetection.enabled>true</junit.jupiter.extensions.autodetection.enabled>
          </systemPropertyVariables>
          <parallel>classes</parallel>
          <threadCount>${parallel.tests}</threadCount>
          <forkCount>${parallel.tests}</forkCount>
        </configuration>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.11.0</version>
        <configuration>
          <source>17</source>
          <target>17</target>
          <compilerArgs>
            <arg>-parameters</arg>
          </compilerArgs>
        </configuration>
      </plugin>
      <plugin>
        <groupId>net.serenity-bdd.maven.plugins</groupId>
        <artifactId>serenity-maven-plugin</artifactId>
        <version>${serenity.version}</version>
        <configuration>
          <tags>${tags}</tags>
        </configuration>
        <executions>
          <execution>
            <id>serenity-reports</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>aggregate</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

[Totiii]

Am I the only one having this problem ? Or are there any news on this issue ?

[wakaleo]

@WithTagValuesOf is not supported in JUnit 5, use the native JUnit 5 tags instead.

[Totiii]

I tried using @tag from org.junit.jupiter.api.Tag. It works with the tests that are annotated @test, but it doesn't work with the tests annotated with @ParameterizedTest.

[wakaleo]

Make sure you put the @tag annotation above the parameterised test itself and not the class as a whole:

[Totiii]

I did and as you can see, in the reports the tags are not here.

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.extension.ExtendWith;

@ExtendWith(SerenityJUnit5Extension.class)
@Tag("Tag-on-class-parameterized")
public class Scenario_Test {

  private static Stream<Arguments> getParams() throws Exception {
    // return a stream of params of type Params;
  }

  @ParameterizedTest
  @Tag("Tag-on-test-parameterized")
  @MethodSource("getParams")
  public void TestToSeeIfJunit5TagsWorks(Params p) {
  // test content
  }
}

[wakaleo]

OK, there might be an issue adding the tags for parameterised tests. I'm not sure why that might be. Have a look at the use of the JUnit5Tags and SerenityJUnit5Extension classes in serenity-junit5 if you would like to investigate.

[Totiii]

I tried investigating on my side. So as i said before, the tests without the @ParameterizedTest annotation works well. But when we have the @ParameterizedTest, you can't aggregate your tests based on the tag like so: mvn serenity:aggregate -Dtags="myTag" and the tags does not appear in the report.
I tried watching in the SerenityJUnit5Extension what was happening and i did not find any differences between a run with a @Test and a @ParameterizedTest. For both tests type the tags are retrieved the same way and for both they are what they should be. Based on that, i don't think the issue come from this class, but from somewhere else.

[Totiii]

Hello, i recently updated my serenity to the 3.9.8 instead of the 4.0 and, it looks like the problem is the same. I think this could help you track the regression.

[wakaleo]

I did intestigate this. JUnit 5 tags are not fully supported in Serenity 4 (or Serenity 3, which didn't support JUnit 5). JUnit 5 has it's own tag language which is not portable and needs to be reproduced in the Serenity code and integrated into the test filtering and reporting logic to support it. If this feature is important for your company you could speed up the implementation (or ensure that it is implemented in the near future) by either implementing it on your side, or opting for one of the commerical support packages https://www.serenitydojo.academy/serenity-bdd-support-packages). The commercial support packages are generally the fastest and most cost-effective solution.