Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No tests ran when using tags alongside the Serenity Batching feature #2776

Open
codejco opened this issue Apr 12, 2022 Discussed in #2773 · 2 comments
Open

No tests ran when using tags alongside the Serenity Batching feature #2776

codejco opened this issue Apr 12, 2022 Discussed in #2773 · 2 comments

Comments

@codejco
Copy link

codejco commented Apr 12, 2022

Hi all, I would like to ask if running serenity batches for a specific set of test with tags is possible.

I've tried to follow the link below:
https://serenity-bdd.github.io/theserenitybook/latest/serenity-parallel-batches.html

And I am able to run my whole suite and get it parallelized in batch but when I add the tag on the mvn command, I get 0 tests run.

[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in cucumber.CucumberTestSuite8
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in cucumber.CucumberTestSuite2

For context, I created 8 CucumberTestSuite.java to emulate the SlicedTestRunners on the link.

Sample command:
mvn clean verify -Dserenity.batch.count=1 -Dserenity.batch.number=1 -Dcucumber.filter.tags=@test -P useTheForks

Below is the POM I am using

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <serenity.version>3.2.0</serenity.version>
    <encoding>UTF-8</encoding>
    <tags></tags>
    <webdriver.base.url></webdriver.base.url>
    <parallel.tests>8</parallel.tests>
</properties>
<dependencies>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-core</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-cucumber</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-screenplay</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-screenplay-webdriver</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-screenplay-rest</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-rest-assured</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-junit</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-ensure</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.13.1</version>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.2.10</version>
    </dependency>
    <dependency>
        <groupId>org.assertj</groupId>
        <artifactId>assertj-core</artifactId>
        <version>3.6.2</version>
    </dependency>
    <dependency>
        <groupId>com.microsoft.sqlserver</groupId>
        <artifactId>mssql-jdbc</artifactId>
        <version>10.2.0.jre8</version>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.18.8</version>
        <scope>provided</scope>
    </dependency>
</dependencies>
<repositories>
    <repository>
        <id>projectlombok.org</id>
        <url>http://projectlombok.org/mavenrepo</url>
    </repository>
</repositories>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <includes>
                    <include>**/features/**/*.java</include>
                    <include>**/cucumber/*.java</include>
                </includes>
                <systemPropertyVariables>
                    <webdriver.base.url>${webdriver.base.url}</webdriver.base.url>
                </systemPropertyVariables>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
        <plugin>
            <groupId>net.serenity-bdd.maven.plugins</groupId>
            <artifactId>serenity-maven-plugin</artifactId>
            <version>${serenity.version}</version>
            <configuration>
                <tags>${tags}</tags>
            </configuration>
            <executions>
                <execution>
                    <id>serenity-reports</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>
<profiles>
    <profile>
        <id>useTheForks</id>
        <build>
            <plugins>
                <plugin>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <version>3.0.0-M5</version>
                    <configuration>
                        <includes>
                            <include>**/CucumberTestSuite*.java</include>
                        </includes>
                        <parallel>methods</parallel>
                        <threadCount>${parallel.tests}</threadCount>
                        <forkCount>${parallel.tests}</forkCount>
                        <systemPropertyVariables>
                            <serenity.fork.count>0${parallel.tests}</serenity.fork.count>
                            <serenity.fork.number>0${surefire.forkNumber}</serenity.fork.number>
                        </systemPropertyVariables>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
@wakaleo
Copy link
Member

wakaleo commented Apr 12, 2022

I'm not sure the batch feature works with tags (it is a community-supported feature). What problem are you trying to solve - are you trying to run tests in parallel on a single machine?

@codejco
Copy link
Author

codejco commented Apr 12, 2022

I am currently comparing which parallelization is able to finish tests and found that the batching is able to use more forks compared to the standard one, thus finishing the tests faster. Eventually, we want to run our tests in Jenkins but are also looking for the option to only run specific tags which led me to checking if this works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants