test-infra

Test observability platform

Spec

• ✅ Should be able to list all the tests in the suite
• ✅ Should be able to identify owners for a given test in the suite
• ✅ Should be able to tag each test to a functional area
• ✅ Should be able to get metrics on how many tests belong to a given area
• ✅ Should be able to trigger a slack summary with each individual test failure as a child message in thread
• Should be able to calculate if a test is broken consistently (via report portal)
• Should be able to calculate if a test is flaky consistently (via areport portal)

Running tests

Run all tests

./gradlew clean test --info

Run only groups belonging to a certain group

./gradlew test -DincludedGroups=smoke --info

Run tests belonging to a group but excluding few

./gradlew test -DincludedGroups=smoke -DexcludedGroups=slow --info

Run tests belonging to multiple groups

./gradlew test -DincludedGroups=smoke,regression --info

Run tests only when they belong to both the groups specified

./gradlew runTests -Dgroups=smoke,regression --info

Check test listing dashboard

cd build/resources/main

python3 -m http.server 8000

Open the generated html by navigating to:

http://localhost:8000/static/index.html

Report portal (one time setup)

Download and install docker from docker website depending on your platform and install

brew install docker-compose

You can skip below if you already have docker-compose.yml file

curl -LO https://raw.githubusercontent.com/reportportal/reportportal/master/docker-compose.yml

docker-compose -p reportportal up -d --force-recreate

Explanation

This docker-compose command does the following:

• -p reportportal: Sets the project name to reportportal, which namespaces the containers, networks, and volumes created.
• up: Builds, (re)creates, and starts the services defined in the docker-compose.yml file.
• -d: Runs the containers in detached mode (in the background). --force-recreate: Forces a recreation of containers even if no configuration has changed.

Setup Jenkins

This project shares an example Jenkinsfile with a pipeline to indicate how a build and test run pipeline could look like

Test metrics in report portal

Let's first generate some test data to then organize it within report portal

./gradlew test -DincludedGroups=identity -Drp.launch=identity_tests -Drp.attributes="group:test_infra;test_type:backend;team:identity" --info

./gradlew test -DincludedGroups=onboarding -Drp.launch=onboarding_tests -Drp.attributes="group:test_infra;test_type:backend;team:onboarding" --info

./gradlew test -DincludedGroups=performance -Drp.launch=performance_tests -Drp.attributes="group:test_infra;test_type:backend;team:performance" --info

Gen AI

Copilot edits

Help me create a base test class in testng that all the classes in package io.automationhacks.testinfra.reqres.users; inherit from and that has the common setup method to set the base URL

---

Remove the setup method in child classes

---

Also make sure to remove any unused imports

---

Please apply the same changes in all files in users package

Relative path: src/test/java/io/automationhacks/testinfra/reqres/users

---

Please make below changes in all files in users package

Relative path: src/test/java/io/automationhacks/testinfra/reqres/users

• Ensure the class extends BaseTest
• Remove any unused imports
• Do not change the @Test annotation or anything in the test body