Add the ability to select the test type to run #16788

stuartwdouglas · 2021-04-26T01:28:26Z

Also makes sure unit tests run before @QuarkusTest
based tests, and fixes a dev mode startup time
regression.

quarkus-bot · 2021-04-26T04:27:49Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `2558fe4`

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 15	`Build`	Test failures	Logs	Raw logs
✖	Native Tests - Security3	`Build`	⚠️ Check →	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/oidc/deployment

✖ io.quarkus.oidc.test.CodeFlowDevModeDefaultTenantTestCase.testAccessAndRefreshTokenInjectionDevMode - More details - Source on GitHub

✖ io.quarkus.oidc.test.CodeFlowDevModeTestCase.testAccessAndRefreshTokenInjectionDevMode - More details - Source on GitHub

✖ io.quarkus.oidc.test.SecurityDisabledTestCase. - More details - Source on GitHub

⚙️ JVM Tests - JDK 15 #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.brokenonly.TestBrokenOnlyTestCase.testBrokenOnlyMode line 73 - More details - Source on GitHub

gsmet · 2021-04-26T11:40:05Z

Test failures look suspicious.

We have all sorts of new flaky tests I have never seen before since a few days. I have no idea if they are new or if something weird is going on.

core/deployment/src/main/java/io/quarkus/deployment/dev/testing/TestConfig.java

quarkus-bot · 2021-04-27T13:18:55Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `5c92f2e`

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 15	`Build`	Test failures	Logs	Raw logs
✖	Native Tests - Security2	`Build`	Test failures	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/oidc/deployment

✖ io.quarkus.oidc.test.CodeFlowDevModeDefaultTenantTestCase.testAccessAndRefreshTokenInjectionDevMode - More details - Source on GitHub

✖ io.quarkus.oidc.test.CodeFlowDevModeTestCase.testAccessAndRefreshTokenInjectionDevMode - More details - Source on GitHub

✖ io.quarkus.oidc.test.SecurityDisabledTestCase. - More details - Source on GitHub

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.TestChangeTrackingWhenStartFailsTestCase.testChangeTrackingOnStartupFailure line 77 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.includes.IncludePatternTestCase.checkTestsAreRun line 71 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.tags.IncludeTagsTestCase.checkTestsAreRun line 52 - More details - Source on GitHub

⚙️ JVM Tests - JDK 15 #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.UnitTestTypeTestCase.testUnitMode line 36 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.tags.IncludeTagsTestCase.checkTestsAreRun line 65 - More details - Source on GitHub

⚙️ Native Tests - Security2 #

📦 integration-tests/oidc

✖ io.quarkus.it.keycloak.BearerTokenAuthorizationInGraalITCase.testAccessAdminResourceCustomHeaderNoBearerScheme - More details - Source on GitHub

quarkus-bot · 2021-04-27T16:16:28Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `04f708b`

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 15	`Build`	Test failures	Logs	Raw logs
✖	Maven Tests - JDK 11	`Build`	Test failures	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/oidc/deployment

✖ io.quarkus.oidc.test.CodeFlowDevModeDefaultTenantTestCase.testAccessAndRefreshTokenInjectionDevMode - More details - Source on GitHub

✖ io.quarkus.oidc.test.CodeFlowDevModeTestCase.testAccessAndRefreshTokenInjectionDevMode - More details - Source on GitHub

✖ io.quarkus.oidc.test.SecurityDisabledTestCase. - More details - Source on GitHub

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.tags.ExcludeTagsTestCase.checkTestsAreRun line 73 - More details - Source on GitHub

⚙️ JVM Tests - JDK 15 #

📦 extensions/oidc/deployment

✖ io.quarkus.oidc.test.CodeFlowDevModeDefaultTenantTestCase.testAccessAndRefreshTokenInjectionDevMode - More details - Source on GitHub

✖ io.quarkus.oidc.test.CodeFlowDevModeTestCase.testAccessAndRefreshTokenInjectionDevMode - More details - Source on GitHub

✖ io.quarkus.oidc.test.SecurityDisabledTestCase. - More details - Source on GitHub

⚙️ Maven Tests - JDK 11 #

📦 integration-tests/maven

✖ io.quarkus.maven.it.DevMojoIT.testThatNewResourcesAreServed line 841 - More details - Source on GitHub

quarkus-bot · 2021-04-28T00:43:39Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `532bf51`

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 15

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.TestRunnerSmokeTestCase.checkTestsAreRun line 84 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.tags.ExcludeTagsTestCase.checkTestsAreRun line 73 - More details - Source on GitHub

quarkus-bot · 2021-04-29T03:19:38Z

This workflow status is outdated as a new workflow run has been triggered.

🚫 This workflow run has been cancelled.

Failing Jobs - Building `14fd670`

⚠️ Artifacts of the workflow run were not available thus the report misses some details.

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
⌛	JVM Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
✔️	JVM Tests - JDK 15

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.TestChangeTrackingWhenStartFailsTestCase.testChangeTrackingOnStartupFailure line 82 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.includes.ExcludePatternTestCase.checkTestsAreRun line 58 - More details - Source on GitHub

quarkus-bot · 2021-04-29T04:33:49Z

This workflow status is outdated as a new workflow run has been triggered.

🚫 This workflow run has been cancelled.

Failing Jobs - Building `083bc1a`

⚠️ Artifacts of the workflow run were not available thus the report misses some details.

Status	Name	Step	Test failures	Logs	Raw logs
⌛	JVM Tests - JDK 11	`Build`	⚠️ Check →	Logs	Raw logs
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
⌛	JVM Tests - JDK 15	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - HTTP	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Messaging1	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc1	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc2	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc3	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc4	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Spring	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - gRPC	`Build`	⚠️ Check →	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.includes.IncludePatternTestCase.checkTestsAreRun line 71 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.tags.ExcludeTagsTestCase.checkTestsAreRun line 68 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.tags.IncludeTagsTestCase.checkTestsAreRun line 65 - More details - Source on GitHub

quarkus-bot · 2021-04-29T07:43:55Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `1f7fb01`

Status	Name	Step	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 11
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 15	`Build`	Test failures	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.TestRunnerSmokeTestCase.checkTestsAreRun line 74 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.tags.IncludeTagsTestCase.checkTestsAreRun line 70 - More details - Source on GitHub

⚙️ JVM Tests - JDK 15 #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.TestRunnerSmokeTestCase.checkTestsAreRun line 69 - More details - Source on GitHub

quarkus-bot · 2021-04-30T02:24:54Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `ae30697`

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 15

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.TestRunnerSmokeTestCase.checkTestsAreRun line 74 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.includes.IncludePatternTestCase.checkTestsAreRun line 53 - More details - Source on GitHub

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.TestChangeTrackingWhenStartFailsTestCase.testChangeTrackingOnStartupFailure line 51 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.tags.ExcludeTagsTestCase.checkTestsAreRun line 73 - More details - Source on GitHub

Also makes sure unit tests run before @QuarkusTest based tests, and fixes a dev mode startup time regression.

quarkus-bot · 2021-04-30T08:26:40Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `9499fd0`

Status	Name	Step	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 11
✖	JVM Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
✔️	JVM Tests - JDK 15

quarkus-bot bot added area/core area/vertx labels Apr 26, 2021

geoand approved these changes Apr 26, 2021

View reviewed changes

machi1990 reviewed Apr 26, 2021

View reviewed changes

core/deployment/src/main/java/io/quarkus/deployment/dev/testing/TestConfig.java Outdated Show resolved Hide resolved

stuartwdouglas force-pushed the test-mode branch from 2558fe4 to 5c92f2e Compare April 27, 2021 10:21

gsmet approved these changes Apr 27, 2021

View reviewed changes

gsmet added the release/noteworthy-feature label Apr 27, 2021

stuartwdouglas force-pushed the test-mode branch from 5c92f2e to 04f708b Compare April 27, 2021 13:24

stuartwdouglas force-pushed the test-mode branch from 04f708b to 532bf51 Compare April 27, 2021 21:29

stuartwdouglas force-pushed the test-mode branch 2 times, most recently from 14fd670 to 083bc1a Compare April 29, 2021 03:13

stuartwdouglas force-pushed the test-mode branch from 083bc1a to 1f7fb01 Compare April 29, 2021 04:30

quarkus-bot bot added the area/testing label Apr 29, 2021

stuartwdouglas force-pushed the test-mode branch from 1f7fb01 to bea6ffb Compare April 29, 2021 08:22

gastaldi added the triage/needs-rebase This PR needs to be rebased first because it has merge conflicts label Apr 29, 2021

stuartwdouglas force-pushed the test-mode branch from bea6ffb to ae30697 Compare April 30, 2021 00:03

Add the ability to select the test type to run

9499fd0

Also makes sure unit tests run before @QuarkusTest based tests, and fixes a dev mode startup time regression.

stuartwdouglas force-pushed the test-mode branch from ae30697 to 9499fd0 Compare April 30, 2021 05:02

stuartwdouglas merged commit fcd4971 into quarkusio:main May 1, 2021

quarkus-bot bot added this to the 2.0 - main milestone May 1, 2021

famod removed the triage/needs-rebase This PR needs to be rebased first because it has merge conflicts label May 7, 2021

Add the ability to select the test type to run #16788

Add the ability to select the test type to run #16788

Conversation

stuartwdouglas commented Apr 26, 2021

quarkus-bot bot commented Apr 26, 2021 • edited

Failing Jobs - Building 2558fe4

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/oidc/deployment

⚙️ JVM Tests - JDK 15 #

📦 extensions/vertx-http/deployment

gsmet commented Apr 26, 2021

quarkus-bot bot commented Apr 27, 2021 • edited

Failing Jobs - Building 5c92f2e

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/oidc/deployment

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

⚙️ JVM Tests - JDK 15 #

📦 extensions/vertx-http/deployment

⚙️ Native Tests - Security2 #

📦 integration-tests/oidc

quarkus-bot bot commented Apr 27, 2021 • edited

Failing Jobs - Building 04f708b

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/oidc/deployment

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

⚙️ JVM Tests - JDK 15 #

📦 extensions/oidc/deployment

⚙️ Maven Tests - JDK 11 #

📦 integration-tests/maven

quarkus-bot bot commented Apr 28, 2021 • edited

Failing Jobs - Building 532bf51

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/vertx-http/deployment

quarkus-bot bot commented Apr 29, 2021 • edited

Failing Jobs - Building 14fd670

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/vertx-http/deployment

quarkus-bot bot commented Apr 29, 2021 • edited

Failing Jobs - Building 083bc1a

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

quarkus-bot bot commented Apr 29, 2021 • edited

Failing Jobs - Building 1f7fb01

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

⚙️ JVM Tests - JDK 15 #

📦 extensions/vertx-http/deployment

quarkus-bot bot commented Apr 30, 2021 • edited

Failing Jobs - Building ae30697

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 extensions/vertx-http/deployment

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

quarkus-bot bot commented Apr 30, 2021 • edited

Failing Jobs - Building 9499fd0

quarkus-bot bot commented Apr 26, 2021 •

edited

Failing Jobs - Building `2558fe4`

quarkus-bot bot commented Apr 27, 2021 •

edited

Failing Jobs - Building `5c92f2e`

quarkus-bot bot commented Apr 27, 2021 •

edited

Failing Jobs - Building `04f708b`

quarkus-bot bot commented Apr 28, 2021 •

edited

Failing Jobs - Building `532bf51`

quarkus-bot bot commented Apr 29, 2021 •

edited

Failing Jobs - Building `14fd670`

quarkus-bot bot commented Apr 29, 2021 •

edited

Failing Jobs - Building `083bc1a`

quarkus-bot bot commented Apr 29, 2021 •

edited

Failing Jobs - Building `1f7fb01`

quarkus-bot bot commented Apr 30, 2021 •

edited

Failing Jobs - Building `ae30697`

quarkus-bot bot commented Apr 30, 2021 •

edited

Failing Jobs - Building `9499fd0`