How to handle breaking upgrades while trusting tests

[rgcosma]

Hello!
A Quarkus app consists at a very basic level of src/main and src/test, and there are many convenience libraries like jupiter to help write good unit tests. All good, but this means the tests are run by the same JDK / Maven and read the same pom.xml as the app code. In the event of an upgrade that requires a change in the tests too (example: Wiremock automatically ungzips in version 3), what steps could one take to make sure tests can still be trusted? Consider tests run from outside the app (curl/postman) for comparison.

[cescoffier]

Can't you already do that? You can set up a multi-module project where the app is packaged in one module, and an IT module starts the app (from its package, not by depending on the artifact) and runs blackbox tests?

[rgcosma]

Hi, and thank you for the hint! This does look like something we'd want - app is app and tests are separate in eg Postman - but I do not see this in the field, what I do see when running eg quarkus create app name.cosma:jdbc-gateway --extension='rest' is https://github.com/cosmaname/jdbc-gateway/tree/main/src so they're bundled in a way that means if I upgrade from java 23 to Java 25, whatever breaks in the app also breaks in the tests.

[rgcosma]

Don't know if my question was phrased correctly, what I am looking for is - what is considered best practices? is this the Quarkus way of doing things?

[cescoffier]

It's not necessarily the Quarkus way, as Quarkus does not define a predefined approach for this. But this is something we often see. In general, most implementations of this approach do the following:

• package the application in a container
• The IT module has a Docker Compose that includes the application container + the required infrastructure services (database, Kafka...). The application is configured using environment variables.
• The test starts the docker-compose, waits for readiness, and then executes the test as a black box (invokes the REST API, for example).
• Do not forget to shutdown the containers after your test.

[rgcosma]

All quite sensible, thank you.
My worry is still: if the tests themselves break because I upgrade from Java 23 to 25 (Wiremock dependency being such an example), I need to edit both the tests and the app itself. Once I've done that, I can no longer ask "is the updated app passing the tests" because they are not the same tests. This leaves room for the lazy developer (or lazy AI!) to simply make them all return True.
My question then - and I do realize this is getting more philosophical than practical - is there some Quarkus-compliant way to isolate tests code from the app itself?
Maybe pass a different compiler to package and test?

[cescoffier]

In the approach I gave, your tests run in complete isolation and do not have to use the same JVM version. However, you won't be able to use wiremock, it has to be blackbox tests.