Live reload stopped working on 3.2 when using XA transactions

[imperatorx]

Describe the bug

When upgrading from 3.1 to 3.2, using quarkus:dev mode, the application will no longer reload on code changes, but fail with an exception, if an XA transaction manager is used (quarkus.datasource.jdbc.transactions=xa).

Expected behavior

Application reloads as before

Actual behavior

023-07-06 14:42:10,384 ERROR [io.qua.run.Application] (Quarkus Main Thread) Failed to start application (with profile [dev]) [Error Occurred After Shutdown]: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	at io.quarkus.runtime.Application.start(Application.java:101)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:111)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:124)
	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:104)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.IllegalStateException: ARJUNA012400: Cannot terminate the recovery manager as the implementation is not known. Could be the recovery manager was not started yet?
	at com.arjuna.ats.arjuna.recovery.RecoveryManager.checkState(RecoveryManager.java:489)
	at com.arjuna.ats.arjuna.recovery.RecoveryManager.getModules(RecoveryManager.java:355)
	at com.arjuna.ats.jbossatx.jta.RecoveryManagerService.addXAResourceRecovery(RecoveryManagerService.java:97)
	at io.quarkus.narayana.jta.runtime.QuarkusRecoveryService.addXAResourceRecovery(QuarkusRecoveryService.java:30)
	at io.agroal.narayana.NarayanaTransactionIntegration.addResourceRecoveryFactory(NarayanaTransactionIntegration.java:132)
	at io.agroal.pool.ConnectionPool.init(ConnectionPool.java:133)
	at io.agroal.pool.DataSource.<init>(DataSource.java:42)
	at io.quarkus.agroal.runtime.DataSources.doCreateDataSource(DataSources.java:246)
	at io.quarkus.agroal.runtime.DataSources$1.apply(DataSources.java:129)
	at io.quarkus.agroal.runtime.DataSources$1.apply(DataSources.java:126)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
	at io.quarkus.agroal.runtime.DataSources.getDataSource(DataSources.java:126)
	at io.quarkus.agroal.runtime.DataSources.fromName(DataSources.java:122)
	at io.quarkus.agroal.runtime.AgroalRecorder.agroalDataSourceSupplier(AgroalRecorder.java:23)
	at io.quarkus.deployment.steps.AgroalProcessor$generateDataSourceBeans109901991.deploy_0(Unknown Source)
	at io.quarkus.deployment.steps.AgroalProcessor$generateDataSourceBeans109901991.deploy(Unknown Source)
	... 11 more

How to Reproduce?

modules: agroal, narayana-jta, postgres jdbc driver

quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=postgres
quarkus.datasource.password=postgres
quarkus.datasource.jdbc.url=jdbc:postgresql://localhost:5432/postgres
quarkus.datasource.jdbc.transactions=xa
quarkus.transaction-manager.enable-recovery=true

    @Inject
    DataSource ds;

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() throws Exception{
        QuarkusTransaction.requiringNew()
                .run(()->{
                    try(var c = ds.getConnection()) {

                    } catch (Exception e) {
                        throw new RuntimeException(e);
                    }
                });
        return "blah"; // <-- change output to test live reload
    }

Output of uname -a or ver

No response

Output of java -version

20

GraalVM version (if different from Java)

20

Quarkus version or git rev

3.2.0.Final

Build tool (ie. output of mvnw --version or gradlew --version)

maven

Additional information

No response

[quarkus-bot]

/cc @mmusgrov (narayana)

[mmusgrov]

I can't reproduce this issue. Can you include a better reproducer.

I created an app using https://code.quarkus.io/?e=jdbc-postgresql&e=agroal&e=narayana-jta&e=resteasy (with a test JAX-RS endpoint), so the latest.

I tested it using mvn quarkus:dev, ran the test and typed ctrl-C.

[mmusgrov]

Note that I didn't write any JPA code, but if that is what is required to exhibit the issue then it needs to be included in a reproducer.

[imperatorx]

Reproducer: https://github.com/imperatorx/recover-reload-reproducer
(tested using JDK 20 and 21)

Steps:

• run mvnw quarkus:dev
• load http://127.0.0.1:8080/hello
• in the IDE, change "Hello from RESTEasy Reactive" to "Hello from RESTEasy Reactive2" in the GreetingResource. Do NOT press CTRL+C
• reload the browser
• see error in the browser and in the logs

[zyc]

Same problem here

[gsmet]

@mmusgrov any chance someone from the Narayana team could have a look? This looks highly problematic for 3.2.

/cc @yrodiere @Sanne @geoand

[mmusgrov]

Yes, @marcosgopen is investigating, he will provide an update when he has done more analysis.

[gsmet]

Yeah I can confirm I can reproduce the issue with the reproducer above.

[mmusgrov]

The issue is the NarayanaJtaRecorder#handleShutdown method isn't resetting the recoveryManagerService instance.
I'll get the PR raised ...

[marcosgopen]

The issue is the NarayanaJtaRecorder#handleShutdown method isn't resetting the recoveryManagerService instance. I'll get the PR raised ...

Thanks @mmusgrov !