TOMEE-4166 - Setup tck for JAX-RS

[Sollder11]

The tck currently relies on all the test-files to be present in the test folder (want to change that in the future).
Also, not all Tests are passing at the moment.

[Sollder11]

Rerun the tests with the tck dependency and still got:
[ERROR] Tests run: 2796, Failures: 12, Errors: 17, Skipped: 128
So no regressions and much cleaner that way.

[rzo1]

A full build with mvn --show-version --fail-at-end clean install -Pall-adapters -Dfile.encoding=UTF-8 -DskipTests=true

fails atm:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:3.2.0:jar (default-jar) on project jax-rs-tck: Error assembling JAR: Invalid automatic module name: 'org.apache.tomee.jax-rs-tck' -> [Help 1]

Regarding the user stuff:

Maybe have a look here:

• https://github.com/apache/tomee-tck/blob/tomee-10-with-ee-91/src/test/tomee-plume/conf/tomcat-users.xml

and here:

• https://github.com/apache/tomee-tck/blob/tomee-10-with-ee-91/src/test/resources/testsuite.properties
• https://github.com/apache/tomee-tck/blob/tomee-10-with-ee-91/src/test/tomee-plume/conf/system.properties
• https://github.com/apache/tomee-tck/blob/tomee-10-with-ee-91/src/test/tomee-plume/conf/cxf.properties

I think, that we just need to configure them ;)

There might be other properties required as well, so perhaps worth a look anyway.

[rzo1]

Guess we need to add these users

as written in the JAX-RS TCK docs.

It might be enought to just copy paste the tomcat-users.xml from the current tck and use that for the setup.

[Sollder11]

I think the ci does not like the snapshots tag either... maybe I will just remove it?

[Sollder11]

Hmm this failure is weird, it only seems to appear if you run the whole build, building it alone is no problem. I tried changing the name to jaxrs (so no dashes) and it compiles...

[rzo1]

The issue was related to source/jar building in the module, which we actually do not need.

So for me it now looks like this:

[ERROR] Tests run: 2796, Failures: 10, Errors: 16, Skipped: 128
[ERROR] Failures: 
[ERROR]   JAXRSClientIT.<init>:59->usersetup:97 user not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSClientIT.<init>:59->usersetup:97 user not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSClientIT.dynamicFeatureIsRegisteredTest:111->JAXRSCommonClient.assertEquals:721 [Ljava.lang.Object;@238291d4 ==> expected: <true> but was: <false>
[ERROR]   JAXRSClientIT.featureIsRegisteredTest:93->JAXRSCommonClient.assertEquals:721 [Ljava.lang.Object;@7ac48e10 ==> expected: <true> but was: <false>
[ERROR]   JsonbContextProviderIT.shouldUseApplicationProvidedJsonbInstance:115 
Expected: is "Origin(1728219222),CustomSerializer(CLIENT),CustomDeserializer(SERVER),EchoResource,CustomSerializer(SERVER),CustomDeserializer(CLIENT)"
     but: was "Origin(1728219222),CustomSerializer(CLIENT),CustomDeserializer(CLIENT),EchoResource,CustomSerializer(CLIENT),CustomDeserializer(CLIENT)"
[ERROR] Errors: 
[ERROR]   JAXRSClientIT.clientSideReaderIsNotUsedOnServerTest:124->JaxrsCommonClient.invoke:219 » Fault [JAXRSCommonClient] null failed!  Check output for cause of failure.
[ERROR]   JAXRSClientIT.clientSideReaderIsUsedOnClientTest:195->JaxrsCommonClient.invoke:219 » Fault [JAXRSCommonClient] null failed!  Check output for cause of failure.
[ERROR]   JAXRSClientIT.clientSideWriterIsNotUsedOnServerTest:159->JaxrsCommonClient.invoke:219 » Fault [JAXRSCommonClient] null failed!  Check output for cause of failure.
[ERROR]   JAXRSClientIT.clientSideWriterIsUsedOnClientTest:234->JaxrsCommonClient.invoke:219 » Fault [JAXRSCommonClient] null failed!  Check output for cause of failure.
[ERROR]   JAXRSClientIT.serverSideReaderIsNotUsedOnClientTest:177->JaxrsCommonClient.invoke:219 » Fault [JAXRSCommonClient] null failed!  Check output for cause of failure.
[ERROR]   JAXRSClientIT.serverSideReaderIsUsedOnServerTest:107->JaxrsCommonClient.invoke:219 » Fault [JAXRSCommonClient] null failed!  Check output for cause of failure.
[ERROR]   JAXRSClientIT.serverSideWriterIsNotUsedOnClientTest:213->JaxrsCommonClient.invoke:219 » Fault [JAXRSCommonClient] null failed!  Check output for cause of failure.
[ERROR]   JAXRSClientIT.serverSideWriterIsUsedOnServerTest:141->JaxrsCommonClient.invoke:219 » Fault [JAXRSCommonClient] null failed!  Check output for cause of failure.
[ERROR]   SeBootstrapIT.shouldBootInstanceDespiteUnknownConfigurationParameters:236 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingConvenienceMethods:148 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingDefaults:71 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingExternalConfiguration:186 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingImplementationsDefaultIpPort:317 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingProperties:107 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingSelfDetectedFreeIpPort:280 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   JAXRSSigTestIT.signatureTest:415 Fault signatureTest failed with an unexpected exception

I think, we need to fix the user stuff and than figure out, which of these other tests are actually failing because of the new spec version or if we need to fix something in the setup (which is present in tomee-tck).

The SEBootstrap stuff is new, so expected to fail atm. SigTests are also a candidate for failure atm + dynamic provider extensions (cf. https://jakarta.ee/specifications/restful-ws/3.1/)

[Sollder11]

The issue was related to source/jar building in the module, which we actually do not need.

Oh, I see, makes sense.

So for me it now looks like this:

Same

I think, we need to fix the user stuff

Yes, I will try to configure that, it seems like one can define them in an arquillian.xml and arquillian-tomee-embedded -> EmbeddedTomEEContainer will pick them up.

[rzo1]

So... just added another property so it looks like

[ERROR] Failures: 
[ERROR]   JAXRSClientIT.<init>:59->usersetup:97 user not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSClientIT.<init>:59->usersetup:97 user not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSBasicClientIT.<init>:59->JAXRSClient.setup:53 user was not set ==> expected: <true> but was: <false>
[ERROR]   JAXRSClientIT.dynamicFeatureIsRegisteredTest:111->JAXRSCommonClient.assertEquals:721 [Ljava.lang.Object;@238291d4 ==> expected: <true> but was: <false>
[ERROR]   JAXRSClientIT.featureIsRegisteredTest:93->JAXRSCommonClient.assertEquals:721 [Ljava.lang.Object;@7ac48e10 ==> expected: <true> but was: <false>
[ERROR]   JsonbContextProviderIT.shouldUseApplicationProvidedJsonbInstance:115 
Expected: is "Origin(672977989),CustomSerializer(CLIENT),CustomDeserializer(SERVER),EchoResource,CustomSerializer(SERVER),CustomDeserializer(CLIENT)"
     but: was "Origin(672977989),CustomSerializer(CLIENT),CustomDeserializer(CLIENT),EchoResource,CustomSerializer(CLIENT),CustomDeserializer(CLIENT)"
[ERROR] Errors: 
[ERROR]   SeBootstrapIT.shouldBootInstanceDespiteUnknownConfigurationParameters:236 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingConvenienceMethods:148 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingDefaults:71 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingExternalConfiguration:186 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingImplementationsDefaultIpPort:317 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingProperties:107 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingSelfDetectedFreeIpPort:280 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   JAXRSSigTestIT.signatureTest:415 Fault signatureTest failed with an unexpected exception
[INFO] 
[ERROR] Tests run: 2796, Failures: 10, Errors: 8, Skipped: 128

If the users are added and are working, we have a good view, I guess.

[Sollder11]

Fixed the issue with the user, they go through now... interestingly enough, the config necessary has almost nothing to do with what is described in the documentation. My last run now looks like this:

[ERROR] Failures: 
[ERROR]   JAXRSClientIT.dynamicFeatureIsRegisteredTest:111->JAXRSCommonClient.assertEquals:721 [Ljava.lang.Object;@fe42a09 ==> expected: <true> but was: <false>
[ERROR]   JAXRSClientIT.featureIsRegisteredTest:93->JAXRSCommonClient.assertEquals:721 [Ljava.lang.Object;@19bfbe28 ==> expected: <true> but was: <false>
[ERROR]   JsonbContextProviderIT.shouldUseApplicationProvidedJsonbInstance:115 
Expected: is "Origin(2124707916),CustomSerializer(CLIENT),CustomDeserializer(SERVER),EchoResource,CustomSerializer(SERVER),CustomDeserializer(CLIENT)"
     but: was "Origin(2124707916),CustomSerializer(CLIENT),CustomDeserializer(CLIENT),EchoResource,CustomSerializer(CLIENT),CustomDeserializer(CLIENT)"
[ERROR] Errors: 
[ERROR]   SeBootstrapIT.shouldBootInstanceDespiteUnknownConfigurationParameters:236 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingConvenienceMethods:148 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingDefaults:71 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingExternalConfiguration:186 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingImplementationsDefaultIpPort:317 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingProperties:107 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   SeBootstrapIT.shouldBootInstanceUsingSelfDetectedFreeIpPort:280 » AbstractMethod Receiver class org.apache.cxf.jaxrs.impl.RuntimeDelegateImpl does not define or inherit an implementation of the resolved method 'abstract jakarta.ws.rs.SeBootstrap$Configuration$Builder createConfigurationBuilder()' of abstract class jakarta.ws.rs.ext.RuntimeDelegate.
[ERROR]   JAXRSSigTestIT.signatureTest:415 Fault signatureTest failed with an unexpected exception
[INFO] 
[ERROR] Tests run: 2796, Failures: 3, Errors: 8, Skipped: 128

[rzo1]

Perhaps time to sent a mail to dev@ 🙃 - looks like those tests are all EE10/JAX-RS 3.1 related.

Maybe setting up the signature tests (even If they fail) would be a good addition.

[Sollder11]

Hi Richard, I added another module for the Signature-test. It seems to run fine and output a sig-file similar to that generated by cdi-signature-test. Any news from the cxf Team about a Jakarta-10 ready implementation?

If this is fine with you, I would send a mail to dev to ask for feedback/approval on this.

[Sollder11]

I am bascially thinking of adding the failing tests into exclusions (for now) and open related Jira issues for it, so we get the setup into "main" pretty soon.

Ok cool, I just rebased and running the tests again.
It seems the Signature Tests Module Fails now, but that was not set up correctly anyway, I believe.

[Sollder11]

Results are pretty much the same, for the matter of SeBootstrap it states in the jdoc:

The SeBootstrap class is available in a Jakarta EE container environment as well; however, support for the Java SE bootstrapping APIs is not required in container environments.

So I am not really sure if we even need to support these tests. I think I will dig a bit in the specification about that.

[rzo1]

We are a container env, so I guess it's save to exclude ;-)

[Sollder11]

We are a container env, so I guess it's save to exclude ;-)

I would agree, I excluded the test with a matching explanation.
I also exluded the tck-sig-test for now, as it breaks the build.

That leaves four Failures:

[ERROR] Failures: 
[ERROR]   JAXRSClientIT.sseBroadcastTest:193->JAXRSCommonClient.assertEquals:721 [Ljava.lang.Object;@38c460e8 ==> expected: <true> but was: <false>
[ERROR]   JAXRSClientIT.dynamicFeatureIsRegisteredTest:111->JAXRSCommonClient.assertEquals:721 [Ljava.lang.Object;@4a31ed12 ==> expected: <true> but was: <false>
[ERROR]   JAXRSClientIT.featureIsRegisteredTest:93->JAXRSCommonClient.assertEquals:721 [Ljava.lang.Object;@5b8853 ==> expected: <true> but was: <false>
[ERROR]   JsonbContextProviderIT.shouldUseApplicationProvidedJsonbInstance:115 
Expected: is "Origin(736163598),CustomSerializer(CLIENT),CustomDeserializer(SERVER),EchoResource,CustomSerializer(SERVER),CustomDeserializer(CLIENT)"
     but: was "Origin(736163598),CustomSerializer(CLIENT),CustomDeserializer(CLIENT),EchoResource,CustomSerializer(CLIENT),CustomDeserializer(CLIENT)"
[INFO] 
[ERROR] Tests run: 2788, Failures: 4, Errors: 0, Skipped: 128

[rzo1]

Just running a build with the updated TCK (the 3.1.5 isn't available yet on Maven, raised jakartaee/rest#1260, but if the CI looks fine (with the failures expected), we are going to integrate it (and exclude the tests for now) -> stay tuned.

[rzo1]

Thanks @Sollder1