Question: POSTMAN test runner support with test containers

[ericwyles]

Hi! I am not sure if this is a bug, an unimplemented feature, or an error on my side, so I'm opening as a question for now.

I'm following along and making a sample application modeled after the api-lifecycle/shift-left-demo/spring-boot-order-service project to do contract tests against a sample API that I made for learning. But the behavior Im' seeing is present even in the spring-boot-order-service when I attempt to run using the POSTMAN runner.

Specifically, if you go into api-lifecycle/shift-left-demo/spring-boot-order-service/src/test/java/org/acme/order/api/OrderControllerContractTests.java and change this:

   @Test
   public void testOpenAPIContract() throws Exception {
      // Ask for an Open API conformance to be launched.
      TestRequest testRequest = new TestRequest.Builder()
            .serviceId("Order Service API:0.1.0")
            .runnerType(TestRunnerType.OPEN_API_SCHEMA.name())
            .testEndpoint("http://host.testcontainers.internal:" + port + "/api")
            .build();

To

   @Test
   public void testOpenAPIContract() throws Exception {
      // Ask for an Open API conformance to be launched.
      TestRequest testRequest = new TestRequest.Builder()
            .serviceId("Order Service API:0.1.0")
            .runnerType(TestRunnerType.POSTMAN.name()) // <-- CHANGED TO POSTMAN HERE
            .testEndpoint("http://host.testcontainers.internal:" + port + "/api")
            .build();

      TestResult testResult = microcksContainer.testEndpoint(testRequest);

The tests now fail with this error:

2023-10-17T15:01:20.678-05:00  INFO 18525 --- [           main] i.g.m.testcontainers.MicrocksContainer   : Caught a ConditionTimeoutException for test on http://host.testcontainers.internal:54320/api

And examining the docker container logs for the microcks-uber container that was used shows this log:

19:10:49.574 ERROR 1 --- [    task-3] i.g.m.u.p.PostmanTestStepsRunner         : IOException while executing request

org.apache.hc.client5.http.HttpHostConnectException: Connect to http://localhost:3000 [localhost/127.0.0.1] failed: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
	at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:547)
	at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:602)
	at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
	at java.base/java.net.Socket.connect(Socket.java:633)
	at org.apache.hc.client5.http.socket.PlainConnectionSocketFactory.lambda$connectSocket$0(PlainConnectionSocketFactory.java:85)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
	at org.apache.hc.client5.http.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:84)
	at org.apache.hc.client5.http.socket.ConnectionSocketFactory.connectSocket(ConnectionSocketFactory.java:113)
	at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:181)
	at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:447)
	at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:162)
	at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:172)
	at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:142)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
	at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
	at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
	at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:106)
	at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:93)
	at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
	at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66)
	at io.github.microcks.util.postman.PostmanTestStepsRunner.runTest(PostmanTestStepsRunner.java:192)
	at io.github.microcks.util.postman.PostmanTestStepsRunner.runTest(PostmanTestStepsRunner.java:58)
	at io.github.microcks.service.TestRunnerService.launchTestsInternal(TestRunnerService.java:199)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:756)
	at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
	at org.springframework.util.concurrent.FutureUtils.lambda$toSupplier$0(FutureUtils.java:74)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

My guess here is there is no postman executor runtime available when the test container is spinning up the microcks uber jar. Enabling postman was a separate step in my microcks installation, even as a docker extension, so my suspicion is postman is not enabled in the test container so the runner is failing. But that's just an educated guess.

Should this be possible? Is there something else I need to do?

[github-actions]

👋 @ericwyles

Welcome to the Microcks community! 💖

Thanks and congrats 🎉 for opening your first issue here! Be sure to follow the issue template or please update it accordingly.

📢 If you're using Microcks in your organization, please add your company name to this list. 🙏 It really helps the project to gain momentum and credibility. It's a small contribution back to the project with a big impact.

If you need to know why and how to add yourself to the list, please read the blog post "Join the Microcks Adopters list and Empower the vibrant open source Community 🙌"

Hope you have a great time there!

🌟 ~~~~~~~~~ 🌟

📢 If you like Microcks, please ⭐ star ⭐ our repo to support it!

🙏 It really helps the project to gain momentum and credibility. It's a small contribution back to the project with a big impact.

[lbroudoux]

Hi Eric,

Thanks for raising this issue. Your guess is correct: this feature is not implemented at the moment. Currently, we're just starting the main container of Microcks, so the POSTMAN and ASYNCAPI_SCHEMA test runners are unavailable. I hope I'll get some time to have a look at this in the forthcoming weeks.

[lbroudoux]

I created #24 to track this one

[ericwyles]

Hi Laurent, thanks for confirming, and glad to see I had the correct understanding. I'll close this one in favor of #24 since my question is answered.

[lbroudoux]

I started working on something today 😉 Would you be OK/have some time for a test ride even if not finalized? Just to be sure it would fit your needs...

[ericwyles]

Sure, I can give it a try. I'm just in experimental mode attempting to understand what all is possible from a shift-left kind of perspective, so looking to see how the behavioral conformance tests will work in that context. I'd be happy to take it for a test ride.

[lbroudoux]

Awesome! I've got something that's working fine on my test application. I'll push it to GitHub in a minute or two. However, you'll have to rebuild the lib by yourself to get a SNAPSHOT version of it.