Constant timeout errors

[MinniArora]

Hi

For one of our services, suddenly we started getting the following timeout errors in gateway.

The timeout period of 1000ms has been exceeded while executing GET /v0/....
The timeout period of 1000ms has been exceeded while executing POST /v0/...

Just as we restarted the gateway, it started working.

Happened for only one service, the rest were working fine.

Please help figure out the cause behind.

Noticed a few 408 errors in the nginx logs of service but why would gateway hang due to that when we have connect & read timeouts in place (1s each)

[MinniArora]

The stacktrace below:

[vert.x-eventloop-thread-6] io.apiman.gateway.engine.beans.exceptions.Con nectorException: The timeout period of 1000ms has been exceeded while executing POST /v0/... for host ... io.apiman.gateway.engine.beans.exceptions.ConnectorException: The timeout period of 1000ms has been exceeded while executing POST /v0/.... for host ... at io.apiman.gateway.platforms.vertx3.connector.HttpConnector$ExceptionHandler.handle(HttpConnector.java:327) at io.apiman.gateway.platforms.vertx3.connector.HttpConnector$ExceptionHandler.handle(HttpConnector.java:324) at io.vertx.core.http.impl.HttpClientRequestBase.handleException(HttpClientRequestBase.java:136) at io.vertx.core.http.impl.HttpClientRequestImpl.handleException(HttpClientRequestImpl.java:51) at io.vertx.core.http.impl.HttpClientRequestBase.timeout(HttpClientRequestBase.java:183) at io.vertx.core.http.impl.HttpClientRequestBase.handleTimeout(HttpClientRequestBase.java:168) at io.vertx.core.http.impl.HttpClientRequestBase.lambda$setTimeout$0(HttpClientRequestBase.java:126) at io.vertx.core.impl.VertxImpl$InternalTimerHandler.handle(VertxImpl.java:784) at io.vertx.core.impl.VertxImpl$InternalTimerHandler.handle(VertxImpl.java:755) at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:324) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:445) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.TimeoutException: The timeout period of 1000ms has been exceeded while executing POST /v0/....

[MinniArora]

Is there a possibility of landing into problems with apiman if I send a POST request without Content-Length header.

[MinniArora]

Is anyone looking into it. We have checked that those 408 errors were caused by apiman. When we bypassed apiman, everything was working fine. Was happening for only few requests, not all.

[msavy]

Apologies for slow responses. I can only work on Apiman in my spare time.

Do the POST requests you describe actually take a very long period to execute? For example, sending a large payload?

[msavy]

Regarding headers -- content-length or transfer-encoding:chunked would be expected.

[MinniArora]

Thank you for the reply. No, the requests aren't huge, contain only 3 small fields. Although noticed that a few requests had Content-Length header missing. Not sure if exactly those requests landed into 408s.