Failing simulations are blocked instead of terminated with gatling 3.10.1

[nvervelle]

Following the problem with gatling 3.10.0 (see #4494), I tried upgrading from gatling 3.9.5.6 to gatling 3.10.1.

It seems I have another problem though : I replaced calls CoreDsl.exitBlockOnFail(...) by CoreDsl.exitBlockOnFail().on(...) to take into account the breaking change in gatling DSL, but now errors fail to exit the simulation, the simulation seems to be stucked instead of finishing

I see logs stating that a request has failed, but nothing more happens to the simulation :
11:37:09.847 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'applicant-submit-challenge' failed for user 2: status.find.is(200), but actually found 415

Any idea what can be causing such problem ?

[slandelle]

Please provide a reproducer.

The following simulation works as expected for me: req2 fails so req3 doesn't get executed).

import io.gatling.javaapi.core.*;
import io.gatling.javaapi.http.*;

import static io.gatling.javaapi.core.CoreDsl.*;
import static io.gatling.javaapi.http.HttpDsl.*;

public class ExitSimulation extends Simulation {

    HttpProtocolBuilder httpProtocol =
        http.baseUrl("https://computer-database.gatling.io")
            .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
            .acceptLanguageHeader("en-US,en;q=0.5")
            .acceptEncodingHeader("gzip, deflate")
            .userAgentHeader(
                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0"
            );

    ScenarioBuilder users = scenario("Users")
            .exitBlockOnFail().on(
                    http("req1").get("/"),
                    http("req2").get("/").check(status().is(999)),
                    http("req3").get("/")
            );

    {
        setUp(
            users.injectOpen(atOnceUsers(1))
        ).protocols(httpProtocol);
    }
}

2023-12-19 11:57:23                                           0s elapsed
---- Requests ------------------------------------------------------------------
> Global                                                   (OK=3      KO=1     )
> req1                                                     (OK=1      KO=0     )
> req1 Redirect 1                                          (OK=1      KO=0     )
> req2                                                     (OK=1      KO=0     )
> req2 Redirect 1                                          (OK=0      KO=1     )
---- Errors --------------------------------------------------------------------
> status.find.is(999), but actually found 200                         1 (100,0%)

[nvervelle]

I will try to find a way to reproduce it with a simple example.

What you describe is what I experience with 3.9.5.6, but with 3.9.10.1, I have the following difference :

• The error is only visible in the logs as DEBUG, but not in the textual report (there's no request in KO status, no - Errors ----...)
• The logs only contain one line for the error, not the detailed version (with the Request, Session, HTTP request and HTTP response parts)
• The simulation simply doesn't do anything more and never stops.

[nvervelle]

Hi @slandelle , I did some tests with 3.9.10.1 and 3.9.5.6 with a simplified example based on yours. I notice a really strange difference in the console output between the two versions :

With 3.9.5.6, the logs shows the body of the HTTP response :

12:38:44.766 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'req2 Redirect 1' failed for user 1: status.find.is(999), but actually found 200
12:38:44.776 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - 
>>>>>>>>>>>>>>>>>>>>>>>>>>
Request:
req2 Redirect 1: KO status.find.is(999), but actually found 200
=========================
Session:
Session(Users,1,HashMap(gatling.http.cache.baseUrl -> https://computer-database.gatling.io, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@cbcf4ab, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@3f15b0db, gatling.http.referer -> https://computer-database.gatling.io/computers, 3d865a5c-99ef-4f85-955d-0cd4d73e727f -> 0),OK,List(TryMaxBlock(3d865a5c-99ef-4f85-955d-0cd4d73e727f,tryMax-1-inner,KO)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$563/0x00007f16d84fbe68@1ef1d272,io.netty.channel.epoll.EpollEventLoop@7bfc3126)
=========================
HTTP request:
GET https://computer-database.gatling.io/computers
headers:
        accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
        accept-language: en-US,en;q=0.5
        accept-encoding: gzip, deflate
        user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0
        host: computer-database.gatling.io
=========================
HTTP response:
status:
        200 OK
headers:
        Date: Tue, 19 Dec 2023 11:38:44 GMT
        Content-Type: text/html; charset=UTF-8
        Transfer-Encoding: chunked
        Connection: keep-alive
        Server: Finch
        content-encoding: gzip

body:
<html><head>...</head><body>...</body></html>
<<<<<<<<<<<<<<<<<<<<<<<<<

With 3.10.0.1, the logs show a different error with a StringIndexOutOfBoundsException, as if gatling failed after the request failed :

12:45:06.262 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'req2 Redirect 1' failed for user 1: status.find.is(999), but actually found 200
12:45:06.273 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - 
>>>>>>>>>>>>>>>>>>>>>>>>>>
Request:
req2 Redirect 1: KO status.find.is(999), but actually found 200
=========================
Session:
Session(Users,1,HashMap(gatling.http.cache.baseUrl -> https://computer-database.gatling.io, e2b0dd1b-c91e-43c2-8aa7-f595c435b8b9 -> 0, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@690e039a, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@27f22535, gatling.http.referer -> https://computer-database.gatling.io/computers),OK,List(TryMaxBlock(e2b0dd1b-c91e-43c2-8aa7-f595c435b8b9,tryMax-1-inner,KO)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$642/0x00007f9c04525598@5e21417c,io.netty.channel.epoll.EpollEventLoop@36a7abe1)
=========================
HTTP request:
GET https://computer-database.gatling.io/computers
headers:
        accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
        accept-language: en-US,en;q=0.5
        accept-encoding: gzip, deflate
        user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0
        host: computer-database.gatling.io
=========================
HTTP response:
version:
        HTTP/1.1
status:
        200 OK
headers:
        Date: Tue, 19 Dec 2023 11:45:06 GMT
        Content-Type: text/html; charset=UTF-8
        Transfer-Encoding: chunked
        Connection: keep-alive
        Server: Finch
        content-encoding: gzip

body:
Could not decode response body: j.l.StringIndexOutOfBoundsException: Range [0, 500000) out of bounds for length 2455
<<<<<<<<<<<<<<<<<<<<<<<<<

Simplified simulation used :

import io.gatling.javaapi.core.CoreDsl;
import io.gatling.javaapi.core.ScenarioBuilder;
import io.gatling.javaapi.core.Simulation;
import io.gatling.javaapi.http.HttpDsl;
import io.gatling.javaapi.http.HttpProtocolBuilder;

@SuppressWarnings("PMD.NonStaticInitializer")
public class ExitSimulation extends Simulation {

  HttpProtocolBuilder httpProtocol =
      HttpDsl.http
          .baseUrl("https://computer-database.gatling.io")
          .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
          .acceptLanguageHeader("en-US,en;q=0.5")
          .acceptEncodingHeader("gzip, deflate")
          .userAgentHeader(
              "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0");

  ScenarioBuilder users =
      CoreDsl.scenario("Users")
          .exitBlockOnFail()
          .on(CoreDsl.exec(HttpDsl.http("req2").get("/").check(HttpDsl.status().is(999))));

  {
    setUp(users.injectOpen(CoreDsl.atOnceUsers(1))).protocols(httpProtocol);
  }
}

[slandelle]

@nvervelle Your help is much appreciated, but really, this is a but tracker.
Please stick to 1 ticket per issue. Different issues have different causes and will get fixed by different commits, possibly not shipped in the same release.

I've opened a ticket for the other StringIndexOutOfBoundsException issue: #4496 and already fixed it.

Waiting for your reproducer about exitBlockOnFail.

[slandelle]

Can you reproduce without enable DEBUG/TRACE logging?

[nvervelle]

I'm still trying to reproduce the problem with a simple example, but still no success... I'm doing other attempts and will keep you posted.

I posted the problem with the exception here, because I thought it may have a link to the problem I see. Apparently no, so I will try further to produce a simple reproducer.

[nvervelle]

My problem is related, at least partially, to #4497 : the 415 errors I was receiving were due to the HTTP Content-Type passed by the new version of gatling. With 3.10.2, the requests don't fail, so my simulations are working, but I have to check what happens if a request fail. I will keep you posted.

[nvervelle]

I tried again running the simulations in a real load testing session, with failures in the requests : I don't see the problem anymore. Maybe the various fixes you did were sufficient to fix my problem ?

I suggest to close this issue, and I will post again if I have again this problem.

[slandelle]

OK. Thanks for your feedback!