Skip to content


Subversion checkout URL

You can clone with
Download ZIP


Log more request informations #446

slandelle opened this Issue · 19 comments

2 participants


such as:

  • request uuid
  • url or even (Request -> String) hook
  • response status or even (Response -> String) hook



Hi Stephane,
I am thinking about working on this issue.

Do you still think that enabling user-specified com.ning.http.client.{Response, Request} data extraction functions is a good / reasonable idea?

If so, what do you think about the following changes:

  • add options to GatlingConfiguration with data-extraction functions for:
    • (Request -> List[String])
    • (Response -> List[String])
  • add Request and Response parameters to DataWriter.logRequest and RequestRecord
  • modify FileDataWriter to invoke user-specified data-extraction functions when building a log line
  • modify FileDataReader to accept variable-length ACTION records

I examined the *DataWriter classes a bit, and I think it would be preferable for the data extraction functions to return a List of Strings corresponding to fields that should be logged. This keeps those data extraction functions from having to know the exact format in which the data will be logged, particularly the field separator. This should keep users from having to know how FileDataReader works, as well.

My main concerns with this feature are:
1. keeping Request / Response objects around longer; do you think additional logging latency will affect simulation throughput significantly?
2. exposing Request / Response to user-provided code where one can do expensive actions like response.getResponseBody(); do you think Request / Response should be wrapped and only expose "safe" operations?
3. preserving FileDataReader's ability to read ACTION records when they can vary in length; solvable, though annoying



Hi Stephen,

Great news, help would be much appreciated!

First, here are my concerns:

  • GatlingConfiguration, FileDataWriter, FileDataReader and RequestRecord are protocol agnostic, and must remain this way is we wand to be able to implement other protocols some day. So we can't have them depend on Request and Response that are AHC classes.
  • Passing Request and Response instances over would increase their lifespan and their chance of surviving minor collections, hence increasing full collections.
  • Response instances may not be complete, as we don't collect BodyPart in GatlingAsyncHandler if there's no check defined on the body content.

So here's the solution I had in mind:

  • have (Request -> List[String]) and (Response -> List[String])
  • have those functions be defined on HttpProtocolConfiguration
  • have GatlingAsyncHandlerActor perform them so that Request and Response are quickly discarded
  • add an optional List[String] parameter on RequestRecord
  • have FileDataWriter prepend those extra fields
  • have FileDataReader support varying length records (implies removing pattern matching) so that additional fields are discarded
  • issue some warnings on those functions usage: you're on your own for not blowing the engine and analyzing the extra data.

I still havent puzzled out how this would impact other DataWriter implementation that we might have in the future, such as database persistence. Schema based database would probably not support such a feature.




That makes pretty good sense to me, particularly defining extraction functions on HttpProtocolConfiguration.

I'm not sure about prepend versus append of the data, but I can experiment with both. When I think about using the data via cut/awk/perl, it makes more sense (to me) to find standard data at the beginning of the line and custom data at the end.

I agree that custom data makes implementing some DataWriter implementations efficiently more difficult, particularly RDBMS. However, I think it is reasonable for different DataWriters to record varying amounts of data. This is already true for Console and File DataWriter.

I'm not sure it makes sense to record simulation logs to an RDBMS in real-time. It seems like this could/should be done in bulk after the simulation is complete for performance reasons, probably with the database's native tools for loading data.

Gatling (conveniently) outputs the simulation log as a tab-delimited file, so loading the data into another system for analysis should be pretty trivial. This is, after all, what I plan to do with custom data that is written into simulation.log:
1. annotate request data with major attributes
2. load action data into R
3. perform analysis

If one needs to see simulation results in another system in real-time, then I'd suggest:

  • the system under test should throw off appropriate telemetry to its regular monitoring systems, e.g. graphite, cacti, New Relic
  • implementing a custom DataWriter

Ah ah, sorry, I meant append, not preprend.

I'll try to reply in length tomorrow.


Hello Stephen,

Sorry for the late reply.
@nire will be working on #542 so we'll have a complete feature there. So you can start working on this.

Let's move the discussion about database persistence into the dedicated issue: #62
The discussion about server monitoring integration is related to #359, but I'm also think of opening a new one.




Thanks for the update. I've read the referenced issues and will add thoughts, as relevant.

Just as an fyi...I probably won't get a chance to work on this issue until the weekend.


Thanks a lot for your help!


@skuenzli Hi Stephen! Do you know when you'll be done with this one? I don't want to rush you, just to know what I can put in 1.2.2 and when I can release it.


Hi Stephane,
I would not count on this for 1.2.2. It is close, but I haven't tested it enough and I think I ran into an issue with the DataReader. I plan to work some more on this tonight and will post an update then.


I did not get a chance to work on this issue tonight. I will push progress nightly (for me) to and issue a pull request when it is ready.

Feel free to start reviewing the change, if you like.


OK, no pro.


I've submitted #565


I've added some built-ins.
I wonder if we should let httpProtocolConfig have varargs parameters that we would combine into a composite, so that people can use multiple built-ins.


yes, I absolutely think this is valuable and was going to provide some in a separate change.




Just added some more built-ins in c76e273.
Regarding composite extractor, let's do it when needed/requested.

@slandelle slandelle closed this

Will have to add composite some day though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.