Feeders

krishnenc edited this page Jun 12, 2012 · 14 revisions

This page references all the feeders available in Gatling.

Understanding Feeders #

Why use feeders? #

If you want to simulate 100 users, each with its own credentials for example, you need to inject data in the simulation for each user. Another similar use case could be requests consuming data to simulate the creation of records in the database via the tested application (new products, carts, etc.)

This is what feeders are made for. They are used in the simulation thanks to the method feed. Each time the feed method is called, a record from the feeder is injected in the session (overriding the previous one if any).

Depending on where you place the feed method, the data injected in the session won't be the same. Imagine the following scenario extract:

val userCredentials = csv("user_credentials.csv").queue
val recordsInformation = csv("records_information.csv").queue

val scn = scenario("My Scenario")
  .feed(userCredentials)
  .loop(
    chain
      .feed(recordsInformation)
      .http( ...)
  ).times(20)
  // End of Scenario

List(
  scn.configure.users(100)
)

At the End of Scenario line, the data contained in the session for one user is half predictible:

  • The data from the userCredentials feeder will match the userId of the session (the first launched scenario will get the first value from the file and so on)
  • The data from the recordsInformation feeder could be anything. Indeed, each user will get values from the feeder simultaneously (because of the loop). Therefore, one cannot predict what records will be created by which user.

Note: If the first feed method was called after the loop, there would be no guarantees that the 20th user would have taken the 20th line of the feeder.

Do not forget that feeders are accessed concurrently during the simulation.

What compose a feeder? #

From a user point of view, feeders are made of two parts: the source and the strategy. In the following example, the source is csv("my_feeder.csv") and the strategy is queue.

scenario("My Scenario")
  .feed( csv("my_feeder.csv").queue )

The source is the place from where data will be retrieved; the strategy defines how these data should be used.

Feeder Sources #

Separated Values Files #

Since v1.0

Using CSV files as feeders

Gatling Feeders provide support for separated values files. There are three types of separated values files supported:

  • Comma Separated Values (CSV).
  • Semicolon Separated Values (SSV).
  • Tabulation Separated Values (TSV).

Do declare a feeder with a separated values file as source, use the following methods:

csv( filename: String ) // If the values are _comma_ separated
ssv( filename: String ) // If the values are _semi colon_ separated
tsv( filename: String ) // If the values are _tabulation_ separated

Note: The separated values file must be in the user-files/data folder.

The first line of the file must contain the label of the values:

username,password
john,smith21
john,doe43

These labels will be used as keys to the values stored in the session, for example ${username}.

Using Escaping Character

Since v1.1

If you choose to create a CSV file but some values contain the , character, you can escape them with the \ character: Gatling uses OpenCSV to parse separated values files.

If you want to use another escaping character, you can set one while declaring your feeder:

val userCredentials = csv("user_credentials.csv", '#')

JDBC #

Since v1.1

Gatling feeders provide support for JDBC data. You can ask a feeder to get values from a database instead of a file. The method signatures are all the same for every supported database:

<db_code>Feeder( databaseURL: String, username: String, password: String, sql: String)

The databaseURL must be a JDBC URL (ie: jdbc:postgresql:gatling), the username and password are the credentials to access the database and sql is the request that will get the values needed. Here are the supported Databases along with the method to use and the JDBC connector to use

Database Method to use
DB2 db2Feeder
HSQLDB hsqldbFeeder
H2 h2Feeder
SQL Server mssqlFeeder
MySQL mysqlFeeder
Oracle oracleFeeder
PostgreSQL postgresqlFeeder
Sybase sybaseFeeder

Note: Do not forget to add the required connector jar in the classpath (lib/ folder in the bundle)

Redis #

Since v1.2.2

Redis can be used as a feeder for gatling. The dynamic data can be loaded on a Redis List , popped out for each http request and the value placed in a session variable. Since v2.1.14 Redis supports mass insertion of data from a file. It is possible to load millions of keys in a few seconds in Redis and Gatling will read them off memory directly.

For i.e : A simple Scala function to generate a file with 1 million different urls ready to be loaded in a Redis list named URLS :

  import com.excilys.ebi.gatling.core.feeder.redis.util._

  def generateOneMillionUrls() = {
    val fileLocation = "/tmp/loadtest.txt"
    val writer = new PrintWriter(new File(fileLocation))
    try {
      for (i <- 0 to 1000000) {
        val url = "test?id=" + i.toString()
        writer.write(generateRedisProtocol("LPUSH", "URLS", url))
      }
    } finally {
      writer.close()
    }
  }

The urls can then be loaded in Redis using the following command :

cat /tmp/loadtest.txt | redis-cli --pipe

An example simulation to use this redis feeder can be viewed here

At the moment only queue feeder strategy is supported for Redis.

Feeder Strategies #

Once the values have been retrieved from the source, the feeder can be consumed. The way it is consumed depends on the strategy used.

Queue strategy #

Since v1.0

The default strategy (ie: if you don't specify one) is the queue strategy. Each time feed is called, the first record of the feeder is removed from the queue and injected into the session.

Be careful while using this strategy, the feeder source must contain enough records for the simulation; if not, the simulation will stop when the queue is empty.

Example:

csv("user_credentials.csv").queue
csv("user_credentials.csv")        // It is the same as above since queue is the default strategy

Random strategy #

Since v1.0

For this strategy, each time feed is called, a random record is picked inside the feeder and injected into the session.

The records are not removed from the feeder when injected.

Circular strategy #

Since v1.1

The circular strategy will loop on the values contained in the feeder. If there are N values in the feeder and the feed method has been called N+1 times, then, the record #1 will be injected.

Custom Feeders #

Coming soon...