Random number of entries in payload

[tomhelmer]

Hi
I would be nice if it was possible to randomize the number of entries in the payload. An example could be orders with a random number of order lines.

Let's say I have some orders:

   payload:
      - path: "orders.csv"
        fields:
          - "order_number"
          - "first_name"
          - "last_name"

.. and some lines:

   payload:
      - path: "order-lines.csv"
        fields:
          - "product_number"
          - "price"
          - "quantity"

Then it would be nice to be able to randomize the number of order lines, eg. like this:

json:
   order_number: "{{ order_number }}"
   first_name:   "{{ first_name }}"
   last_name:    "{{ last_name }}"
   lines:
     random-array:
       min: 1
       max: 5
       product_number: "{{ product_number }}"
       price:          "{{ price }}"
       quantity:       "{{ quantity }}"          

[erasmussenBHTP]

I would also find this very useful as I would like 1-4 items in a request. My workaround for this is to repeat the payload 4 times and use variable names with the index appended (e.g. val0, val1, val2). I have a beforeRequest javascript function that generates a random number, and I push that many items into the array while referencing the appropriate index for each item.

[hassy]

Not sure I understand fully - would you like to be able to grab several rows from a CSV file and have the values assigned to variables but those variables become arrays now? e.g.:

   payload:
      - path: "order-lines.csv"
        fields:
          - "product_number"
          - "quantity"
        getMultipleRows:
          number: 2 # we grab 2 rows at a time, or we could use "min" and "max" for a randomized number of rows

And then in a request:

- post:
    url: /some/endpoint
    json:
      lines:
        - quantity: "{{ quantity[0] }}"
          product_number: "{{ product_number[0] }}"
        - quantity: "{{ quantity[1] }}"
          product_number: "{{ product_number[1] }}"

Something like that?

[tomhelmer]

no, "random-array" should be a new type with min/max as options.
so lines would end up as an array of lines but it should be handled by the "random-array" entry.
In your example I would have to duplicate each line entry and could not have a random number of lines.

[hassy]

I see. For that to work we'd need to introduce custom YAML tags for those nodes and/or built-in functions similar to how AWS CloudFormation does it, e.g. with Fn::Base64 this example: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-general.html#scenario-userdata-base64

[erasmussenBHTP]

Here is what I'm doing for my workaround. There is a possibility of duplicates with this approach. I'm working with a large input CSV and my system needs to be able to handle duplicates.

payload:
    -
      path: 'flights.csv'
      skipHeader: true
      fields:
        - 'airline0'
        - 'flightNumber0'
        - 'depAirport0'
        - 'arrAirport0'
    -
      path: 'flights.csv'
      skipHeader: true
      fields:
        - 'airline1'
        - 'flightNumber1'
        - 'depAirport1'
        - 'arrAirport1'
    -
      path: 'flights.csv'
      skipHeader: true
      fields:
        - 'airline2'
        - 'flightNumber2'
        - 'depAirport2'
        - 'arrAirport2'
    -
      path: 'flights.csv'
      skipHeader: true
      fields:
        - 'airline3'
        - 'flightNumber3'
        - 'depAirport3'
        - 'arrAirport3'

My beforeRequest javascript does the following

function setJSONBody(requestParams, context, ee, next) {
    const flightCount = Math.floor(Math.random() * (3)) + 1;
    for (let i = 0; i < flightCount; i++) {
        requestParams.json.flights.push({
            'flightDesignator': `{{ airline${i} }}{{ flightNumber${i} }}`,
            'departureAirportCode': `{{ depAirport${i} }}`,
            'arrivalAirportCode': `{{ arrAirport${i} }}`,
            'departureDate': moment().add(i > 1 ? 4 : 3, 'days').format('YYYY-MM-DD')
        });
    }
    
    return next();
}

[hassy]

Thanks for sharing that @erasmussenBHTP, that's very helpful! We'll need to size up the effort of adding support for tags/functions that operate on the node itself.