Missing allow quoted data in DslCsvDataSet

[SeanLS]

Issue

When using CSV data which then contains JSON data and with a different type of delimiter (specifying \n or other types as a delimter) the JMeter DSL runner fails to run and reads the data as

Sample JSON CSV data

HeaderValue
[{ "keyValue1" : "dataValue1"}]
[{ "keyValue2" : "dataValue2"}]
[{ "keyValue3" : "dataValue3"}]

Url to pass values into:
https://fakeurl.com/testing/
Body: ${HeaderValue}

------ Current behaviour -----
Url: https://fakeurl.com/testing/
Body: '< EOF >'

----Expected behaviour----
Url: https://fakeurl.com/testing/
Body: [{ "keyValue1" : "dataValue1"}]

Url: https://fakeurl.com/testing/
Body: [{ "keyValue2" : "dataValue2"}]

etc....

Additional information

Option is avaliable in Jmeter. Please see screenshot below.

• https://jmeter.apache.org/images/screenshots/csvdatasetconfig.png

Please add the "Allow quoted data?" flag to the .Net configuration.

Notes

Love the library and the simplification for .Net yet with this feature missing it makes it very difficult to post preconfigured JSON test data to an endpoint. Other alternatives is having a "fixed" JSON payload in the body section, which does work, yet you loose the dynamic data insertion for a steadyload and rampup of changing datra for a given endpoint that may be configured to take multiple JSON inputs.

[rabelenda]

Hello, thank you for trying and asking about this and providing so well formatted report.

Currently the CsvDataSet element already sets allow quoted data to true. This was done to standarize the formatting of CSV files regardless if random order is required or not (random order uses Blazemeter plugin which contains Random CSV Data Set Config element, and such element does not provide a setting to disable such behavior).

In your case you can adapt the csv by enclosing with quotes and replacing each quote with double quotes (a standard CSV format for quoted values). Eg:

HeaderValue
"[{ ""keyValue1"" : ""dataValue1""}]
"[{ ""keyValue2"" : ""dataValue2""}]
"[{ ""keyValue3"" : ""dataValue3""}]

Do you think that with such workaround is still required to add support for the "unquoted" option? That would require extending the BM plugin to support the unquoted case)?

[SeanLS]

I will try your suggestion tomorrow and feedback to you. Adding this in the CSV docs section may help others as well should it work.

[SeanLS]

@rabelenda - While thinking about your suggestion, I think the Allow Quoted data should be configurable as your suggestion means that the payload is not conforming to the JSON standard. This requires re-working of any JSON data to have an invalid JSON payload which is means you are diverging from standards.

While I appreciate that you are trying to standardise things, sticking to the standards for a given language should be a priority.

Additional examples.... If you are posting XML, HTML, JS, etc directly you should have the ability to send the data verbatim for that given row in a CSV. Exactly like producer and consumer would act and form/produce/send the markup for each request.

(Extra info: Allow quoted data would be set to False in order to allow this to be processed and not the default of true as it is currently set to.)

[rabelenda]

Hello, thank you for your feedback.

Regarding the content not conforming with JSON standard, keep in mind that we are talking about a CSV file, and not a JSON file, so you should expect to do some transformations on your contents when copying one format inside another.

For instance, this JSON will not be parsed correctly regardless the allow quoted flag, and it will even not be parsed as you would expect for most of CSV parsers:

VAL
["val","test"]

Unless you escape it properly to be a CSV file you will get VAL=["val"

Same happens with other formats you describe as XML, HTML, JS, etc. CSV parsers usually split by the comma, unless the field starts with double quotes.

Other examples that will fail parsing:

{"field": "val, ue"}

<field tags="t1,t2">val</field>

<field>my, val</field>

You will always need to make sure that your "inner" format complies with CSV.

The easiest rule to make your format comply with CSV, is replacing each double quotes with two double quotes and add double quotes at the beginning and end of each value. This is the rule I applied to your example.