Can't make DataDriver read variables from CSV file with more than one column

[immo-huneke-zuhlke]

I have found that if I create a CSV file with a single column, headed e.g. ${key_id}, then it works like a charm. But if the CSV file contains multiple columns separated by commas, the whole line is read as a single value, leaving the ${key_id} undefined. I have even tried explicitly specifying the delimiter as a comma. The documentation (https://pypi.org/project/robotframework-datadriver/#structure-of-data-file) states that the first column must be named *** Test Cases *** and this is what I did, but I wonder whether that is actually what is confusing DataDriver? That heading doesn't look like a Robot variable.

[Snooz82]

Can you please provide an example of your csv file and your robot code to reproduce?

If you want to use comma as delimiter you shall change the dialect=userdefined and set the separator. If the dialect is not set to userdefined, the other arguments are ignored.

You could also just use dialect=excel.

Cheers

[immo-huneke-zuhlke]

Thanks for the clarification. Please update the documentation accordingly. At the moment, it looks as if the standard CSV file created by default using csv.DictWriter should be parsed successfully. It explicitly states that the default field separator is a comma, but I had overlooked the information that this only applies in certain non-default dialects.

[Snooz82]

Can you please point me where you see the gap in docs?

It is written two times that Excel-EU with semicolon is the default:
https://github.com/Snooz82/robotframework-datadriver#file-encoding-and-csv-dialect

The default is Excel-EU which is a semicolon separated file.

https://github.com/Snooz82/robotframework-datadriver#defaults

Kind regards

[immo-huneke-zuhlke]

You are right, the information is there - but I plain failed to see it until you pointed it out. I think it might be easier to spot if the default defaults were presented before the excel and unix ones.

As a matter of fact, what I was really trying to achieve was to read something from AWS (in this case, all customer-managed KMS master keys) and then apply the test case to all of them. Reading a CSV file wasn't part of the plan, but we were forced down this route by the way Robot Framework does things. We now include a custom Python library whose constructor function queries the AWS account and writes a CSV file with the output, before including the DataDriver library that reads the file back in. If there were a way to pass the data directly to DataDriver in a list variable, that would fulfil the requirement much more neatly.

So we were forced to find a way to produce a CSV file. Without spending hours and hours reading the documentation of the csv library, it didn't occur to us that it would output a different dialect of CSV than the one assumed by DataDriver. In fact none of the dialects that DataDriver knows appears to correspond exactly to the format output by csv.DictWriter - minimal quoting, comma separator, UTF encoding, OS-dependent newline. See https://docs.python.org/3/library/csv.html for more information.

I'm happy for you to close this issue if you like.

[Snooz82]

But the csv dialect `excel` is the default of csv.writer, isn't ist? I decided to not use this one, because Excel in Europe creates a different format. It is semicolon separated, because european countries uses dot as thousands separator and comma as decimal separator. Because i am in Europe and Finnland is also, i decided to take this default.

…

Am 07.07.2020 um 10:23 schrieb Immo Hüneke ***@***.***>:  You are right, the information is there - but I plain failed to see it until you pointed it out. I think it might be easier to spot if the default defaults were presented before the excel and unix ones. As a matter of fact, what I was really trying to achieve was to read something from AWS (in this case, all customer-managed KMS master keys) and then apply the test case to all of them. Reading a CSV file wasn't part of the plan, but we were forced down this route by the way Robot Framework does things. We now include a custom Python library whose constructor function queries the AWS account and writes a CSV file with the output, before including the DataDriver library that reads the file back in. If there were a way to pass the data directly to DataDriver in a list variable, that would fulfil the requirement much more neatly. So we were forced to find a way to produce a CSV file. Without spending hours and hours reading the documentation of the csv library, it didn't occur to us that it would output a different dialect of CSV than the one assumed by DataDriver. In fact none of the dialects that DataDriver knows appears to correspond exactly to the format output by csv.DictWriter - minimal quoting, comma separator, UTF encoding, OS-dependent newline. See https://docs.python.org/3/library/csv.html for more information. I'm happy for you to close this issue if you like. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.