Change API parameters from comma-separated to arrays

[martinburchell]

According to the REDCap developers multiple API parameters should not be comma-separated and should be represented as arrays instead. So I've changed PyCap to work in this way.

Background:
I noticed that when I passed an invalid form to Project.export_records(), I got all records for all forms. If I passed valid forms I got the records I expected for just those forms.

In the REDCap API playground (at least for version 9.5.13 which we're using) the example curl commands do not have the parameters comma-separated. Experimenting with curl:

  /usr/bin/curl -H Content-Type: application/x-www-form-urlencoded -H Accept: application/json -X POST -d token=MYTOKEN&content=record&format=json&type=flat&fields[0]=record_id&forms[0]=bmi&forms[1]=patient_record&forms[2]=flooble&rawOrLabel=raw&rawOrLabelHeaders=raw&exportCheckboxLabel=false&exportSurveyFields=false&exportDataAccessGroups=false&returnFormat=json https://myserver/api/
         
   /usr/bin/curl -H Content-Type: application/x-www-form-urlencoded -H Accept: application/json -X POST -d token=MYTOKEN&content=record&format=json&type=flat&fields[0]=record_id&forms=bmi,patient_record,flooble&rawOrLabel=raw&rawOrLabelHeaders=raw&exportCheckboxLabel=false&exportSurveyFields=false&exportDataAccessGroups=false&returnFormat=json https://myserver/api/

In the first case I got:

{"error":"The following values in the parameter \"forms\" are not valid: 'flooble'"}

In the second case it's as if the forms parameter was omitted and I get results for all forms. If I remove 'flooble' from the list of forms so the parameter is

forms=bmi,patient_record

I get results for just those two forms

I asked the question in the (members only) forum:
https://community.projectredcap.org/questions/77217/export-records-api-silently-ignores-invalid-form-w.html

and it seems that even though comma separated values is working in some cases, the REDCap developers say that the correct way to call the API is with arrays.

So I've changed export_records(), export_metadata() and export_fem(). Oddly export_records() is the only call that returns a BadRequest if it gets and invalid form. So I'm also handling that.

[sburns]

Excellent work and write-up! Do we know how far back this is compatible API-wise?

[martinburchell]

Rob says "not sure, but i do know that that particular API code in REDCap that deals with the API request parameters has not changed in the past 3 years, to my knowledge."

[pwildenhain]

Is there anything preventing this from being merged?

[martinburchell]

@pwildenhain not that I'm aware of. My understanding is @sburns is continuing to maintain this as a favour in his spare time and he doesn't have a lot of that.

[pwildenhain]

Great! I've recently been made a collaborator on this repo, so I can help further develop and maintain this project. Since @sburns seemed primarily concerned with backwards-compatability, and you double-checked that with the developers, then I would say we're good to go!