Searching/sorting on fields with leading spaces fail (Grid or API)

[aliceh75]

How to reproduce

Upload the following CSV to the Datapusher:

my field, another field, third field
value 1, value 2, value 3

That this will create fields named " another field" and " third field".

In the grid view, it is not possible to sort or search on either of those fields. I assume using the API it would also not be possible to sort on those fields (though search should work).

The problem

Parts of the code does not trim field names (for instance the datapusher, and the datastore_create API); however other parts of the code does trim field names (for instance the code that splits a sort string: " another field, third field" becomes ['another field', 'third field'])

A solution

Given that there is no practical use for having field names that contain leading or ending spaces, I suggest the datastore API should trim field names.

[aliceh75]

See PR #1971 for CKAN, and PR ckan/datapusher#47 for the datapusher.

[aliceh75]

Now that #1971 has been merged, it will not be possible to insert new data with fields that start/end with whitespaces. So this issue now only affects already uploaded data.

[wardi]

I think it's reasonable to suggest users fix and re-upload their data in this case.