Not quoted empty strings are now null, implement a trim-blanks options.

[dimitri]

When loading data from CSV file into a database (PostgreSQL in my case), it's important to be able to distinguish in between empty string and NULL values. This patch considers non-quoted empty strings as NIL.

Also, it's quite important to be able to retain blanks found around non-quoted values, so I also implemented a new option trim-blanks that works as shown here:

CL-USER> (cl-csv:read-csv "'a', 'b',,'',   ,  plop  ,'c'" :quote #\' :trim-blanks nil)
(("a" "b" NIL "" "   " "  plop  " "c"))
CL-USER> (cl-csv:read-csv "'a', 'b',,'',   ,  plop  ,'c'" :quote #\' :trim-blanks t)
(("a" "b" NIL "" NIL "plop" "c"))

I've made the previous behaviour the default, that is trim-blanks defaults to t. I don' t think the unquoted empty string to NIL behavior needs an option, though.

[bobbysmith007]

I am sorry to have missed this... I will look at this post-haste

[bobbysmith007]

Looking at the whitespace tests, this seems to be a bug you have discovered rather than a feature that needs to be added. We already do not trim quoted values, but unfortunately this was missed on white-space only strings.

I think I will leave it returning nil for empty quoted values, but otherwise we should be returning the spaces always. (It is relatively easy to trim after the fact and the semantics are easier to understand if "everything in quotes is returned").

[bobbysmith007]

Never mind I miss read, apologies, applying now

[bobbysmith007]

This should be fixed in 982931d Note that I changed the var/arg to empty-string-is-nil because trimming usually involves removing characters while this arg controls the disposition of the empty string.

[bobbysmith007]

Apologies again for missing this, github didnt notify me and It was not on my github dashboard (though I am unsure why).

[dimitri]

In fact, if I'm reading your commit correctly, what you did is not helping my use case at all. I need to be able to distinguish in between those two cases, which have different representations in databases:

• unknown data (NULL)
• data is known to be an empty string (empty string)

So in my pull request what I had done is to provide a way to distinguish between no value between separators (raises nil) and an explicit empty string (raises "").

To be able to do that, you also have to consider whether a, ,b and a,,b are the same thing or not, and I added an option to let the user choose about that. I could use your implementation of it provided that I'm still able to distinguish in between a,,b and a,"",b too, which doesn't seem to be the case yet.

[bobbysmith007]

18dadc8

I just committed a patch that I think covers all the circumstances discretely. Please let me know if this now meets your needs

[bobbysmith007]

As a note there was a bug in the tests related to lisp-unit2 that I have fixed.

[dimitri]

Looks pretty good, thanks! will integrate tomorrow, I guess.