Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proper CSV output for tools by escaping quotes #43

Open
tturocy opened this issue May 22, 2022 · 0 comments
Open

Proper CSV output for tools by escaping quotes #43

tturocy opened this issue May 22, 2022 · 0 comments

Comments

@tturocy
Copy link
Contributor

tturocy commented May 22, 2022

The DiamondWare file format does not permit embedded quotes (") in text strings. However, occasionally one does slip through in files.

Chadwick takes the position that output is undefined for invalid input files, although attempts are made to muddle through as best as possible. In particular, the parser does tolerate such embedded quotes.

However, when there are strings with embedded quotes, these are output by the command-line tools with the quotes not escaped. As a result, the output files are not valid in standard CSV dialects.

It would be nice to do something sensible that resulted in clean CSV files in this case. The most straightforward solution would be to escape any embedded double-quotes on output. However this would require changing code in a substantial number of places. Further, one of the objectives of the command-line tools is that they are fast - any change to implement escaping of embedded double-quotes should not have a significant overhead that would slow output down in the case in which there aren't any - because after all there "shouldn't" be any in well-formatted input files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant