Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSV BOM-strip clashes with double-quoted header line: "unwrapped double quote at line 0" #161

Closed
johnkerl opened this issue Dec 20, 2017 · 1 comment
Labels

Comments

@johnkerl
Copy link
Owner

https://github.com/johnkerl/miller/releases/tag/v5.2.0 introduced stripping for the CSV BOM (leading 0xef 0xbb 0xbf). However, it incorrectly handles the case when the header line itself contains double quotes:

$ cat 1.csv
a,b,c
"1","2","3"
"1","2","3"

$ cat 2.csv
"a","b","c"
"1","2","3"
"1","2","3"
$ hex 1.csv
00000000: ef bb bf 61  2c 62 2c 63  0a 22 31 22  2c 22 32 22 |...a,b,c."1","2"|
00000010: 2c 22 33 22  0a 22 31 22  2c 22 32 22  2c 22 33 22 |,"3"."1","2","3"|
00000020: 0a                                                 |.|
$ hex 2.csv
00000000: ef bb bf 22  61 22 2c 22  62 22 2c 22  63 22 0a 22 |..."a","b","c"."|
00000010: 31 22 2c 22  32 22 2c 22  33 22 0a 22  31 22 2c 22 |1","2","3"."1","|
00000020: 32 22 2c 22  33 22 0a                              |2","3".|
$ mlr --csv cat 1.csv
a,b,c
1,2,3
1,2,3
$ mlr --csv cat 2.csv
mlr: syntax error: unwrapped double quote at line 0.
@johnkerl
Copy link
Owner Author

johnkerl commented Jan 1, 2018

Fixed by 7d19697

@johnkerl johnkerl closed this as completed Jan 1, 2018
@johnkerl johnkerl removed the active label Sep 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant