Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSV.parse can't handle double double-quoted column at Ruby 2.6.0 #66

Closed
Watson1978 opened this issue Dec 27, 2018 · 5 comments
Closed

Comments

@Watson1978
Copy link
Contributor

Watson1978 commented Dec 27, 2018

At first, I don't know whether the following CSV is valid format.

a,""b""

Ruby 2.5's CSV library handles it well.
However, Ruby 2.6's shows different behavior.

Ruby 2.6

$ ruby -v t.rb
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-darwin18]
Traceback (most recent call last):
	5: from t.rb:7:in `<main>'
	4: from /Users/watson/.rbenv/versions/2.6.0/lib/ruby/2.6.0/csv.rb:683:in `parse'
	3: from /Users/watson/.rbenv/versions/2.6.0/lib/ruby/2.6.0/csv.rb:1180:in `read'
	2: from /Users/watson/.rbenv/versions/2.6.0/lib/ruby/2.6.0/csv.rb:1180:in `to_a'
	1: from /Users/watson/.rbenv/versions/2.6.0/lib/ruby/2.6.0/csv.rb:1171:in `each'
/Users/watson/.rbenv/versions/2.6.0/lib/ruby/2.6.0/csv/parser.rb:273:in `parse': Do not allow except col_sep_split_separator after quoted fields in line 1. (CSV::MalformedCSVError)

Ruby 2.5

$ ruby -v t.rb
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin18]
[["a", "\"b\""]]

Test code

require 'csv'

csv =<<CSV
a,""b""
CSV

p CSV.parse(csv)
@kou
Copy link
Member

kou commented Dec 27, 2018

It should be parsable.

@kou
Copy link
Member

kou commented Dec 29, 2018

Ah, this isn't a valid CSV. It should be quoted:

a,"""b"""

@kou
Copy link
Member

kou commented Dec 29, 2018

This is a bug of csv in Ruby 2.5.

@kou
Copy link
Member

kou commented Dec 29, 2018

But I don't like backward incompatibility.
I'll consider how to handle this.

@kou kou closed this as completed in 516f96e Jan 5, 2019
@kou
Copy link
Member

kou commented Jan 5, 2019

I've added liberal_parsing: {double_quote_outside_quote: true} option. The following code works on both Ruby 2.5 and Ruby 2.6 + csv master:

require 'csv'

csv =<<CSV
a,""b""
CSV

p CSV.parse(csv, liberal_parsing: {double_quote_outside_quote: true})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants