Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

read_csv: when the first section of a cell is quoted, the row is not parsed correctly #814

Closed
gregrs-uk opened this issue Mar 14, 2018 · 5 comments

Comments

@gregrs-uk
Copy link
Contributor

@gregrs-uk gregrs-uk commented Mar 14, 2018

In the examples below, cell 1 is always entirely unquoted, cell 3 is always entirely quoted and cell 2 has a particular section quoted.

If cell 2 has a quoted section at the end or in the middle, the row is parsed correctly.

library(readr)

read_csv('1,2,3\ncell 1,cell "2","cell 3"\n')

#> # A tibble: 1 x 3
#>   `1`    `2`          `3`
#>   <chr>  <chr>        <chr>
#> 1 cell 1 "cell \"2\"" cell 3

If cell 2 has a quoted section at the start, the row is not parsed correctly.

read_csv('1,2,3\ncell 1,"cell" 2,"cell 3"\n')

#> Warning in rbind(names(probs), probs_f): number of columns of result is not
#> a multiple of vector length (arg 1)
#> Warning: 3 parsing failures.
#> row # A tibble: 3 x 5 col     row col   expected           actual    file         expected   <int> <chr> <chr>              <chr>     <chr>        actual 1     1 2     delimiter or quote " "       literal data file 2     1 2     delimiter or quote c         literal data row 3     1 <NA>  3 columns          2 columns literal data
#> # A tibble: 1 x 3
#>   `1`    `2`                 `3`
#>   <chr>  <chr>               <chr>
#> 1 cell 1 "cell\" 2,\"cell 3" <NA>

Cell 2 should be "cell" 2 and cell 3 should be cell 3.

@jimhester
Copy link
Member

@jimhester jimhester commented Mar 15, 2018

readr assumes the entire field is quoted if any quotes appear. In this case you need to disable quoting by setting quote = "".

readr::read_csv('1,2,3\ncell 1,"cell" 2,"cell 3"\n', quote="")
#> # A tibble: 1 x 3
#>   `1`    `2`          `3`         
#>   <chr>  <chr>        <chr>       
#> 1 cell 1 "\"cell\" 2" "\"cell 3\""

Created on 2018-03-15 by the reprex package (v0.2.0).

@gregrs-uk
Copy link
Contributor Author

@gregrs-uk gregrs-uk commented Mar 16, 2018

Thanks @jimhester. It only seems to assume the field is quoted if a quote appears at the beginning of the cell i.e. "here" is the cell value rather than anywhere else e.g. here is the "cell" value or here is the cell "value". The latter two parse correctly and the cell value includes the quotation marks.

I'm trying to read a CSV file with less than ideal quoting i.e. unquoted cell values which themselves contain quotation marks, but I guess the best way will be to read_csv with quote = "" then sort out the mess afterwards!

EDIT: unfortunately that approach doesn't work as setting quote = "" means that commas within quoted cells are treated as a delimiter.

@jimhester
Copy link
Member

@jimhester jimhester commented Mar 16, 2018

Then you are going to have to preprocess your data in some fashion, we do not support the format you describe.

@gregrs-uk
Copy link
Contributor Author

@gregrs-uk gregrs-uk commented Mar 17, 2018

Thanks for your time, @jimhester.

@lock
Copy link

@lock lock bot commented Sep 25, 2018

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/

@lock lock bot locked and limited conversation to collaborators Sep 25, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants