parsing failure- read_fwf includes a column of NAs when told to ignore the column?

[ajdamico]

sorry if i'm misreading something. thanks!

library(readr)
fwf_sample <- system.file("extdata/fwf-sample.txt", package = "readr")
cat(read_lines(fwf_sample))

# works
read_fwf(fwf_sample, fwf_widths(c(2, 5, 3)),col_types='ddd')

# includes a column of NAs because the column contains data even though it was told to skip?
read_fwf(fwf_sample, fwf_widths(c(2, 5, 3)),col_types='d-d')

[mlaviolet]

I ran into the same problem, getting parsing errors while trying to read selected fields from a large BRFSS survey data file. Took a couple of hours to realize what was happening. The help page for read_fwf now says "The width of the last column will be silently extended to the next line break" so R reads the entire remainder of the line following what's supposed to be the last field. I don't understand the rationale for this change; the old behavior needs to be restored or an argument added to say "I'm done, go to the next line."

[ghaarsma]

A temporary workaround, is to first create the col_positions and then remove the column names that you don't need.

fwf_sample <- system.file("extdata/fwf-sample.txt", package = "readr")
cat(read_lines(fwf_sample))
col_positions <- fwf_widths(c(2, 5, 3))
col_types <- 'd-d'

col_positions$col_names <- col_positions$col_names[!strsplit(col_types,'')[[1]] %in% c('_','-')]

read_fwf(fwf_sample,col_positions = col_positions ,col_types = col_types)

Note that there are other issues with the read_fwf (See #300) that can be related.

[hadley]

Deleted the irrelevant comments. Possibly related to #371