in fread, NAs do not interact properly with whitespace

[st-pasha]

This small file contains numeric values only, as well as some NAs.

> DT = fread('A,   B,    C\n17,  34, 2.3\n3.,  NA,   1\nNA ,  2, NA \n0,0.1,0', verbose=TRUE)
Input contains a \n or is "". Taking this to be text input (not a filename)
[01] Check arguments
  Using 8 threads (omp_get_max_threads()=8, nth=8)
  NAstrings = [<<NA>>]
  None of the NAstrings look like numbers.
  show progress = 1
  0/1 column will be read as boolean
[02] Opening the file
  `input` argument is provided rather than a file name, interpreting as raw text to read
[03] Detect and skip BOM
[04] Arrange mmap to be \0 terminated
  \n has been found in the input and different lines can end with different line endings (e.g. mixed \n and \r\n in one file). This is common and ideal.
[05] Skipping initial rows if needed
  Positioned on line 1 starting: <<A,   B,    C>>
[06] Detect separator, quoting rule, and ncolumns
  Detecting sep automatically ...
  sep=','  with 5 lines of 3 fields using quote rule 0
  Detected 3 columns on line 1. This line is either column names or first data row. Line starts as: <<A,   B,    C>>
  Quote rule picked = 0
  fill=false and the most number of columns found is 3
[07] Detect column types, good nrow estimate and whether first row is column names
  Number of sampling jump points = 2 because (59 bytes from row 1 to eof) / (2 * 59 jump0size) == 0
  Type codes (jump 000)    : A7A  Quote rule 0
  'header' determined to be true due to column 2 containing a string on row 1 and a lower type (float64) in the rest of the 4 sample rows
  All rows were sampled since file is small so we know nrow=4 exactly
[08] Assign column names
[09] Apply user overrides on column types
  After 0 type and 0 drop user overrides : A7A
[10] Allocate memory for the datatable
  Allocating 3 column slots (3 - 0 dropped) with 4 rows
[11] Read the data
  jumps=[0..1), chunk_size=1048576, total_size=46
  jumps=[0..1), chunk_size=1048576, total_size=46
Read 4 rows x 3 columns from 59 bytes file in 00:00.000 wall clock time
[12] Finalizing the datatable
  Type counts:
         3 : string    'A'
=============================
   0.000s ( 18%) Memory map 0.000GB file
   0.000s ( 42%) sep=',' ncol=3 and header detection
   0.000s (  4%) Column type detection using 4 sample rows
   0.000s (  4%) Allocation of 4 rows x 3 cols (0.000GB) of which 4 (100%) rows used
   0.000s ( 32%) Reading 1 chunks (0 swept) of 1.000MB (-2147483648 rows) using 1 threads
   +    0.000s (  4%) Parse to row-major thread buffers (grown 0 times)
   +    0.000s (  2%) Transpose
   +    0.000s ( 26%) Waiting
   0.000s ( 12%) Rereading 1 columns due to out-of-sample type exceptions
   0.000s        Total
Column 2 ("B") bumped from 'float64' to 'string' due to <<  NA>> on row 1

Few problems can be seen here:

• Type codes (jump 000) : A7A, meaning the first and the last column were recognized as strings, not numeric. NAs were not properly parsed.
• The data was re-read ("Column 2 bumped from 'float64' to 'string' due to << NA>> on row 1"). No re-reads should be happening for values that are within the sampling lines.
• Number of sampling jump points = 2 it's a trivial thing of course, but why 2 sampling points for a tiny file?
• Reading 1 chunks (0 swept) of 1.000MB (-2147483648 rows) : negative number of rows reported.

[AmyMikhail]

I am using data.table_1.10.5 and get the following error (which sounds like a description of the same problem):

> mydt <- fread("My file.csv", stringsAsFactors = F, na.strings = c("", " "), verbose = T)
Input contains no \n. Taking this to be a filename to open
[01] Check arguments
  Using 4 threads (omp_get_max_threads()=4, nth=4)
Error in fread("My file.csv", stringsAsFactors = F, na.strings = c("",  : 
  freadMain: NAstring << >> has whitespace at the beginning or end

If I use read.csv() I am able to read in the same file without problems, i.e.:

> mydt <- data.table(read.csv("My file.csv", stringsAsFactors = F, na.strings = c("", " ")))