'Failed to convert data to 'utf8' encoded string' with RNFetchBlob.fs.readStream

[Kenny806]

Hi! I am trying to read a CSV file with RNFetchBlob.fs.readStream, but keep getting this error:

Failed to convert data to 'utf8' encoded string, this might due to the source data is not able to convert using this encoding. source = *** -[__NSPlaceholderDictionary initWithObjects:forKeys:count:]: attempt to insert nil object from objects[1]

Curious thing is, that when I use readFile() instead of readStream(), the operation runs just fine and I get a result. The problem is, I would like to use readStream() because some of the files I need to read are too big to be efficiently handled by readFile().

Also, other files in the same directory and with the same format can be processed with both readFile() and readStream().

Here is my code:

readFiles(path) {
    let data ='';
    RNFetchBlob.fs.readStream(`${path}/trips.txt`, 'utf8')
    .then((ifstream) => {
      ifstream.open();
      ifstream.onData((chunk) => {
        data += chunk;
      });
      ifstream.onError((err) => {
        console.log(err);
      });
      ifstream.onEnd(() => {
        this.processCSVData(data);
      });
    }).catch((err) => {
      console.log(err);
    });
  }

This is the part that works on some files and does not work on others (for example the trips.txt). I have not found anything that makes the files that work different from the files that don't. Worth noting is, that it always crashes inside the readStream() call, without ever getting to any of the callbacks.

What always works (except for very large files, obviously :-) ), is this:

RNFetchBlob.fs.readFile(`${path}/trips.txt`, 'utf8').then((contents) => {
  this.processCSVData(contents);
});

my RN and RNFetchBlob versions:

"react-native": "0.50.1",
 "react-native-fetch-blob": "^0.10.8",

Any help is appreciated.

Thanks!

[sumitnitsurat]

hey have you resolved this issue . I am facing the same issue.

[Kenny806]

No, not really, I had to use a workaround. If the error occurs, I use RNFetchBlob.fs.readFile to read the file's contents. Luckily for me, the error only occurs on files that are not too big to read with readFile (I have a more or less consistent dataset, and it's always the same two files that produce this error, even if the content changes). If the file is so big you need 'readStream', then you're out of luck I'm afraid.

[sumitnitsurat]

hey Thanks for the reply. Ill apply the same. Can you confirm me one more thing? I have xls file with 1000 rows and 10 column. RNFetchBlob.fs.readFile would work fine with it ? whats the limit of big for readFile().

[Kenny806]

I am reading a csv with ~70k rows and 6 columns with readFile, so you should be fine. There is no "limit" in the real sense of the word, the processing time just increases rapidly and at some point it becomes useless. Haven't tried to find the point, it probably also depends on your computing power. Just try it :-)

BTW a csv with ~1.8M rows is too large, the script crashes when I attempt to read it with readFile

[sumitnitsurat]

Ahh..okay Thanks for the info. One last thing have you tried reading the .xls or .xlsx file. I am able to read .csv file but not .xls or .xlsx file.