big csv

[sureshvarman]

I'm having 1 million records of CSV file, I need to apply some transform on each record, how can I achieve this with read and write stream

[Keyang]

Hi,
Take a look at this:
https://github.com/Keyang/node-csvtojson#synchronouse-transformer
some example:

var Converter=require("csvtojson").Converter;
var csvConverter=new Converter({constructResult:false}); //important to
turn off result construction

csvConverter.transform=function(json,row,index){
    json["rowIndex"]=index;
    /* some other examples:
    delete json["myfield"]; //remove a field
    json["dateOfBirth"]=new Date(json["dateOfBirth"]); // convert a field
type
    */
};

var rs=require("fs").createReadStream("./millionsRows.csv");
rs.pipe(csvConverter)
//.pipe(downstream)

~Keyang

On 18 February 2016 at 17:49, suresh varman notifications@github.com
wrote:

I'm having 1 million records of CSV file, I need to apply some transform
on each record, how can I achieve this with read and write stream

—
Reply to this email directly or view it on GitHub
#79.

[sureshvarman]

thanks for the quick response, previously the transform was not working, so we were using the "end_parsed" we faced a problem while processing the csv file of 125 MB its because we were using the version 0.4.5 now the problem is been resolved by upgrading to newer version. i am closing this thanks again.