Skip to content
This repository
Newer
Older
100644 227 lines (167 sloc) 6.011 kb
a0e3aa87 »
2012-10-03 Add generated documentation
1 ---
2 language: en
3 layout: page
4 title: "Node CSV"
5 date: 2012-10-02T15:35:24.008Z
6 comments: false
7 sharing: false
8 footer: false
9 navigation: csv
10 github: https://github.com/wdavidw/node-csv
11 ---
12
13
14 This project provides CSV parsing and has been tested and used
15 on a large input file (over 2Gb).
16
17 * Follow the NodeJs streaming API
18 * Async and event based
19 * Support delimiters, quotes and escape characters
20 * Line breaks discovery: line breaks in source are detected and reported to destination
21 * Data transformation
22 * Support for large datasets
23 * Complete test coverage as sample and inspiration
24
25 Important, this documentation cover the current version of the node
26 csv parser. The documentation for the current version 0.1.0 is
27 available [here](https://github.com/wdavidw/node-csv-parser/tree/v0.1).
28
29 Quick example
30 -------------
31
32 The following example illustrate 4 usages of the library:
33 1. Plug a readable stream by defining a file path
34 2. Direct output to a file path
35 3. Transform the data (optional)
36 4. Listen to events (optional)
37
38 ```javascript
39
40 // node samples/sample.js
41 var csv = require('csv');
42
43 csv()
44 .from.stream(fs.createReadStream(__dirname+'/sample.in')
45 .to.path(__dirname+'/sample.out')
46 .transform( function(data){
47 data.unshift(data.pop());
48 return data;
49 })
50 .on('record', function(data,index){
51 console.log('#'+index+' '+JSON.stringify(data));
52 })
53 .on('end', function(count){
54 console.log('Number of lines: '+count);
55 })
56 .on('error', function(error){
57 console.log(error.message);
58 });
59
60 // Print sth like:
61 // #0 ["2000-01-01","20322051544","1979.0","8.8017226E7","ABC","45"]
62 // #1 ["2050-11-27","28392898392","1974.0","8.8392926E7","DEF","23"]
63 // Number of lines: 2
64
65 ```
66
67 Pipe example
68 ------------
69
70 The module follow a Stream architecture
71
72 |-----------| |---------|---------| |---------|
73 | | | | | | |
74 | | | CSV | | |
75 | | | | | | |
76 | Stream | | Writer | Reader | | Stream |
77 | Reader |.pipe(| API | API |).pipe(| Writer |)
78 | | | | | | |
79 | | | | | | |
80 |-----------| |---------|---------| |---------|
81
82 in = fs.createReadStream('./in')
83 out = fs.createWriteStream('./out')
84 in.pipe(csv()).pipe(out)
85
86 Installing
87 ----------
88
89 Via [npm](http://github.com/isaacs/npm):
90 ```bash
91 npm install csv
92 ```
93
94 Via git (or downloaded tarball):
95 ```bash
96 git clone http://github.com/wdavidw/node-csv-parser.git
97 ```
98
99 Events
100 ------
101
102 By extending the Node `EventEmitter` class, the library provides
103 a few useful events:
104
105 * *record*
106
107 ```javascript
108 Emitted by the stringifier when a new row is parsed and transformed. The data is
109 the value returned by the user `transform` callback if any. Note however that the event won't
110 be called if transform return `null` since the record is skipped.
111 The callback provides two arguments. `data` is the CSV line being processed (an array or an object)
112 and `index` is the index number of the line starting at zero
113 ```
114
115 * *data*
116
117 ```javascript
118 Emitted by the stringifier on each line once the data has been transformed and stringified.
119 ```
120
121 * *drain*
122 * *end*
123
124 ```javascript
125 Emitted when the CSV content has been parsed.
126 ```
127
128 * *close*
129
130 ```javascript
131 Emitted when the underlying resource has been closed. For example, when writting to a file with `csv().to.path()`, the event will be called once the writing process is complete and the file closed.
132 ```
133
134 * *error*
135
136 ```javascript
137 Thrown whenever an error occured.
138
139 ```
140
141 Columns
142 -------
143
144 Columns names may be provided or discovered in the first line with
145 the read options `columns`. If defined as an array, the order must
146 match the one of the input source. If set to `true`, the fields are
147 expected to be present in the first line of the input source.
148
149 You can define a different order and even different columns in the
150 read options and in the write options. If the `columns` is not defined
151 in the write options, it will default to the one present in the read options.
152
153 When working with fields, the `transform` method and the `data`
154 events receive their `data` parameter as an object instead of an
155 array where the keys are the field names.
156
157 ```javascript
158
159 // node samples/column.js
160 var csv = require('csv');
161
162 csv()
163 .from.path(__dirname+'/columns.in', {
164 columns: true
165 })
166 .to.stream(process.stdout, {
167 columns: ['id', 'name']
168 })
169 .transform(function(data){
170 data.name = data.firstname + ' ' + data.lastname
171 return data;
172 });
173
174 // Print sth like:
175 // 82,Zbigniew Preisner
176 // 94,Serge Gainsbourg
177 ```
178
179
180 <a name="pause"></a>`pause()`
181 ---------
182
183 Implementation of the Readable Stream API, requesting that no further data
184 be sent until resume() is called.
185
186
187 <a name="resume"></a>`resume()`
188 ----------
189
190 Implementation of the Readable Stream API, resuming the incoming 'data'
191 events after a pause()
192
193
194 <a name="write"></a>`write(data, [preserve])`
195 -------------------------
196
197 Implementation of the Writable Stream API with a larger signature. Data
198 may be a string, a buffer, an array or an object.
199
200 If data is a string or a buffer, it could span multiple lines. If data
201 is an object or an array, it must represent a single line.
202 Preserve is for line which are not considered as CSV data.
203
204
205 <a name="end"></a>`end()`
206 -------
207
208 Terminate the parsing. Call this method when no more csv data is
209 to be parsed. It implement the StreamWriter API by setting the `writable`
210 property to "false" and emitting the `end` event.
211
212
213 <a name="transform"></a>`transform(callback)`
214 ---------------------
215
216 Register the transformer callback. The callback is a user provided
217 function call on each line to filter, enrich or modify the
218 dataset. More information in the "transforming data" section.
219
220
221 <a name="error"></a>`error(error)`
222 --------------
223
224 Unified mechanism to handle error, emit the error and mark the
225 stream as non readable and non writable.
226
Something went wrong with that request. Please try again.