Home

stream-csv-as-json

stream-csv-as-json is a micro-library of Node.js stream components for creating custom CSV processing pipelines with a minimal memory footprint. It can parse CSV files far exceeding available memory by streaming individual primitives using a SAX-inspired token API.

It is a companion library for stream-json and stream-chain, fully compatible with them. All components are factory functions returning flushable closures, composed via stream-chain.

The parser produces {name, value} tokens that are fully compatible with the stream-json token protocol. This means you can use stream-json utilities downstream — filters, streamers (like streamValues), and other components work seamlessly with CSV token streams.

Features

• Parse CSV files compliant with RFC 4180 — quoted fields, embedded newlines. Reads CRLF, LF, and bare CR row terminators; strips a leading UTF-8 BOM.
• Customizable field separator (comma, tab, pipe, etc.).
• Stream individual field values piece-wise for minimal memory use.
• Convert rows to object token streams using the first row as field names.
• Convert token streams back to CSV text.
• Token protocol compatible with stream-json — use its filters, streamers, and utilities downstream.
• TypeScript declarations included.

Install

npm i stream-csv-as-json

Quick start

Examples use ESM (import). CommonJS (require) is also supported — see Modules below.

import fs from 'node:fs';
import {parser} from 'stream-csv-as-json';
import asObjects from 'stream-csv-as-json/as-objects.js';
import chain from 'stream-chain';

// a,b,c
// 1,2,3

const pipeline = chain([fs.createReadStream('data.csv'), parser(), asObjects()]);

pipeline.on('data', token => console.log(token));
// {name: 'startObject'}
// {name: 'keyValue', value: 'a'}
// {name: 'stringValue', value: '1'}
// ...
// {name: 'endObject'}

Modules

This package supports both ESM (import) and CommonJS (require).

ESM (recommended):

import {parser} from 'stream-csv-as-json';
import asObjects from 'stream-csv-as-json/as-objects.js';
import stringer from 'stream-csv-as-json/stringer.js';

CommonJS:

const {parser} = require('stream-csv-as-json');
const asObjects = require('stream-csv-as-json/as-objects.js');
const stringer = require('stream-csv-as-json/stringer.js');

Components

Component	Import path	Description
parser	stream-csv-as-json/parser.js	Streaming CSV parser producing {name, value} tokens
as-objects	stream-csv-as-json/as-objects.js	Header row → object token stream
stringer	stream-csv-as-json/stringer.js	Token stream → CSV text
utils/with-parser	stream-csv-as-json/utils/with-parser.js	CSV-specific withParser utility
Main module	stream-csv-as-json	Parser + emit() for event-based processing

Each component is a factory function: call it to get a flushable function for use in chain(), or use .asStream() to get a Duplex stream.

Token protocol

The parser represents CSV as a stream of {name, value} tokens:

Token	Value	Meaning
startArray	—	Start of a CSV row
endArray	—	End of a CSV row
startString	—	Start of a field value
stringChunk	string	Piece of a field value
endString	—	End of a field value
stringValue	string	Complete field value (packed)

By default both streamed (startString/stringChunk/endString) and packed (stringValue) tokens are emitted. This is controlled by packing and streaming options — see parser for details.

After as-objects, additional tokens appear:

Token	Value	Meaning
startObject	—	Start of a data row
endObject	—	End of a data row
startKey	—	Start of field name
stringChunk	string	Piece of field name
endKey	—	End of field name
keyValue	string	Complete field name (packed)

Common patterns

CSV round-trip

import fs from 'node:fs';
import stringer from 'stream-csv-as-json/stringer.js';

chain([fs.createReadStream('input.csv'), parser(), stringer(), fs.createWriteStream('output.csv')]);

Compressed CSV

import fs from 'node:fs';
import zlib from 'node:zlib';

chain([fs.createReadStream('data.csv.gz'), zlib.createGunzip(), parser(), asObjects()]);

Custom separator (TSV)

import fs from 'node:fs';

chain([fs.createReadStream('data.tsv'), parser({separator: '\t'}), asObjects()]);

Event-based processing

import fs from 'node:fs';
import makeParser from 'stream-csv-as-json';

const stream = makeParser();
stream.on('startArray', () => console.log('row started'));
stream.on('stringValue', val => console.log('field:', val));
fs.createReadStream('data.csv').pipe(stream);

Ecosystem integration

Because the parser produces tokens compatible with stream-json, you can use stream-json utilities downstream:

import fs from 'node:fs';
import chain from 'stream-chain';
import {parser} from 'stream-csv-as-json';
import asObjects from 'stream-csv-as-json/as-objects.js';
import streamValues from 'stream-json/streamers/stream-values.js';

// Convert CSV rows to actual JavaScript objects
chain([
  fs.createReadStream('data.csv'),
  parser(),
  asObjects(),
  streamValues(),
  ({value}) => console.log(value) // {name: 'John', age: '30', ...}
]);

Available stream-json utilities that work with CSV token streams:

• streamers: streamValues — assembles tokens into complete values
• filters: Filter, Pick, Ignore — filter the token stream
• utils: emit — emits named events for each token type (used by the main module)

Performance

See Performance for tips on optimizing pipelines.

Migrating from 1.x

See Migration from 1.x to 2.x for a detailed guide on upgrading.

V1 documentation

Documentation for the 1.x API is preserved with a V1 prefix: V1-Home · V1-Parser · V1-AsObjects · V1-Stringer · V1-Main-module · V1-Performance

Credits

The test file tests/sample.csv.gz is Master.csv from Lahman's Baseball Database 2012. The file is copyrighted by Sean Lahman. It is used here under a Creative Commons Attribution-ShareAlike 3.0 Unported License. In order to test all features of the CSV parser, the file was minimally modified: row #1000 has a CRLF inserted in a value, row #1001 has a double quote inserted in a value, then the file was compressed by gzip.