-
-
Notifications
You must be signed in to change notification settings - Fork 2
Performance
Eugene Lazutkin edited this page Mar 31, 2026
·
3 revisions
stream-csv-as-json is designed to process huge files. Even a microsecond per operation adds up: over 1 billion operations, a microsecond becomes ~16.5 minutes; a millisecond becomes ~11.5 days.
-
Disable streaming if you only need packed values. Use
{packValues: true, streamValues: false}to emit onlystringValuetokens. This halves the number of tokens flowing through the pipeline. -
Disable packing if you only need streamed chunks. Use
{packValues: false}to skip accumulating complete values. Useful when processing fields piece-wise. -
Use value mode in Stringer (
{useStringValues: true}) together with pack-only parsing for the most efficient round-trip. - Keep pipelines short. Every component adds overhead. Combine logic into a single function where possible.
For general stream pipeline performance advice, see stream-json's Performance document. The same principles apply — options named similarly exist for the same tuning purposes in both toolkits.
Start here
Components
Tuning
Reference
stream-csv-as-json 1.x (legacy)
Built on stream-chain and stream-json