Skip to content

Performance

Eugene Lazutkin edited this page Mar 31, 2026 · 3 revisions

Performance

stream-csv-as-json is designed to process huge files. Even a microsecond per operation adds up: over 1 billion operations, a microsecond becomes ~16.5 minutes; a millisecond becomes ~11.5 days.

Tips

  • Disable streaming if you only need packed values. Use {packValues: true, streamValues: false} to emit only stringValue tokens. This halves the number of tokens flowing through the pipeline.
  • Disable packing if you only need streamed chunks. Use {packValues: false} to skip accumulating complete values. Useful when processing fields piece-wise.
  • Use value mode in Stringer ({useStringValues: true}) together with pack-only parsing for the most efficient round-trip.
  • Keep pipelines short. Every component adds overhead. Combine logic into a single function where possible.

For general stream pipeline performance advice, see stream-json's Performance document. The same principles apply — options named similarly exist for the same tuning purposes in both toolkits.

Clone this wiki locally