v10.4.0

✨ Enhancements

• Add an opt-in tokenizer buffer allocation limit @Borewit (#1320)

  ⚠️ Recommendation: enable a buffer cap for untrusted input

  Applications parsing untrusted input should set a finite maxBufferSize when creating a tokenizer. The cap is opt-in and defaults to Infinity; choose a limit appropriate for your application, for example 64 MiB:

  import { fromStream } from 'strtok3';
  
  const tokenizer = await fromStream(stream, {
    maxBufferSize: 64 * 1024 * 1024
  });

  This limits dynamically sized tokenizer buffers and stream lookahead spans. Oversized buffer or lookahead requests reject with BufferSizeError. It does not cap total process memory, caller-owned buffers, allocations inside a token's get() method, or chunks supplied by stream sources.

  See the tokenizer options documentation for details.

🐛 Bug Fixes

• fix: respect AbortSignal lifecycle and detach listeners @Borewit (#1324)
• fix: detach cancelled Node stream read requests @Borewit (#1323)
• fix: handle AbortSignal cancellation rejections @Borewit (#1322)

📚 Documentation

• docs: clarify Web stream source cancellation behavior @Borewit (#1325)
• docs: correct README API documentation and examples @Borewit (#1316)

Full Changelog

📦 NPM release

NPM release: strtok3@10.4.0