Skip to content

v2.8.35

Choose a tag to compare

@gildas-lormeau gildas-lormeau released this 08 Aug 15:34
· 11 commits to master since this release

What's Changed

This release focuses on streaming I/O: readers can now hand over a native stream instead of
being read chunk by chunk, which removes a copy for blobs and a whole series of HTTP requests
for remote archives read with range requests. It also fixes several zip64 and split-archive issues, and replaces the
hand-maintained list of property names protected from minification with a derived one checked
at build time.

New features

  • Reader#createReadable(options), introduced internally in 2.8.31, is now part of the documented
    API, with the new CreateReadableOptions type (offset, size, chunkSize). Custom readers can
    override it to return a stream provided natively by the underlying data source instead of relying
    on readUint8Array(). It also accepts a byte range, and no longer takes a diskNumberStart
    option: split readers now use linear disk offsets
  • BlobReader now returns blob.stream(), or blob.slice(offset, end).stream() for a range,
    instead of reading the data in chunkSize slices through readUint8Array()
  • When range requests are used, i.e. with HttpRangeReader or with the useRangeHeader and
    forceRangeRequests options of HttpReader, the data of an entry is now read with range requests
    of 16MB at most whose response bodies are streamed, instead of one range request per chunkSize
    bytes. Reading a 4GB entry sends 256 requests instead of 65536, and no request is ever sized after
    the entry. This requires fetch, the useXHR option still reads the data chunk by chunk
  • Add the maximumRangeSize option to HttpReader and HttpRangeReader to tune the size of these
    range requests, e.g. to lower it behind a proxy closing long-lived responses
  • These two changes only affect how the bytes are obtained: the chunks emitted by the platform are
    still normalized to chunkSize before reaching the codec, and the chunks received by writers are
    unchanged
  • Add the closeDisk() method to SplitDataWriter to close the disk being written, the next
    disk being opened when more data is written
  • Add the checkResourceChanges option to HttpRangeOptions and the ERR_HTTP_RESOURCE_CHANGED
    error constant: range requests now detect a resource modified while being read by comparing the
    ETag, Last-Modified and total size headers against the ones returned by the first request
  • Add resetConfiguration() to restore the default configuration of zip.js

Behavior changes

  • checkResourceChanges defaults to true, so reading an HTTP resource that changes mid-read now
    throws ERR_HTTP_RESOURCE_CHANGED instead of silently producing corrupt data. Headers missing
    from the responses are ignored; note that Access-Control-Expose-Headers must include them when
    the resource is fetched cross-origin
  • Entries requiring no codec work at all, i.e. no compression, no CRC-32 and no encryption, now
    always bypass web workers. They previously went through a worker when transferStreams was
    enabled, i.e. by default. In practice this covers reading stored entries without the
    checkSignature option, and passThrough transfers; entries needing the CRC-32, which includes
    every entry written by add() without passThrough, still go through a worker
  • Writers always receive a Uint8Array that owns its entire buffer, so chunk.buffer inside a
    custom writeUint8Array() implementation is the chunk and nothing more
  • Non-split writers no longer get diskNumber, diskOffset, availableSize and maxSize
    properties assigned on them by ZipWriter
  • The chunkSize option of createReadable() defaults to the chunkSize value of the global
    configuration instead of a hardcoded 64KB. It only applies to the default implementation, the
    readers overriding it emit chunks sized by the platform

Fixes

  • Fix configure({ transferStreams }) being silently ignored: the option was documented and
    honored per entry, but absent from the list of configurable properties, so it could not be set
    globally. It is now a global option defaulting to true
  • Fix the chunkSize option being ignored when the codec runs without a web worker: the codec
    output is now rechunked in that path too
  • Fix reading zip64 archives whose end of central directory record is not on the last disk: the
    disk number is now read from the zip64 end of central directory locator instead of being assumed
  • Fix prependZip() writing a wrong zip64 offset flag and leaking the disk numbers of the source
    archive into the central directory of the new archive
  • Fix createSyncAccessHandle being mangled away in the minified builds, which broke
    createOPFSTempStream() in dist/*.min.js, index.min.js and index-native.min.js
  • Make the error message reported for uncaught errors more informative
  • Fix the web worker configuration of the test suite in dist mode
  • Fix the build-dev configuration, broken by the removal of mini-lz

Performance

  • Make ChunkStream accumulate a queue of pending chunks instead of concatenating into a growing
    buffer, which removes the copies that the iterative rechunking introduced in 2.8.27 still made
    on every incoming chunk
  • Copy a chunk sent through the web worker message protocol only when it is a partial view of its
    buffer, and stop copying chunks received from a worker
  • Return the cached data view instead of a copy when an HTTP read is served from the buffer
  • Remove the redundant copies made when reading extra fields
  • Initialize the reader lazily when prepending a zip file, so nothing is read if no entry is added
  • Let rollup drop the default configuration table from the web worker bundles

Build and packaging

  • Derive the property names reserved from minification instead of maintaining them by hand: they
    are now collected from the lib.dom/lib.webworker TypeScript declarations, the export table of
    the WASM module, index.d.ts, and an explicit list of the names crossing postMessage
  • Fail the build when the set of mangled property names changes, so a name newly exposed at a
    boundary has to be audited instead of being silently renamed in one bundle only
  • Add a test recording every property name crossing postMessage during worker round trips
    (deflate, signed, AES-256, AES-128, ZipCrypto, raw password, with and without transferStreams)
    and asserting each one is declared
  • Do not publish the .github folder on NPM and JSR

Tests

  • Add tests for zip64 split boundaries, HTTP range requests, HTTP resource changes, exact chunks
    passed to writers, and SplitDataWriter edge cases
  • Reset the configuration between tests
  • Make the Deno MessagePort leak repro runnable with deno test (still reproduced with Deno 2.9.5)

Internal

  • Centralize the array helpers in a util/array.js module

Documentation

  • Add a bundle size section to the README documenting tree-shaking and the smaller entry points
  • Clarify the meaning of the native suffix in the dist README: every bundle uses the native
    CompressionStream/DecompressionStream APIs when available, the suffix names the
    implementation embedded for everything else
  • Clarify the comment describing the Blob.slice() workaround in BlobReader

Full Changelog: v2.8.34...v2.8.35