v2.8.54

What's Changed in v2.8.54

Breaking changes

• ZipWriter#prependZip() now reads an array of readers as the disks of a split zip file, which is what an array denotes everywhere else in the API, and accepts a SplitDataReader instance the same way. The disks are read in order and the entries are relocated to the positions they get in the output. An array used to be concatenated and read as a single archive, which produced wrong offsets for a real split zip file. If you were passing an array of byte ranges of one zip file, concatenate them yourself and pass a single reader
• ZipEntry#moveTo() is removed. It was deprecated and undeclared in the TypeScript definitions, and was a one-line alias of ZipFS#move(), which is the method to use
• The undeclared ZipFS#addData() and ZipDirectoryEntry#addData() methods are removed. They were internal, never documented and never declared. The typed addText(), addBlob(), addUint8Array(), addData64URI(), addHttpContent(), addReadable(), addFile(), addFileSystemEntry() and addFileSystemHandle() methods cover what they did

Security

Both fixes below are reachable from an untrusted zip file read with ZipReader. Upgrading is recommended for anyone reading archives they did not produce.

• FileEntry#getData() now throws the new ERR_ENTRY_DATA_OUT_OF_BOUNDS when the declared data of an entry, i.e. its offset plus its compressed size, ends past the end of the zip file. Such an entry used to make getData() hang for ever, with no error and no CPU use, so nothing timed out and nothing showed up in a profile. Honestly truncated archives are affected as much as malformed ones. A read past the end of the source now ends the stream instead of stalling it, which also covers the entries the bounds check cannot detect in advance
• The output of FileEntry#getData() is no longer allocated from the declared uncompressed size of the entry. The size was reserved before a byte was read, so a 131-byte archive declaring 3 GiB reserved 3 GiB. The allocation is clamped to what the compressed data can decode to, compressedSize * 1032 for a compressed entry and compressedSize for a stored one, 1032 being the maximum expansion ratio of Deflate. The clamp never binds on real data, a legitimate archive still preallocates exactly its uncompressed size

New features

• ZipFS, ZipEntry, ZipFileEntry and ZipDirectoryEntry are now exported at the top level, and the fs namespace is deprecated. Replace new zip.fs.FS() with new zip.ZipFS(), and zip.fs.ZipFileEntry with zip.ZipFileEntry. zip.fs keeps working and the library emits no runtime warning, the deprecation is documentation only. ZipEntry is now a value as well, so entry instanceof zip.ZipEntry works. The three entry classes were already declared as top-level exports but existed at runtime under zip.fs.* only, so importing them type-checked and then failed. In TypeScript, the FS type is deprecated and kept as an alias of ZipFS, so let fs: FS keeps compiling
• New ZipEntry#setOptions() method and ZipEntry#options property in the filesystem API. setOptions() merges the options into the ones the entry was added with, an option set to undefined being removed instead of stored, and they are applied when the zip file is exported. It is the way to set the options of an entry imported from a zip file, which has none until it is called. The options describing the data of an entry exported with passThrough, e.g. compressionMethod and uncompressedSize, are ignored, they are always the ones of the original entry, and so are directory and the progress callbacks
• ZipWriter#prependZip() now writes a correct split zip file when the writer is a split zip file writer. The whole prepended archive used to be copied into the first disk, so every entry recorded an offset on the wrong disk. The data is copied disk by disk now, a disk is closed before an entry whose local file header would not fit in what is left of it, and each entry records the disk it starts on and its offset in that disk. The output also starts with the split zip file signature, unless the prepended zip file already carries one
• TextWriter now decodes CP437. new TextWriter("cp437") used to return the data decoded as UTF-8, since the encoding was handed to FileReader#readAsText(), which falls back to UTF-8 for a label it does not know. It goes through the same decoder as the filenames and the comments now. It also decodes with TextDecoder instead of FileReader, which removes the last dependency on that class, missing from some worker scopes. The byte order mark is still removed, whichever branch decodes the data
• The second argument of the codec stream constructors is now typed, by the exported CompressionStreamOptions and DecompressionStreamOptions interfaces. They document which members are set for which class, e.g. deflate64 only for the deflate implementations, and rawBitFlag, compressionMethod and uncompressedSize only for the codecs registered with registerCodec(). Configuration#CompressionStream, Configuration#DecompressionStream, their *Fallback and deprecated *Zlib forms and CodecDefinition are declared with them instead of the untyped TransformStreamLike. This only concerns you if you pass a custom stream implementation or call registerCodec()
• Configuration#baseURI is now declared. It resolves the relative workerURI, wasmURI and codecURI values, and defaults to the URL of the module of zip.js
• WritableWriter#size is now declared. zip.js sets it to 0 before the first write and keeps it updated, so a custom Writer can read how many bytes have been written so far, e.g. to compute the offset of a disk. It is declared on Writer, TextWriter, BlobWriter, SplitDataWriter and Uint8ArrayWriter as well
• Members that existed and were not declared: HttpReader#url, TextWriter#encoding, BlobWriter#contentType, Data64URIWriter#contentType, EntryError#overlappingEntry and EntryError#reason. overlappingEntry is the only way to identify the other entry of the pair reported by ERR_OVERLAPPING_ENTRY, and reason describes the ambiguity reported by ERR_AMBIGUOUS_ARCHIVE

Behavior changes

The options listed first used to accept values of the wrong type and produced a wrong, empty or silently dropped result. They throw now. If your code passes the documented types, nothing changes.

• lastModDate, lastAccessDate and creationDate must be Date instances and throw the new ERR_INVALID_DATE otherwise. An invalid Date used to be written as an entry carrying no timestamp at all. A timestamp expressed in milliseconds is the natural mistake and is rejected as well: pass new Date(file.lastModified), not file.lastModified
• The comment option of an entry must be a string and throws the new ERR_INVALID_ENTRY_COMMENT_TYPE otherwise. A Uint8Array used to be coerced and its textual representation written into the archive. Decode the bytes to a string before passing them
• The extraField option must be a Map, and throws the new ERR_INVALID_EXTRAFIELD otherwise. Its keys must be integers between 0 and 65535, and ERR_INVALID_EXTRAFIELD_TYPE now covers a non-integer or a negative key as well as a key above 65535. Its values must be Uint8Array instances, and throw the new ERR_INVALID_EXTRAFIELD_DATA_TYPE otherwise
• The readerOptions option of ZipDirectoryEntry#export*(), ZipDirectoryEntry#getExportedSize() and ZipDirectoryEntry#exportFileSystemHandle() must be an object and throws the new ERR_INVALID_READER_OPTIONS otherwise. A value of another type was silently ignored: a password passed as a string instead of an object failed with the unrelated ERR_ENCRYPTED, while the other options were dropped without any error. An unknown property of a readerOptions object is still ignored, as everywhere else in the API
• The options expecting a function throw the new ERR_INVALID_FUNCTION_OPTION when they are given a value of another type: encodeText, decodeText, createTempStream, signCentralDirectory and decryptCentralDirectory. A falsy value keeps meaning "use the default"
• The signal option throws the new ERR_INVALID_SIGNAL when it does not look like an AbortSignal, i.e. when it does not expose an addEventListener() method and a boolean aborted property. Duck-typed signals and signals coming from another realm keep working
• The password and rawPassword options are now checked on the reader side as well, throwing ERR_INVALID_PASSWORD_TYPE. A value of another type used to fail with the unrelated ERR_ENCRYPTED or ERR_INVALID_PASSWORD
• msdosAttributesRaw throws ERR_INVALID_MSDOS_ATTRIBUTES when the value is not an integer, and accepts a numeric string like the other numeric options. The range check used to be the only one, and the bitwise arithmetic folding the value into the external file attributes did the rest quietly: a fractional value was truncated, and a value that is not a number at all passed both comparisons and was written as 0. msdosAttributes throws ERR_INVALID_MSDOS_DATA on an array, which used to be accepted as an object and wrote 0 as well, since none of the flag properties exist on it
• configure() and setDefaultConfiguration() reject two kinds of bad input instead of storing them. maxWorkers must be an integer greater than 0 and throws the new ERR_INVALID_MAX_WORKERS otherwise: a value lower than 1 used to deadlock ZipWriter#add() for ever, since no entry could start and none could release the next one. createWorker and the CompressionStream and DecompressionStream options, including their *Fallback and deprecated *Zlib forms, must be functions and throw ERR_INVALID_FUNCTION_OPTION otherwise, a falsy value still meaning "use the default". The numeric options accept a numeric string and are coerced, like the numeric options of the reader and the writer. Nothing is stored unless the whole call passes, so a rejected call leaves the configuration untouched
• chunkSize is normalized wherever it is read. A value lower than 64 is raised to 64, as before, and a value that is not an integer greater than 0 now falls back to the default of 65536 instead of being used as it is. This applies to the global configuration and to the chunkSize option of Reader#createReadable()

The rest of this section changes results rather than rejecting input.

• The configuration is read when it is used instead of when the reader or the writer is constructed. A configure() call made between new ZipWriter() and the first add(), or between new ZipReader() and the first getData(), used to be ignored and is honored now. It affects maxWorkers, chunkSize, the compression stream implementations and the deflate support detection
• The executable option now counts as Unix metadata, like unixMode. It means a mode of 0o755, and it was the only Unix metadata option taking the MS-DOS branch, so an entry written with executable set next to msDosCompatible, msdosAttributes or msdosAttributesRaw lost its executable bit without a word. It wins over the three of them now, and selects the Unix platform for the "Version made by" field. executable set to false changes nothing, as before
• The entries of the filesystem API are dated when they are added, not when they are written. An entry added without a lastModDate option used to be stamped with the current date at export time, so exporting an unchanged tree twice produced different bytes. The four sources of the date of an exported entry now rank as follows, weakest first: the moment the entry was added, the date of the entry the tree was imported from, the lastModDate option passed to the export, and the lastModDate option passed when the entry was added
• The directories implied by the name of an imported entry are no longer written back when the tree is exported. Importing a zip file storing "a/b.txt" and no directory entry creates a navigable "a" entry, which used to be exported as an entry of its own, so a round trip gained one entry per path component. Only the directories carried by the source zip file and the ones created with addDirectory() are written now. getExportedSize() and the progress callbacks count them the same way
• The options passed when an entry was added no longer override the values describing the data of an entry exported with passThrough. compressionMethod, uncompressedSize and the other pass-through values now win over the per-entry options, as they already did over the options passed to the export
• The options passed when an entry is added are now copied. The object was stored as it was, so mutating it afterwards, or reusing one object for several entries and mutating it in between, changed entries that had already been added
• The split zip file signature at the start of a zip file is no longer reported as prepended data. A single-disk archive written by SplitDataWriter starts with the four bytes PK\x07\x08, which used to be read as prepended data by extractPrependedData and rejected as an ambiguous archive by strictness set to "strict" or by checkAmbiguity. The temporary spanning marker PK00, which PKZIP writes at the start of the first disk while a spanned archive is being created, is accepted the same way. If you read archives produced by SplitDataWriter with either of those options, they are no longer misreported

Bug fixes

• ZipWriter#prependZip() no longer copies the central directory of the zip file it prepends. The whole source was piped into the output, then the entries were rewritten after it, so every prepended archive carried its old central directory as dead bytes in the middle of the result. Only the data region is copied now, i.e. everything before the first byte of the central directory. Archives already produced this way are still readable, they are only larger than they need to be
• ZipWriter#prependZip() reads the zip file it prepends once instead of twice. A reader providing readUint8Array() was buffered into a Blob because the central directory was read through reader.readable, so a BlobReader or an HttpRangeReader was fully downloaded into memory before anything was written
• The transferred streams are no longer broken in a web worker installing a polyfill of the Streams API, the setup documented under Configuration#createWorker below. The worker received a native ReadableStream and a native WritableStream through postMessage() and piped them into streams of the polyfill, which rejected them. They go through the same compatibility wrappers as the streams created in the worker now

Documentation

• msdosAttributesRaw and msdosAttributes now describe the platform they select. The behavior is unchanged: setting either of them selects the MS-DOS platform for the entry exactly as msDosCompatible set to true does, and overrides that option when it is explicitly set to false, so versionMadeBy loses its Unix upper byte and no Unix mode is written. What counts is that the option is provided, not its value, so 0 and {} select it too. Any Unix metadata option wins over the three of them, with the MS-DOS attributes written into the low byte, see the executable change above
• ZipWriter#prependZip() now states that the data of the zip file is copied, its central directory rebuilt and its entries relocated, so the disks of a split zip file passed as input are unrelated to the disks of the output
• ZipReader#comment and the comment option of an entry now explain why one is bytes and the other a string. The encoding of the comment of an entry is recorded in its header by the general purpose bit 11, the encoding of the global comment is recorded nowhere, so it can only be decoded with the encoding agreed with the producer of the zip file
• Configuration#createWorker now documents how to install a polyfill of the Streams API in the scope of the worker, which is the way to run the web workers on the engines where TransformStream is missing from that scope, e.g. Firefox before version 102. A polyfill imported by the page does not help, because the worker reads the globals of the Streams API from its own scope
• Configuration#workerURI now states that the worker is created as a module worker, unless the URI is a Data URI or a Blob URI, in which case it is created as a classic worker
• ZipDirectoryEntryExportOptions now documents the precedence of the four sources of the last modification date of an exported entry, and ZipDirectoryEntry#importZip() documents that the directories implied by an entry name are not written back
• File#lastModified is now a link to MDN in the generated documentation, instead of an unresolved reference

Tests and continuous integration

• A new audit checks that every property name reaching the public API is either declared in index.d.ts or mangled on purpose. It reads the terser configuration and the declarations, walks the objects the library builds at runtime and reports the names belonging to neither list. It found eight internal fields shipped under their source names in the minified builds, e.g. the pending characters of Data64URIWriter and the source blob of BlobReader, which are mangled now, two of them renamed on the way. The parameter names of the declarations are no longer reserved either: a parameter name is not a property name, and reserving it kept a field of the same name readable in every build
• The internal methods writing the records of a zip file are renamed away from the DOM property names they shared, so that they are mangled instead of being kept by the terser reserved list
• Every public class is instantiated by the audit, so a member appearing only on an instance is covered. The classes reached only through a subclass, e.g. ZipEntry, are matched by walking the prototype chain
• The deprecated checkSignature option is covered by a test of its own, and the rest of the suite uses checkCrc32. The option had lost all its usages when the tests were swept, so nothing exercised it any more
• The zip.fs namespace is covered by a test of its own, for the same reason
• The polyfill of the Streams API in the worker is tested with a classic worker, a module worker and the native build, one test per build since the WASM worker and the native worker bundle different codecs. The web runner gained a nativeBuild feature probe and caches the build probe instead of running it per test
• New tests: the entries whose declared data extends past the end of the archive, the options of an entry of the filesystem API, the pass-through values against the per-entry options, the stability of the dates of an exported tree, the directories implied by an imported entry name, the encoding of TextWriter, the single read of prependZip(), the bytes it copies, its split zip file output, and the strictness of the split zip file signature
• An option validation assertion is split into a test file of its own, since it needs the signal option of pipeTo(), which Chrome 76 to 79 ignore. The runner skips the file there rather than reporting a failure
• The release is now scripted. npm run bump-patch bumps the version in package.json, package-lock.json and deno.json and commits it, the version script of npm syncing deno.json. A workflow step verifies that the three declared versions agree, and the release workflow verifies that the released tag matches the version declared in package.json

Credits

• Andrew Chin, Brian J Lee and Youngjoon Kim, SSLab at Georgia Tech, reported the declared uncompressed size of an entry driving the allocation of the output
• Claude (Opus 5) contributed to every change listed above

Full Changelog: v2.8.53...v2.8.54