1.3.0

This release makes some improvements to the Zip backend's internals.

Position independent substructures

Many of the classes that represent binary structures have been rewritten so that the dynamically sized data is fetched independently of the byteOffset of the class/structure instance. This should make the backend more resistant to byteOffset weirdness (like Buffers almost always having a different byteOffset from the position in a file). This is also a requirement for the next feature:

Custom Data Sources

The data option of ZipOptions now accepts a new interface, ZipDataSource, that allows you to specify an arbitrary function for fetching data. This can be used to lazily get data only when it is needed, or to work with APIs that don't allow retriving all of the data at once.

The simplified interface is:

interface ZipDataSource {
	size: number;
	get(offset: number, length: number): Uint8Array | Promise<Uint8Array>;
}

A good example of this is lazily reading from a file on Node.js:

const fd = openSync('your-archive.zip', 'r');
const { size } = fstatSync(fd);

await configureSingle({
	backend: Zip,
	data: {
		size,
		get(offset, length) {
			const data = new Uint8Array(length);
			const read = readSync(fd, data, { position: offset, length });
			return data;
		},
	},
});

Streams (#3)

It is now possible to stream a zip file by configuring data as fromStream(readableStream, sizeInBytes). In general this probably isn't super useful since streams will go from front to back, while zip files' metadata is at the very end.

Lazy mode

lazy was added to ZipOptions a year ago, but unfortunately wasn't implemented until now.

When you enable lazy, file contents will not be preloaded during configuration. This means you can't synchronously read until after having either performed an asynchronous read or waited long enough after a failed synchronous read that threw EAGAIN.

What that looks like in practice:

fs.readFileSync('/archive/xyz') // throws EAGAIN

// In a different program...

await fs.promises.readFileSync('archive/xyz') // cached after this
fs.readFileSync('/archive/xyz') // all good!

A few things to note:

• Reading any part of a file inside the zip archive will decompress that entire file
• Metadata like file names and comments are not affected by lazy mode
• After reading a file (including when sync. reads throw EAGAIN), the file contents are cached in memory (and available synchronously)