Skip to content

v1.1.0 — range access, reproducible-for-any-archive, issues #9–#12, supply-chain parity

Latest
Latest

Choose a tag to compare

View all tags
@Nizoka Nizoka released this 08 Oct 20:55
v1.1.0
1e2ab42
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

zipnative v1.1.0

Released 2026-10-08

The first minor since the freeze, and a statement about what a minor is: zero breaking changes. The public surface grew from 77 to 106 exports with none removed, the 39 error codes are unchanged, every new behaviour is opt-in, and the deterministic: true bytes are unchanged — proven by a byte-level baseline of the 33 pre-existing sample archives and by the previous release's own test suite running against the new sources in CI. Existing inputs produce byte-identical output, with one deliberate exception listed in the Upgrade section: dates outside the DOS range, whose previous output was wrong. The four issues reported by the satellites are closed at the source; the Zip64 streaming opt-in designed in 0.9 ships; five adoption features arrive — random access over remote and huge archives, cancellation and progress, raw transplant, canonicalisation of any archive, legacy name decoding — and the repository reaches hardening parity with pdfnative 1.8.0.

Highlights

  • Open a 50 GB archive with the memory of its central directory — openZipRange() reads an archive served by ranges (HTTP Range, Blob.slice(), a file handle, object storage) through a ByteRangeSource you inject; the engine never fetches, every limit and cross-check of openZip() applies, and a counting source shows one tail window, the central directory, then only the entry you read. Guide · playground.
  • Make any archive reproducible — analyzeDeterminism() names what would drift (timestamps, order, extra fields…); canonicalizeZip() and saveCompact({ canonical: true }) rewrite a Gradle, Maven, zip or Explorer archive into the canonical form without recompressing a payload, idempotently, under the frozen byte contract (golden SHA-256 hashes over two committed foreign fixtures); addRaw() / addFromReader() assemble artefacts from existing parts in O(bytes copied). Guide.
  • Issues #9–#12 closed at the source — UTC DOS timestamps and honest clamping (#9), one error classification for corrupt deflate on every tier (#10), custom codecs streamed by the forward reader through a new createDecompressor() contract (#11), verifyEntry().skipped with a real localHeaderMatch (#12). The satellites' workarounds become unnecessary.
  • Zip64 streaming opt-in — addStream(name, source, { zip64: true }) for entries that may exceed 4 GiB: the APPNOTE 4.3.9.2 layout, validated by the ISO/IEC 21320-1 gate and extracted byte-for-byte by the six-tool interop matrix; a real 4 GiB + 1 stream is exercised in the conformance workflow.
  • Supply-chain parity with pdfnative 1.8.0 — scripts/gate.ts profiles, harden-runner and --ignore-scripts on every job, OIDC publishing behind the npm-publish environment with SLSA Build L2 provenance and SBOM attestation, dependency review, weekly audit, committed rulesets, a byte baseline of every sample, 25 new verify-docs rules, release-prepare, and a compat-previous job that runs the 1.0.0 suite against 1.1.0.

Security

  • fix(security): verifyZip() / verifyEntry() no longer fabricate localHeaderMatch for encrypted entries (it was hard-coded true in one path and false in the other); the local header is cross-checked even when the payload cannot be verified, and an encrypted entry whose local header contradicts the central directory now fails the archive (CWE-1288, issue #12). Verdicts never loosened: unknown methods still fail the archive.
  • fix(security): corrupt deflate payloads surface as ZipFormatError ZIP_DEFLATE_CORRUPT / ZIP_DEFLATE_TRUNCATED on every inflate tier — node:zlib, DecompressionStream, pure TS — instead of a runtime-dependent class (CWE-703, issue #10); ZIP_DECOMPRESSION_FAILED is reserved for registered or injected codecs.
  • feat(security): encrypted entries are labelled by scheme — feature: 'aes' for WinZip AES (method 99 or the 0x9901 extra), 'zipcrypto', 'strong-encryption' — and still never decrypted in 1.x (CWE-327 context; AES is a 2.0 candidate).
  • feat(security): one new named bound, maxEntryCompressedSize (1 GiB + 1 MiB, CWE-770), bounds the byte-range reader's whole-payload fetches; the in-memory reader uses zero-copy views and never consults it, so no 1.0.0 archive is newly refused. A nameDecoder runs before path sanitisation, never around it (CWE-22); a lying byte-range source is ZIP_RECORD_TRUNCATED (CWE-20).
  • feat(ci): every workflow starts with step-security/harden-runner and installs with npm ci --ignore-scripts (.npmrc makes that the local default); actions stay SHA-pinned; dependency-review.yml (high severity, licence allow-list) and a weekly audit.yml; publish.yml is three jobs with one artifact between them: verify (no publishing token; fetches the previous tag, runs the full publish gate, packs the tarball once with its SBOM and SHA256SUMS) → publish (the npm-publish environment, id-token: write, a pinned npm ≥ 11.5.1, a sparse checkout of .nvmrc only, the handed-on tarball digest-checked then published through OIDC with provenance) → attest (the same tarball compared byte for byte with what the registry serves, then SLSA Build L2 provenance for it and the CycloneDX SBOM through actions/attest-build-provenance, attached to the GitHub Release); no release job restores a dependency cache; a workflow_dispatch dry run stops at npm publish --dry-run; branch and tag rulesets committed under .github/rulesets/.
  • fix(deps): npm audit --audit-level=high is green again — adm-zip 0.6.1, brace-expansion 1.1.21 / 5.0.12 and source-map-js 1.2.2 (dev dependencies only; the engine has none).

Added

  • feat(parser): openZipRange(source, options?), ByteRangeSource, ZipRangeReader, rangeSourceFromBytes() — random access over an injected byte-range source; shared per-entry defences with openZip() (src/parser/zip-entry-checks.ts); 256 KiB ranged streaming through createDecompressor(); the cached tail window serves the central directory and small archives without a round trip, and local headers are prefetched with 1 KiB of slack.
  • feat(core,parser): signal?: AbortSignal and onProgress?: ZipProgressHandler on every asynchronous call — on the options of extractZipStream, iterateZipEntries, openZip / openZipRange (every read of the reader), createZip / createParallelZip (every stream() and toBytes()), and per call on readEntryStream(entry, options) and stream(options); abort rejects with signal.reason after resources are released (stream locks released without cancel, worker pools closed); ZipProgress is monotonic, a skipped entry counts as done, the parallel toBytes() reports per settled entry.
  • feat(core): ZipWriter.addRaw(name, payload, meta) and addFromReader(reader, entry, options?) — transplant without recompression; addFromReader verifies the source entry first unless verify: false; RawEntryMeta, RawEntryReader, AddFromReaderOptions; the modifier's raw-copy plan moved to core/zip-segments.ts so both writers share one implementation.
  • feat(parser): canonicalizeZip(bytes, options?), CanonicalizeOptions; ZipModifier.saveCompact({ canonical }) with CanonicalOptions / CompactOptions; analyzeDeterminism(bytes) with DeterminismReport, DeterminismOffender, DeterminismConcern — the verdict the satellites re-derived.
  • feat(core,parser): nameDecoder option on every read path (applied when flag bit 11 is clear, and as the fallback when a UTF-8-flagged name is not valid UTF-8; entry.nameEncoding becomes 'custom'); getExtendedTimestamps(entry) (UT 0x5455, NTFS 0x000a, DOS fallback — ExtendedTimestamps) and getUnixIds(entry) (ux 0x7875 — UnixIds); externalAttributesFromUnixMode(mode, options?), the write-side mirror of getUnixMode.
  • feat(core): dosTimeMode: 'local' | 'utc' (DosTimeMode) on every option bag that writes or reads lastModified; full DOS-date clamping with the new ZIP_TIMESTAMP_CLAMPED diagnostic (the twelfth; issue #9).
  • feat(codecs): ZipCodec.createDecompressor?(maxOutput) → ZipDecompressor { push, end } — the incremental contract the forward reader streams in O(chunk); the built-in store and deflate codecs implement it (issue #11).
  • feat(core): AddEntryOptions.zip64 — the per-entry Zip64 streaming opt-in for addStream(): version 45 in both records, sentinel sizes and a (0, 0) Zip64 extra in the local header (lfhZip64Fields moved to core/zip-extra-fields.ts), 24-byte data descriptor, both sizes in the central record; assertStreamSizesInRange names the option in its refusal.
  • feat(parser): EntryVerification.skipped?: EntrySkipReason ('encrypted' | 'stream-only-codec' | 'unsupported-method'); verifyZip() delegates every entry to verifyEntry() (issue #12).
  • feat(samples): five samples — zip64/zip64-streamed, incremental/merged-from-two, names-encoding/shift-jis-legacy (an expected ISO non-conformant: non-ASCII names without bit 11), attributes/unix-ids, attributes/ntfs-timestamps — 38 in the corpus, 33/5 in the ISO canary; two interop write cases (zip64-streamed, transplanted) — 19 validation cases (6 producer reads + 13 write cases, each run against every extractor present).
  • feat(tooling): scripts/gate.ts (--fast / --ci / --publish, --require-all, --only, --from, --json, dist probes), scripts/verify-samples.ts + tests/regression/samples.test.ts + tests/regression/baselines/samples.sha256.json (SHA-256 of every sample with a since chain), scripts/release-prepare.ts (the bump in one pass), scripts/run-compat.ts (compat:previous), scripts/build-claude-rules.ts (agents:rules), scripts/install-git-hooks.mjs (opt-in pre-commit / pre-push), scripts/lib/prose-language.ts, scripts/lib/agent-config.ts; tests/tools/ (workflows, gate, guard, agent-config, verify-issue, release-prepare, prose-language, api-compat).
  • feat(docs): 25 verify-docs rules (52 total, catalogued by --rules): no-control-bytes, playground-syntax, export-named, limits-table, since-tags, derived-counts, count-tokens, version-token, errors-verified-on, sample-regression, prose-language, claude-md-budget, governance-sources, node-pin-parity, ruleset-parity, agent-config-parity, claude-rules-sync, claude-rules-budget, pr-template-parity, eol-lf, skills-shape, sitemap-lastmod-vs-git, bench-parity, rules-list, and a since check in surfaces-shape; docs/assets/ecosystem.json carries derived.* figures recomputed from the tree and declared.tests / coverage held to the last gate run.
  • feat(docs): two guides — Large and remote archives, Reproducible builds; the remote range access playground (a dropped file read by Blob.slice() or a URL served with HTTP Range, with a live read counter and Abort); four recipes (remote-range, determinism-report, zip64-stream, utc-dates); nine 1.1.0 capabilities in docs/data/surfaces.json and the choose guide; the inspector shows the determinism verdict and extended timestamps, the modifier offers canonical compaction, the streaming page demonstrates progress and cancellation, the toolkit pins dates with dosTimeMode.
  • feat(governance): CLAUDE.md, .claude/settings.json (deny list, HITL command denies), .claude/hooks/guard.mjs, .claude/rules/ rendered from .github/instructions/, the release-audit skill; .github/AGENT_RULES.md and ai-governance.json on the pdfnative schema; .github/ISSUE_TEMPLATE/maintenance.md, .github/prompts/compliance-audit.prompt.md, release-notes/PR_TEMPLATE.md; CODEOWNERS; .nvmrc / .node-version / .npmrc.

Changed

  • fix(codecs): corrupt or truncated deflate payloads raise ZIP_DEFLATE_CORRUPT / ZIP_DEFLATE_TRUNCATED (ZipFormatError) on the node:zlib and DecompressionStream tiers too, through normalizeInflateError — see the ledger (C2). getCodec(8).compressSync with an invalid level is ZIP_INVALID_OPTION on every tier.
  • fix(core): dates below 1980-01-01 clamp to the DOS epoch, dates above 2107-12-31 23:59:58 to that maximum, and an invalid Date to the epoch, each with ZIP_TIMESTAMP_CLAMPED — previously 1975-07-15 became 1980-07-15 and NaN produced a silent zero field (C1).
  • fix(parser): verifyZip() computes localHeaderMatch for every entry, encrypted ones included (C3); unknown methods are reported skipped: 'unsupported-method' while still failing the archive, as before (C4).
  • chore(parser): ZipUnsupportedFeature gains 'aes', ZipEntry.nameEncoding gains 'custom', ZipDiagnosticCode gains ZIP_TIMESTAMP_CLAMPED (C5); the messages of ZIP_UNSUPPORTED_ZIP64_STREAMING and ZIP_UNSUPPORTED_CODEC_MODE name the new remedies (C8).
  • chore(tooling): scripts/validate-zip.ts keys the data-descriptor width on the local header's Zip64 extra (APPNOTE 4.3.9.2) rather than on the central record (WF/DESCRIPTOR-WIDTH); conformance.yml runs test:generate, verify:samples, validate:zip and test:interop through the gate on Linux and Windows and the 4 GiB streaming test behind ZIPNATIVE_BIG_TESTS; ci.yml gains the os (Windows, macOS) and compat-previous jobs and drops its path filter; sample-regression.yml, dependency-review.yml, audit.yml; vitest.config.ts pins TZ=UTC and pool: 'forks'.
  • docs: recipes/custom-codec.ts demonstrates method 97 (99 is WinZip AES — C9); docs/data/errors.json re-audited for nine codes; ROADMAP's post-1.0 items resolved (Zip64 opt-in shipped, AES moved to the 2.0 candidates); SECURITY.md gains the supply-chain section, the 1.1.x support row and three threat rows; AGENTS.md fits the 120-line budget; the homepage, README, llms.txt and the agent brief describe 1.1.

Fixed

  • #9 — DOS timestamps depended on the process time zone and clamped only the year: dosTimeMode: 'utc' on the write and the read side, full clamping with a diagnostic (#9).
  • #10 — node:zlib and DecompressionStream errors leaked or surfaced under a different class than the pure tier: normalised to ZIP_DEFLATE_* on every tier, four-tier parity test (#10).
  • #11 — iterateZipEntries() ignored registered codecs: codecs exposing createDecompressor() stream in O(chunk); others are refused before the first byte with ZIP_UNSUPPORTED_CODEC_MODE and skip() keeps working (#11).
  • #12 — verifyEntry() reported encrypted and stream-only entries as failed without a reason and with a fabricated localHeaderMatch: skipped names the reason, the local header is always really cross-checked (#12).
  • Final audit (2026-10-08, second pass) — canonicalizeZip() and analyzeDeterminism() forward nameDecoder (a Shift-JIS name without bit 11 was frozen as CP437 mojibake, re-encoded as UTF-8 with bit 11 set) and signal / onProgress; a pre-aborted signal no longer leaves a deflate worker pool behind (the pool is created after the check and inside the try/finally); the parallel writer reports one monotonic ZipProgress series per call (toBytes() ended on a fresh tracker at 0 / null); every synchronous entry point taking ZipCommonOptions (openZip and readEntry, createZipModifier and its save() / saveCompact(), canonicalizeZip, analyzeDeterminism) checks signal once on entry, as its TSDoc said.

Install

npm install zipnative@1.1.0

Upgrade

No breaking changes. Drop-in replacement for 1.0.0. The compatibility ledger — every observable change, its class and the guard that proves it:

# Change Class Guard
C1 Dates below 1980 / above 2107 clamp correctly (1975-07-15 → 1980-01-01, not 1980-07-15); an invalid Date becomes the DOS epoch; both emit ZIP_TIMESTAMP_CLAMPED (info; fatal under strict) bug fix — bytes change only for inputs whose previous output was wrong; no sample or fixture affected sample baseline (33/33 unchanged), tests/core/zip-dos-time*.test.ts, 12 diagnostics (additive)
C2 On the node:zlib and DecompressionStream tiers a corrupt payload raises ZipFormatError ZIP_DEFLATE_CORRUPT / ZIP_DEFLATE_TRUNCATED instead of ZipDataError ZIP_DECOMPRESSION_FAILED classification fix — the pure tier and browsers already raised ZIP_DEFLATE_*; both codes exist since 0.8.0 and the satellites map both to E_DATA tests/codecs/inflate-tier-parity.test.ts, errors.json
C3 Encrypted entries get a real localHeaderMatch; an encrypted entry whose local header contradicts the central directory now fails the archive tightening — the 1.0.0 value was fabricated; never a looser verdict tests/parser/zip-verify-reasons.test.ts, compat-previous
C4 Unknown methods report skipped: 'unsupported-method' and still fail the archive, as in 1.0.0 additive tests/parser/zip-verify-reasons.test.ts
C5 Unions widened: ZipUnsupportedFeature + 'aes', EntrySkipReason (new) with 'unsupported-method', ZipEntry.nameEncoding + 'custom', ZipDiagnosticCode + ZIP_TIMESTAMP_CLAMPED additive (an exhaustive consumer switch adds a case) tests/tools/api-compat.test.ts
C6 Interfaces extended: ZipWriter (+ addRaw, addFromReader), ZipModifier.saveCompact(options?), ZipCodec.createDecompressor?, ZipCommonOptions (+ dosTimeMode, signal, onProgress, nameDecoder), AddEntryOptions.zip64?, ZipLimits.maxEntryCompressedSize, AnalyzeDeterminismOptions (+ nameDecoder, signal), StreamControl.tracker (@internal) additive — factory-returned objects; a hand-written complete ZipLimits literal must add the new field tests/tools/api-compat.test.ts (77/77 signatures, VERSION aside)
C7 maxEntryCompressedSize is consulted only where an allocation from an untrusted length exists (openZipRange); the in-memory reader never refuses a 1.0.0 archive on it additive tests/parser/zip-range-reader.test.ts
C8 ZIP_UNSUPPORTED_ZIP64_STREAMING and ZIP_UNSUPPORTED_CODEC_MODE messages reworded to name the new remedies outside the contract ("branch on err.code, never on the message") —
C9 recipes/custom-codec.ts uses method 97 instead of 99 docs (99 is WinZip AES) tests/docs/recipes.test.ts
C10 Tooling: validate-zip descriptor width, verify-docs rules, workflows (publish.yml in three jobs), .npmrc, vitest.config.ts (TZ=UTC, forks, json-summary, thresholds 88 / 80 / 85 / 90), eol-lf fails on CRLF outside the API tests/tools/*

Sample rebaseline: none of the 33 pre-existing entries changed; the five new samples join tests/regression/baselines/samples.sha256.json with since: "1.1.0".

Known limitations

  • A stream that crosses 4 GiB without { zip64: true } is still refused mid-output (ZIP_UNSUPPORTED_ZIP64_STREAMING, before the crossing chunk); a consumer that started reading the stream sees it end early.
  • An opted-in entry that stays below 4 GiB carries Zip64 records; streaming readers that size the data descriptor from the measured lengths (Java's ZipInputStream) cannot read it — inherent to any speculative Zip64 layout, hence the per-entry opt-in. Every random-access extractor of the interop matrix reads it.
  • deterministic: true buffers each stream entry through the pinned encoder and caps it at 2 GiB (ZIP_INPUT_TOO_LARGE); pure-TS streaming deflate is a 1.2 candidate.
  • The forward reader still refuses store + bit 3, encrypted + bit 3 and custom codecs + bit 3 (not self-delimiting without the central directory).
  • AES-encrypted entries are labelled (feature: 'aes'), never decrypted; AES is a 2.0 candidate.
  • addFromReader() takes the in-memory reader; from a ZipRangeReader use await readEntryRaw() then addRaw().
  • nameDecoder relies on the caller's TextDecoder labels; a Node built with small-icu lacks most legacy code pages.
  • The external validators of the conformance gate (unzip, 7-Zip, bsdtar, Python, jar, Expand-Archive) are the runner image's; their versions are recorded in the job summary, not pinned by checksum.
  • The worker playground remains absent: the CDN-served worker script is cross-origin for a same-origin Worker; a same-origin Blob shim is a 1.2 candidate (ROADMAP).

Downstream integration notes

Both satellites pin zipnative ^1.0.0; 1.1.0 is a drop-in. Bumping to ^1.1.0 lets each delete a workaround:

  • zipnative-cli — remove ZLIB_TO_ZIP (src/utils/ziperr.ts; the engine normalises every tier, so the doc comment saying zlib errors "reach the CLI unwrapped" is obsolete); parseIsoDateUtc (src/utils/zipops.ts) can pass dosTimeMode: 'utc' instead of recomposing dates; verify --entry and modify's survivor check read skipped instead of re-deriving it (and stop attributing an unknown method to ZIP_CRC_MISMATCH); stream --skip-unsupported now works for custom codecs; inspect --check deterministic can delegate to analyzeDeterminism(); the Unix-mode helper (zipops.ts) is externalAttributesFromUnixMode(); docs/KNOWLEDGE_BASE.md's "engine limitation" entries for #10 (the zlib map) and #11 (forward reader + custom codecs) go. Candidates: --url (range source), --timeout + progress bar, merge / split / canonicalize, --name-encoding, --zip64.
  • zipnative-mcp — entry-spec.ts does not compensate for the process time zone and server.ts declares those calls cacheable: pass dosTimeMode: 'utc'; read_zip_entry mode: 'verify' and verify_zip relay skipped; inspect_zip delegates to analyzeDeterminism(); drop the zlib error map. Candidates: a remote-archive resource (openZipRange), merge_zip, canonicalize_zip, a nameEncoding parameter.
  • Both — map the new union members (feature: 'aes', skipped: 'unsupported-method', nameEncoding: 'custom'); no error code was added, so the E_* tables stay as they are. Machine-readable inputs: docs/assets/api.json (106 exports), docs/data/errors.json (re-audited), docs/data/surfaces.json (nine 1.1.0 capabilities with since).

Links

Assets 4
Loading