Repository navigation
Releases
v1.2.0
Additive over 1.1.1: no public API was removed or changed, and compressed bytes are unchanged for every codec and level. Zip archives change only in the local headers of zip64 entries whose sizes follow in a data descriptor, and in raw-copied entries written without a "version needed to extract" (see Fixed).
Added
- Zip raw entry copy, for callers that carry untouched parts from one archive to another without decompressing and recompressing them:
ZipReader.OpenRawEntryStream(ZipEntry)returns an entry's stored bytes (compressed data plus any ZipCrypto header or WinZip AES salt, verifier and authentication code) with no decoding, decryption or CRC check, in seekable,LazyDirectoryand forward-only modes (forward-only needs the sizes in the local header and otherwise raisesArchiveUnsupportedException);ZipWriter.WriteRawEntry/WriteRawEntryAsyncwithZipRawEntry(andZipRawEntry.FromEntry) write an entry from a header and such a stream, rewriting zip64 fields, dropping stale alignment padding and checking that the stream holds exactly the declared compressed size.ZipEntrygainedRawComment,DosTime,DosDateandUncompressedSize. Seedocs/zip.md. - The
zip-raw-copyfuzz target.
Fixed
- A raw-copied zip entry whose
ZipRawEntry.VersionNeededwas not set got "version needed to extract" 0, which some readers treat as damage.VersionNeededis nowushort?: null (the default) picks the value from the method as the ordinary writer does (10 stored, 20 deflate, at least 20 for directories), and an explicit value is still written as given. - Zip64 local headers of entries whose sizes follow in a data descriptor now carry zero sizes instead of 0xFFFFFFFF (APPNOTE 4.4.4); Excel refuses the latter. The zip64 extra field stays, with zeroed sizes, and the descriptor still holds 8-byte sizes.
- A disposed
ZipReaderno longer keeps its central directory alive, so anArchive.EnumerateEntries()sequence held after the loop does not pin it. - Documentation:
Archive.EnumerateEntries()keeps memory flat for a zip only withLazyDirectory(the 1.1.1 note read as if it did on its own).
Changed
- The SDK pin in
global.jsonis 10.0.403 (a runtime patch level that AOT restore needs).
v1.1.1
Additive over 1.1.0; compressed bytes are unchanged for every codec and level.
Added
Archive.EnumerateEntries(): lists entries one at a time without keeping them, so memory stays flat on archives with millions of entries (Archive.Entriesbuilds a list and grows with the count; its documentation now says so). The large-archive example indocs/archive.mduses it.- Extraction of a damaged compound file (opened with
Recovery):ArchiveExtractOptions.DamagedEntries(DamagedEntryPolicy.Extract,Skip,Fail) andOnDamagedEntry, andArchiveExtractResult.EntriesDamagedandDamagedEntryNames. The default still writes the shortened streams, as in 1.1.0, but the result now reports them. CompressionLimitException(format, limitName, limit, total, runningTotal)for limits checked against a running total.
Fixed
- The
MaxTotalSizemessage no longer reads as if the archive needs the reported number: it says the total reached that value when it passed the limit. The exception type and its properties are unchanged. - README and docs: status line, when the speed rows were measured, and a stale "stay preview" remark on zstd levels 16 to 22.
v1.1.0
Additive over 1.0.0: no public API was removed or changed, and compressed bytes are unchanged for every codec and level. Every new mode is off by default.
Added
ArchiveLimits.LargeArchive(4,000,000 entries, 256 GiB declared size, every other limit at its default) and a "Large archives" section indocs/archive.md: withZipReaderOptions.LazyDirectorya million-entry mail export opens, lists and extracts. The defaults are unchanged.- Zip lazy central directory (
ZipReaderOptions.LazyDirectory, also throughArchiveOpenOptions.Zip): a seekable zip is enumerated record by record without holding the directory, so memory no longer grows with the entry count (a million-entry archive: 630 MiB held by the default reader, under 1 MiB by the lazy one), andOpenEntryStream(entry)seeks to any entry the reader returned earlier. The directory is validated before the first entry, with the same overlap and shared-header checks and the same errors;MaxCentralDirectorySizedoes not apply, and a symbolic link's target is read when its entry is handed out. Off by default; the default mode is unchanged. Thezip-read-lazyfuzz target covers it. - Compound file recovery mode (
CompoundFileReaderOptions.Recovery,MaxRecoveryBytes;CompoundFileReader.Damage,IsRecovered;CfbDamage,CfbDamageKind;CfbEntry.IsDamaged,DeclaredSize), also reachable throughArchiveOpenOptions.CompoundFile. A file the strict reader refuses is opened anyway when it can be: a cut-short file, FAT/DIFAT/mini FAT entries that point outside the file, chains that loop or end early, a lost directory start, bad directory entries and links (unreachable entries are attached to the root), header fields that disagree with the file. Streams that cannot be followed to their declared size are shortened and flagged; every problem is listed inDamage. Off by default; reading is unchanged without it, and a file that passes the strict checks reads the same either way. Thecfb-recoverfuzz target covers it.
Changed
- Extraction no longer remembers every extracted file: only formats that can carry hard links (tar, RAR, cpio) keep file paths, the checked-directory cache is capped, and directories without permissions or times to restore are not queued. Extracting a million-entry zip went from 312 MiB held (369 MiB peak working set) to under 1 MiB held (94 MiB, the runtime's baseline). No safety check changed.
v1.0.0
The first stable release. Both packages leave preview: [Experimental] is removed from every public type, so the ONECOMPRESS001-ONECOMPRESS010 and ONEARCHIVE001 diagnostics no longer exist and the NoWarn entries that opted in can be deleted. From 1.0 the API follows semantic versioning and compressed bytes per codec and level are stable within 1.x (README, Stability). The README speed tables are remeasured against the fastest managed .NET libraries on an AMD Ryzen 9 7950X3D and an Apple M2; the two rows still below the 0.87 bar are marked there.
Added
- Zip recovery mode (
ZipReaderOptions.Recovery,MaxRecoveryBytes,ZipReader.Damage,ZipDamage,ZipDamageKind): resynchronises on the next local header after garbage or a damaged header, falls back to the local headers when the end record or central directory is missing or inconsistent, and reports skipped ranges and damaged or truncated entries. Off by default; reading is unchanged without it. Thezip-recoverfuzz target covers it. - Documentation: a README on the family shape with compiled snippets, a docs index with one page per codec family and container, package readmes,
THIRD-PARTY-NOTICES.md, and XML documentation on every public member. Package validation against 0.7.0-preview.1 runs on every pack. bench/Managed: size-matched comparison rows (a row is met when some level of ours is within +1% of the library's size at no less than 0.87 of its speed), small-payload dictionary rows and checksum rows.
Fixed
- LZMA2:
Lzma2Encoder.Flushwith a binary-tree match finder could write a stream that decoded to different bytes; fixed.
Changed
These speed-ups change compressed bytes at the levels listed (sizes stay within the bar's +1% of the managed libraries; bytes remain identical on every platform and thread count). Other levels write the same bytes as 0.7.0-preview.1.
- Zstandard: faster levels 1-4 (leaner match loops, one-read code tables, search state computed once per block); levels 3-9 (shallower search at 7 and 9, faster greedy level 5, one cut per block with long-distance matching, cheaper block cuts); levels 13-18 (insert-only tree walk, thinner insertions, prefetching in the optimal parser, shallower dense tail at 13, wider at 18); and levels 1-4 with a dictionary, which probe the dictionary at every second or fourth position since the last match and extend matches backward. Compression with a dictionary on small payloads (game snapshot data) is considerably faster. The decoder loops are compiled fully optimized on first use.
- LZMA and xz: faster hash-chain encoder with identical bytes for the same search (batched position insertion, bucket prefetch, branch-free bit coding, wider match compares). Preset 0 uses the 4-byte hash chain, presets 0 and 1 skip the look-ahead for matches of 3 and 4 bytes or more, and preset 2 searches 12 candidates, so bytes at presets 0-2 differ (preset 1 was later retuned again, see below).
- Brotli: faster decode (UTF-8 context table, wider root table, fewer refills). Quality 3 is a four-way greedy parse, quality 4 a single-block-type lazy parse, qualities 1 and 2 are retuned, and qualities 10 and 11 price literals in UTF-8 context, skip the dictionary search under a long tree match, and use finer block splitting (four refinement passes at 11), so bytes at q1-q4, q10 and q11 differ.
- xz/LZMA preset 1 searches less and takes 3-byte matches at once: faster, output differs at preset 1 only.
- LZ4: decoding with a dictionary reads the dictionary in place, keeps its history between blocks and copies short pieces exactly, which makes dictionary decode of small blocks about as fast as the fastest managed library. Compressed bytes are unchanged.
- Timestamps in ar, cpio and ISO 9660 headers that cannot be represented are reported as unknown (null) instead of throwing, found by fuzzing.
- Benchmarks time every candidate in fully optimized code from the first call.
v0.7.0-preview.1
Everything is still preview and carries [Experimental]. No codec changed since 0.5.1-preview.1, so the README measurements stand. Version 0.6 is reserved for RAR compressed entries.
Added
- Microsoft compound file writer (
CompoundFileWriter,CompoundFileWriterOptions): versions 3 and 4, FAT, DIFAT, mini stream, red-black directory trees, CLSIDs, state bits and times, reachable asArchive.Create("x.cfb")(docs/cfb.md).
v0.5.1-preview.1
Everything is still preview and carries [Experimental]. The README has the new measurements from both machines.
Changed
- Deflate, zlib and gzip: faster one-shot and streaming compression at levels 1-9. The greedy and lazy parsers no longer store literals and the block writer emits literal runs straight from the input (identical bytes by itself); levels 1-3 and 5-9 use shallower hash-chain searches (level 1 enters fewer positions of long matches, levels 6-9 also give up on the lazy look-ahead sooner), so their compressed bytes differ from 0.5 while staying within 1% of libdeflate's size. Levels 0, 4 and 10-12 and Deflate64 are unchanged.
- Zstandard: levels 5-9 (and 6-8 for sources of 128-256 KiB) enter only every fourth position that a long match skips over into the match table, which saves writes to a table far larger than the cache. Compressed bytes change, within 0.2% in size, and stay identical on every platform.
- LZ4: faster high-compression levels (3-12) through a tighter search, and a lazy level 2 that is within 0.2% of the
lz4tool's size (it was +2.2%). Levels 3-9 search deeper where no match is known yet and only briefly for a longer match one or two bytes later, and take a match of 64 bytes or more without looking further; this makes them 8-19% faster at about the same size. Levels 2-9 are retuned, so their compressed bytes differ from 0.5 while staying within 1% of the tool's size; levels 1 and 10-12 are unchanged. - Zstandard: levels 1-2 compress 3-11% faster (most on text), with identical output.
- Zstandard dictionary training: COVER and fast-COVER training is 3-9 times faster and uses 4-12 times less memory (a 1024 x 8 KiB sample set trains in under a second in under 100 MiB, from 6.6 s and 1.2 GiB). D-mers are counted once per size and the parameter search is guided; sets up to 256 KiB are searched exhaustively. The new
SearchModeoption selects the search. Trained dictionaries differ from 0.5 (still identical on every platform and thread count) and compress held-out game data within about 1% of the previous ones and ofzstd --train(the worst case is +1.2% against the tool, at a 4 KiB dictionary). - Zstandard: levels 10-15 compress 25-45% faster. Levels 10-12 search one step less deep than the
zstdtool, so their bytes change (within 0.6% in size); levels 13-15 are unchanged.
Added
- Microsoft compound file reader (
CompoundFileReader,ArchiveFormat.CompoundFile): .doc, .xls, .ppt and .msg containers, versions 3 and 4, with strict validation andArchivesupport (docs/cfb.md). bench/Managed: compares each codec with the fastest managed .NET implementations (ZstdSharp, K4os, SharpCompress and others), reporting speed and size per row.THIRD-PARTY-NOTICES.md: an entry for the RFC 8878 code tables used by the Zstandard codec.
v0.5.0-preview.1
Everything is still preview and carries [Experimental]. The README has the new measurements from both machines.
Changed
- Deflate64: a fast decode loop; decompression now runs at 2.0-2.6 times the speed of 7-Zip's on both machines.
- PPMd: encoding and decoding are 1.4-1.6 times faster, with identical output. Decoding now runs at 0.89-0.95 of 7-Zip's speed and encoding at 0.85-0.91.
- LZMA, LZMA2, xz and lzip: faster decoding (fewer unpredictable branches per symbol), with identical output.
- Brotli: faster encoding at qualities 0-5 (quality 0 is about 1.5 times as fast as in 0.4). Qualities 0-2 now find matches and emit symbols in one pass, so their compressed bytes differ from 0.4 while staying within 1% of the
brotlitool's size; qualities 3-11 are unchanged. - Zstandard: levels 1-4 compress 2-10% faster, with identical output.
Fixed
- Zstandard: negative compression levels could write invalid frames on large inputs. A sequence whose state and extra bits filled the encoder's 64-bit bit container exactly was written twice, so the
zstdtool and this library's decoder rejected the frame or decoded garbage. The same defect could in principle affect any level; it showed first at negative levels on the Silesia corpus. Frames that were valid are unchanged byte for byte.
v0.4.0-preview.1
Everything new here is preview and carries [Experimental]. The README has the measurements against the reference tools on both machines, misses included.
Added
- PPMd (
ONECOMPRESS010): variants H and I asPpmdH/PpmdI, encoders, decoders andPpmdStream, byte-identical to 7-Zip's output; zip method 98 and the 7z PPMD coder, read and write. The codec is ported from the public-domain reference sources (seeTHIRD-PARTY-NOTICES.md). Levels 8 and 9 use 128 and 256 MiB of model memory, as 7-Zip does on large inputs. New limitArchiveLimits.MaxPpmdMemorySize(256 MiB). Seedocs/ppmd.md. - ISO 9660 read and write, with Joliet and Rock Ridge (
docs/iso.md). - RAR 5 (read only): list and metadata, stored entries, multi-volume sets; compressed entries raise a typed error. RAR 4 archives and header-encrypted archives are also refused with a typed error.
- Codec registration for the archive layer: by default every codec is available, and an app that wants a smaller trimmed or NativeAOT binary sets the
RecursiveAI.OneArchive.AllCodecsfeature switch to false and registers only the codecs it uses withArchiveCodecs. A missing codec raisesArchiveUnsupportedExceptionnaming the format, the method and the call to add. A.tar.zst-only NativeAOT app shrank from 3.82 to 3.07 MB, without the brotli dictionary (docs/archive-codecs.md). - Benchmarks against the reference tools for brotli, bzip2 compress, Deflate64, LZX and PPMd.
Changed
- Brotli: faster decoder (a fast command loop) and a much faster encoder at every quality: a bucketed match finder, a fused meta-block writer for qualities 0-5, smaller tables. Qualities 0-9 are still below the tool's speed (see the README). Compressed bytes at qualities 0-9 differ from 0.3, still identical on every platform.
- bzip2 encoder: about twice as fast (two-phase block sort, vector move-to-front, incremental table choice); on x64 it now runs at 1.11-1.14 of
bzip2. Compressed bytes are unchanged. - Deflate64 encoder: levels 5-9 use an optimal parse with block splitting, and levels 1-4 share one fast class; output is smaller at levels 4-9.
- Deflate64, bzip2, PPMd, AES and ZipCrypto entries in zip, and the 7z Deflate64, PPMd and AES-256 coders (including encrypted headers), are covered by interop tests with 7-Zip and Info-ZIP. AES needs
System.Security.Cryptography.Aes, which the browser does not provide; there only ZipCrypto reading works.
Not supported
- BCJ2 in 7z is listed but refused with a typed "unsupported method" error. Quantum cabinets and cabinets that span files are refused with a typed error.
v0.3.0-preview.1
Everything new here is preview and carries [Experimental]. The LZMA family meets the size target and is below the reference speed in some rows; the README has the measurements from both machines.
Added
- Brotli (RFC 7932,
ONECOMPRESS008): decoder with the built-in dictionary and transforms, and encoder at qualities 0-11 (windows 10-24), in three layers. Single streams.brand.tar.brin the archive layer, detected by file name (brotli streams have no magic number). - bzip2 encoder at levels 1-9 (
ONECOMPRESS004), zip method 12 writing, and.bz2/.tar.bz2writing. - Deflate64 decoder and encoder (
ONECOMPRESS003),Deflate64Stream, zip method 9 read and write, and the 7z Deflate64 coder. - LZX decoder (
ONECOMPRESS009) with verbatim, aligned and uncompressed blocks and call translation. - 7z writing: solid and non-solid folders, LZMA2, LZMA, Copy, Deflate and Zstandard coders, the branch and delta filters, and encoded headers.
- cpio (newc, crc and odc) and ar (common, GNU and BSD) read and write.
- cab read: stored, MSZIP and LZX folders, every header field, checksums, and a typed error for spanned cabinets and Quantum.
- xz and lzip single streams:
.xz,.lz,.tar.xzand.tar.lz, read and write. - Encryption (
ONEARCHIVE001): zip WinZip AES-128/192/256 (AE-1 and AE-2) read and write, traditional zip encryption read; 7z AES-256 read and write, with encrypted headers.ZipReaderOptions.Password,ZipWriterOptions.Password,SevenZipReaderOptions.Password,SevenZipWriterOptions.Password, andPasswordonArchiveOpenOptionsandArchiveCreateOptions. A missing or wrong password is anArchivePasswordException. AES needsSystem.Security.Cryptography.Aes, which the browser does not provide.
Changed
- LZMA-family speed: decoder fast paths and match-finder prefetching. Deflate speed: a faster decoder loop and table build, and faster encoder levels 1-7. Deflate levels 2-4 now hash four bytes, so their output differs from 0.2 (still identical on every platform); other compressed bytes are unchanged.
- 7z reading: empty-stream vectors in the order the format defines, and coders without streams.
Not supported
- PPMd (zip method 98, 7z PPMD) and BCJ2 are listed but refused with a typed "unsupported method" error.
v0.2.0-preview.1
Everything new here is preview and carries [Experimental]. Deflate, zlib and gzip meet the size target and are below the reference speed; the bzip2 decoder meets both. The README has the measurements.
Added
- Deflate (RFC 1951), zlib (RFC 1950) and gzip (RFC 1952): decoder and encoder at levels 0-12 with strategies, preset dictionaries and flushes, in three layers (
ONECOMPRESS003). - bzip2 decoder: multi-stream and randomized blocks, with limits for hostile input (
ONECOMPRESS004). - LZMA and LZMA2 (
ONECOMPRESS005), xz with CRC32, CRC64 and SHA-256 checks, the branch filters (x86, ARM, ARM-Thumb, ARM64, PowerPC, SPARC, IA-64, RISC-V) and Delta (ONECOMPRESS006), and lzip (ONECOMPRESS007): decoders and encoders at every preset. - OneArchive (
ONEARCHIVE001):Archive.Open,Archive.ExtractandArchive.Create, with detection by content, extraction that never writes outside the target, and limits on size, ratio, entry count, path length and nesting.- tar: v7, ustar, pax and GNU (long names, sparse files), read and write.
- zip: read and write with store, Deflate and Zstandard, zip64 and data descriptors; LZMA, xz and bzip2 entries read.
- 7z: read, with Copy, Deflate, BZip2, LZMA, LZMA2, Delta, the branch filters and Zstandard, solid folders and encoded headers.
- Single streams: gz, zst and lz4 (read and write) and bz2 (read), and tar inside them.
Lz4BlockEncoder, a reusable block compressor, and a length-prefixed block form read back byLz4PrefixedBlock(not an LZ4 frame).Lz4Dictionary.Train.ZstdCompressionDictionary.ForDecompressionandToArray, so one trained dictionary serves both directions.ZstdCompressionOptions.SizeHint: an estimate of a stream's length that sizes the window and tables without being written to the frame.
Changed
- Zstandard encoder memory: a stream of unknown length no longer reserves its whole window and binary-tree tables up front (level 22 took 384 MiB for a 1 KiB frame, 640 MiB with a small dictionary; now 129 MiB), and a dictionary's search index is sized by the dictionary, not the source (4 KiB at level 22: 256 MiB, now under 1 MiB). Compressed bytes are unchanged.
- Zstandard and Deflate speed work; compressed bytes of Zstandard are unchanged.
Not supported
- PPMd (zip method 98, 7z PPMD), BCJ2, Deflate64 and encryption are listed but refused with a typed "unsupported method" error.