A Kotlin Multiplatform starting point for a port of
oozextract, under
com.uselessmnemonic.koodle. Supports LZB16 plus stateful LZNA and Bitknit
compressed quanta, raw blocks, fill quanta, whole-match quanta, and uncompressed
small quanta. The shared large-quantum path also supports raw chunks and all
byte-array entropy encodings under Kraken, Mermaid/Selkie, and Leviathan framing.
Both sparse and dense legacy Huffman
tables are supported for the three-stream and six-stream byte-array layouts,
including single-symbol fills. Newer Golomb-Rice Huffman tables and their compact
symbol-range maps are also supported. tANS arrays support both legacy and newer
Golomb-Rice table representations. Recursive arrays cover both sequential
concatenation and entropy-array interleaving. Kraken LZ chunks support ordinary
and delta literals, recent, traditional, and scaled explicit offsets, and
extended lengths, including detached excess-byte streams.
Mermaid/Selkie LZ chunks support near and far offsets, long commands, delta
literals, and split 64 KiB command streams. All six Leviathan modes support raw,
subtractive, laminated, striped, and order-1 literal streams, single or
multi-array commands, recent offsets, traditional and scaled packed offsets, and
extended lengths. Packed offsets and lengths may use recursive byte-array
representations. Packed RLE command buffers may use any supported byte-array
encoding. Checksummed block framing is supported, but the stored checksums are not
verified because the checksum algorithm is absent from the reference implementation.
Kotlin 2.4.21, Gradle 9.5.0, and JDK 17 or newer (JVM bytecode targets 17). The library depends only on the Kotlin standard library and platform APIs. Gradle and Kotlin compiler downloads are build tooling, not library dependencies.
Published coordinates:
commonMain.dependencies {
implementation("com.uselessmnemonic:koodle:<version>")
}./gradlew compileKotlinJvm compileKotlinLinuxArm64 compileKotlinMacosArm64Build the macOS target on macOS with the Apple developer tools required by Kotlin/Native. The first build downloads the Kotlin/Native toolchains.
import com.uselessmnemonic.koodle.decompress
import com.uselessmnemonic.koodle.decompressInto
val decoded = decompress(compressed, outputSize = expectedSize)
val written = decompressInto(compressed, destination)These eager conveniences delegate to OozDecoder, which implements the small
Decoder capability interface. The former scaffold Decompressor class has
been replaced by top-level functions. Source ranges use inclusive starts and
exclusive ends; destination offsets and exact output sizes avoid slice copies.
The enclosing container supplies the decoded size; container headers are not
input to the decoder. Eager operations ignore trailing input.
For incremental decoding, construct OozDecoder(outputSize: Long) and call
decode with input and output chunks. Advance by bytesConsumed and
bytesWritten; retain and resubmit any unconsumed input. NEEDS_INPUT requests
more input, NEEDS_OUTPUT requests destination capacity, and FINISHED leaves
trailing input untouched. Pass endOfInput = true when the supplied range is the
last input, and keep it true when resubmitting its unconsumed suffix. A truncated
final input throws DecompressionException once output capacity is available.
The decoder retains no caller arrays and owns no I/O resources. Arrays must not alias, and instances must not be shared concurrently. Invalid arguments do not change state. Malformed data or unsupported blocks invalidate the instance; discard it after failure. Output can be partially modified on failure. A zero output size finishes without consuming input.
The current implementation validates two-byte block headers, recognizes the six Rust decoder IDs, and copies raw payloads, including across 256 KiB boundaries. An internal incremental quantum-header parser uses sealed data variants for compressed payload sizes, fill values, whole-match distances, and uncompressed quanta. It handles three-byte headers for large quanta and two-byte headers for small quanta. Headers may span calls, including variable-length whole-match distances. Fill and whole-match quanta can continue producing output after input is exhausted. LZB16 decoding lives separately from framing and handles literals, length extensions, and forward-expanding overlapping matches. A bounded ring retains history across quanta. Codec restarts restrict LZB16 references to output since the restart, while whole-match framing can still reference earlier stream output. All produced output feeds history.
Compressed input is buffered one quantum at a time, decoded into a reusable output buffer, then drained into caller-provided chunks. LZB16 uses two 16 KiB buffers; large quanta use two 256 KiB buffers plus as many as two lazy 256 KiB entropy arenas. They are allocated lazily and reused, requiring at most 1 MiB of working buffers plus history, independently of stream size. The shared byte-array layer validates short/long headers and encoded/decoded sizes. Its RLE decoder handles a single-byte fill, unpacked commands, and raw nested command arrays without allocating scratch buffers. Packed command arrays decode into a lazy, reusable 256 KiB entropy workspace; their untouched suffix is appended in that arena before the two-ended RLE pass. Nested decoders use nonoverlapping arena regions, and inputs exceeding the fixed workspace fail before writing. Sequential recursion is limited to 64 nested levels so malformed inputs cannot exhaust the call stack. Interleaved recursive arrays decode as many as 63 entropy sources, then rebuild their output from bounded index, interval-width, and dual-ended variable-bit streams. The tANS decoder uses a bounded scalar lookup table and two-ended bit cursors.
The Huffman decoder uses a portable scalar canonical-code table and bounds-checked bit readers. It supports sparse symbol lists and dense alternating symbol runs, including gamma-coded run lengths and adaptive code-length deltas with all four forced-bit widths. Code lengths through 11 bits are supported. It validates complete tables, duplicate symbols, run bounds, stream splits, and stream consumption. The newer format decodes unary Golomb-Rice quotients, separated remainder bits, adaptive length residuals, truncated-binary metadata, and compact symbol ranges with the same bounded cursor. The scalar implementation is the correctness baseline; platform-specific acceleration can be considered after the remaining entropy codecs are ported and benchmarked.
Whole-match, Kraken, and Mermaid distances may exceed LZB16's 16-bit window.
OozDecoder, decompress, and decompressInto accept historyCapacity in
bytes (default/minimum 65,536). Increase it before decoding streams requiring a
larger window. A reference before the applicable stream or codec restart is
malformed; a valid reference beyond the retained window throws
UnsupportedOperationException with the required distance. History never grows
automatically. Framing distances are parsed as Long with overflow checks;
capacities are limited to Int arrays.
Keep codecs separate from framing, archives, and I/O adapters. Follow familiar Kotlin I/O conventions without importing kotlinx-io or recreating its types. Full project API guidelines are recorded in AGENTS.md.
git submodule update --init
./gradlew jvmTest compileTestKotlinLinuxX64 compileTestKotlinLinuxArm64 compileTestKotlinMacosArm64
# On an ARM64 Mac with full Xcode installed and selected:
./gradlew macosArm64TestThe regression suite is made of standard KMP tests in commonTest and runs on
JVM and macOS ARM64. Both platforms decode the corpus in the pinned oozextract
Git submodule and compare every result with its reference output. Expect/actual
fixture providers isolate NIO and POSIX file access. Each fixture loads only
inside a structured use block and releases its array references afterward, so
the complete corpus is never retained in memory at once. The tests cover
fragmented input/output, block and quantum boundaries, mixed fill/raw quanta,
array ranges, trailing input, truncated and invalid headers, unsupported features,
LZB16 command forms, overlap expansion, history wraparound/restarts, whole-match
distance encodings, overflow, capacity limits, byte-array header variants, RLE
command forms, legacy sparse/dense Huffman tables and stream layouts, 128 KiB
chunk boundaries, Golomb-Rice tables, sparse symbol-range maps, and decoder
failure state. Packed-RLE checks cover nested raw, RLE, Huffman, and Huffman6
arrays plus workspace exhaustion. Recursive checks cover the minimum child count,
many-child arrays, nested concatenation, direct and interleaved multi-arrays, separate and combined
interval metadata, malformed streams, use inside packed RLE, exact input/output
accounting, and the depth limit. tANS checks cover legacy and Golomb-Rice table
construction, symbol ranges, its five interleaved states, recursive nesting,
truncation, and all three shared large-quantum framings. Kraken checks cover
ordinary/delta literals, recent/traditional/scaled offsets, overlapping matches,
and both inline and variable-bit extended lengths. Mermaid checks cover
ordinary/delta literals, near/far offsets, long commands, and the 64 KiB command split.
Leviathan checks cover all six literal modes, single/eight-way command streams,
recent/traditional/scaled offsets, recursive packed metadata, and extended
literal lengths. LZNA checks cover its initial literal and terminal rANS-state
layout. Bitknit checks cover its bootstrap and terminal coder states. The local
corpus exercises both codecs' persistent models, match forms, and restarts.
check uses the normal KMP test lifecycle. kotlin.test is test-scoped and does
not appear in the published library metadata. Native test linking requires full
Xcode; standalone Command Line Tools can compile the library and tests but cannot
link or run macosArm64Test.
commonMain: public API and, eventually, all decoding algorithms.jvmMain: internal memory access using JDK byte-array view VarHandles.nativeMain: shared Native memory access using Kotlin/NativegetIntAt.- Targets: JVM, Linux x64, Linux ARM64, macOS ARM64.
The first memory primitive is a bounds-checked, unaligned 32-bit little-endian load. Native experimental APIs are opted into only in the implementation file. No cinterop, pinning, manual allocation, or macOS-specific implementation is needed for this primitive. Extend the small internal boundary as the port needs it; benchmark before adding unsafe memory or SIMD. LZ overlapping match copies use explicit forward-expansion semantics, not a generic bulk-copy shortcut.
The framing behavior follows src/ooz/mod.rs in the MIT-licensed oozextract
reference: parse_header, parse_quantum_header, parse_whole_match,
read_sync, and the raw/fill/match branches of extract_block. LZB16 follows
src/algorithm/lzb16.rs with a bounded history ring in place of whole-output
indexing. LZNA follows src/algorithm/lzna.rs, retaining its adaptive probability
models across 16 KiB quanta and writing the two terminal rANS states into each
quantum's final eight bytes. Bitknit follows src/algorithm/bitknit.rs, retaining
its cumulative-frequency models and recent-distance permutation across 16 KiB
quanta, with two terminal 16-bit coder states. During this port, the local
lzb16-mixed (300,000 bytes) and lzb16-short (777 bytes) fixtures were decoded
and compared byte-for-byte against the Rust repository's verify files.
Whole matches expand forward on overlap, consistent with
Kraken_CopyWholeMatch in the original ooz implementation.
This intentionally differs from the Rust branch's snapshot-style copy_within,
which can read not-yet-produced output when a whole match overlaps. No C++ code
was copied. Whole-match checks use constructed vectors; no whole-match fixtures
were available in the Rust tests inspected.
The shared byte-array layer follows decode_bytes, decode_rle, and
decode_rle_unpacked in src/decoder/mod.rs. The large-quantum adapter follows
the raw and byte-array branches of decode_quantum. Kraken command processing
follows KrakenLzTable, including the traditional two-ended offset/length bit
stream, scaled offsets with their optional low-byte stream, and forward-expanding
match copies and detached excess-byte streams.
Mermaid/Selkie processing
follows MermaidLzTable, including raw or entropy-coded near offsets, far
offsets, long lengths, and its two 64 KiB command partitions. All Leviathan modes
follow LeviathanLzTable, including laminated, striped, and order-1 literal
selection plus eight-way command streams and recursive packed offset/length
representations. Tests for these paths include constructed command vectors. All
Kraken, Mermaid, Selkie, and Leviathan files in the inspected local corpus—including
Kraken restart and cross-block history fixtures—decoded byte-for-byte to their
reference outputs.
All 12 LZNA files in the inspected local corpus decoded byte-for-byte to their
reference outputs. LZNA uses absolute stream positions for model contexts and
match references even when a restart replaces its adaptive model; retained match
bytes remain bounded by historyCapacity.
All 12 Bitknit files in the same corpus also decoded byte-for-byte to their reference outputs. Its arithmetic uses explicit unsigned 16/32-bit semantics; match copies use the same bounded absolute-stream history as LZNA.
Legacy Huffman behavior follows decode_bytes_type12,
huff_read_code_lengths_old, and the scalar semantics of HuffReader. During
this port, the local single-symbol-huffman.kraken fixture decoded all 100 bytes
and matched its reference output. Multi-symbol checks use constructed canonical
tables because no matching standalone fixture was present in the inspected test
data.
New Huffman behavior follows huff_read_code_lengths_new,
decode_golomb_rice_lengths, decode_golomb_rice_bits, and
convert_to_ranges. The port expresses their optimized table operations as
bounds-checked scalar bit parsing. Constructed checks cover all forced-bit widths,
initial and internal symbol gaps, one and 256-symbol alphabets, and both output
stream layouts.
tANS behavior follows decode_tans, both branches of
TansDecoder::decode_table, and TansDecoder::init_lut. State transitions use
portable bounded cursors over the forward and backward bitstreams. New table
metadata reuses the same bounded Golomb-Rice and symbol-range semantics as the
Huffman implementation.
Packed LZ metadata follows the reference BitReader/unpack_offsets boundary.
A shared two-ended reader owns cursor movement and overlap checks; small pure
integer extensions name the implied-high-bit, traditional/scaled distance,
relative-offset, scale/low-byte, and packed-length transforms. Kraken and
Leviathan compose those primitives rather than duplicating the bit arithmetic.
Packed RLE behavior follows the scratch-buffer branch of decode_rle: a leading
byte array is decoded into bounded workspace, remaining command bytes are
appended, and decode_rle_unpacked consumes literals from the front and commands
from the back. The workspace is owned and reused by OozDecoder; direct internal
byte-array decoding creates one only when a packed RLE path needs it.
Sequential recursive behavior follows the noninterleaved branch of
decode_recursive: the control byte permits at most two child arrays, each child
uses the same byte-array decoder, and decoded results are concatenated into the
exact parent output range. The high-bit form follows decode_multi_array: it
decodes entropy arrays into one bounded arena and interleaves intervals into a
separate destination arena. The generalized internal capability returns exact
logical slices for Leviathan's 2/4/16 literal arrays and eight command arrays;
the one-array recursive path uses the same format semantics.
The checked-in regression suite uses self-contained constructed vectors and does not require that sibling repository.
Publications use com.uselessmnemonic:koodle:<version> and are released under
the MIT License. The publishing configuration creates the KMP root,
JVM, Linux x64, Linux ARM64, and macOS ARM64 publications with sources,
documentation, complete POM metadata, and signatures.
Before the first release, verify the com.uselessmnemonic namespace in the
Maven Central Portal, publish the signing key to a public keyserver, and add these
GitHub Actions secrets:
MAVEN_CENTRAL_USERNAMEMAVEN_CENTRAL_PASSWORDSIGNING_KEY_IDSIGNING_PASSWORDGPG_KEY_CONTENTS
Creating a GitHub release whose tag is v0.1.0 or 0.1.0 runs the publishing
workflow with Maven version 0.1.0. Local builds remain 0.1.0-SNAPSHOT unless
-PVERSION_NAME=<version> is supplied. Install the signed release artifacts and
POMs locally without uploading them with:
./gradlew publishToMavenLocal -PVERSION_NAME=0.1.0Toolchain and API references: KMP compatibility, Native getIntAt.