Gleipnir 1.2.0

What's Changed

• Gleipnir 1.2: model 4 at -9, x86 contexts at every preset, huge pages by @ValisSowilo in #8

Full Changelog: v1.1.0...v1.2.0

Gleipnir 1.1.0

What's Changed

• Docs: re-measure Silesia and enwik9 on the v1.0.2 release by @ValisSowilo in #4
• Docs: Gleipnir 1.0.2 is now listed on both Mahoney benchmarks by @ValisSowilo in #5
• Word transform for text: enwik9 −5.76%, enwik8 −4.16% (1.1.0, format v3) by @ValisSowilo in #6
• Docs and benchmarks for 1.1: single-session Silesia table, graphs, fuzz in CI by @ValisSowilo in #7

Full Changelog: v1.0.2...v1.1.0

Gleipnir 1.0.2

What's Changed

• Harden the decoder, fix Apple Silicon, and add the gate that catches this class by @ValisSowilo in #1
• Clamp threads on extraction, and stop four ways to lose data quietly by @ValisSowilo in #2
• Read every input to end of file, and accept - as standard input by @ValisSowilo in #3

New Contributors

• @ValisSowilo made their first contribution in #1

Full Changelog: v1.0.1...v1.0.2

Gleipnir 1.0.1

Gleipnir is a context-mixing lossless compressor and archiver in one C file. It
predicts each bit with 27 statistical models, blends them with a learned mixer,
and codes the result with an arithmetic coder. There is no dictionary in the
output at all.

New here? Download and install instructions are in the README — plain numbered steps for Windows and Linux.

What's new in 1.0.1

The engine is unchanged from 1.0.0 — output is byte-identical and archives are
compatible in both directions. This release is about getting it into people's
hands:

• One-click Windows installer (gleipnir-1.0.1-setup.exe) — per-user, no
  administrator prompt. Adds gleipnir to your PATH and a Compress with
  Gleipnir entry to the File Explorer right-click menu (plus Extract / Verify
  / List on .gl files).
• The Linux binary is now statically linked, so it runs on any x86-64 Linux
  with nothing to install — no glibc version requirement, no libz.so.1.
• Every artifact here is built and tested by CI for this exact commit
  (Linux + Windows build, round-trip, corruption fuzz, and a cross-platform
  read check), so each hash below is checkable against that run.

Silesia

211,938,580 bytes, one machine, single thread, every gleipnir row round-trip
verified against its SHA-256.

                      total       ratio     bpc     comp     decomp          peak
gleipnir -9           35,582,296       5.96x   1.343     597s       672s       1055 MB
gleipnir -7           36,493,092       5.81x   1.377     451s       467s        922 MB
gleipnir -5           38,273,415       5.54x   1.445     337s       342s        483 MB
zpaq -m5         39,113,069       5.42x   1.476     559s       582s      839 MB *
gleipnir -3           39,510,297       5.36x   1.491     248s       248s        305 MB
lpaq1 -6         43,006,234       4.93x   1.623     173s       186s      199 MB *
gleipnir -1           43,207,157       4.91x   1.631     139s       139s        196 MB
xz -9e           48,456,100       4.37x   1.829     114s         2s      509 MB *

Codecs marked * do not report their own peak RSS, so those figures are sampled
at 50 ms and are a lower bound. The peak column is compression; decompression
peaks 37-40 MB higher at every preset.

Decompression costs the same as compression, because the decoder runs the
same model. That is inherent to context mixing and it is why this is built for
data you keep and rarely read, not for general-purpose use.

Integrity

The gate is that for any input, damaged or not, Gleipnir either exits 0 with
byte-exact output or exits non-zero with a diagnostic.

• 648 round trips, all exact
• 60 round trips across 6 thread counts, all exact
• 300 randomized corruption trials across 7 damage models: no crash, no hang,
  and no case of exit 0 with wrong output

CI builds and fuzzes on Linux and Windows, and verifies that an archive written
on Linux reads on Windows.

Downloads

gleipnir-1.0.1-setup.exe      b577e5f2a6355568cf105c72df45e268d9db35267cf69d7cba4e38141289cb12
gleipnir.exe                  ab04043c4df4ee39aa14079ce2672d82a53c4527205c9af97532278093c6e00b
gleipnir-windows-x86_64.zip   f54e4149fac903bbdcbf6bbe516c890b895ec9a3536ef8eaaf06c6ae96039a4f
gleipnir-linux-x86_64         7d552c810cb47f156a0153e39c9956aa4020685b5d6613df534e2f6af8c85e5e

• gleipnir-1.0.1-setup.exe — Windows one-click installer (recommended). Double-click, click through, done. Per-user, so no administrator prompt. Not code-signed, so SmartScreen warns the first time: More info → Run anyway.
• gleipnir.exe — the whole program in one file, no installer. Statically linked, KERNEL32 and the UCRT only, so it runs on Windows 10 and later with nothing else.
• gleipnir-windows-x86_64.zip — the same gleipnir.exe plus a script install (install.ps1) for people who would rather not run an unsigned installer.
• gleipnir-linux-x86_64 — statically linked, so it runs on any x86-64 Linux with nothing to install. chmod +x it and go.

The binaries are built -march=x86-64-v2 (SSE4.2/POPCNT, any x86-64 CPU from roughly 2009 on), never -march=native.

Licence

GPL-3.0-or-later. First published under the Business Source License 1.1 (as
Nyx 1.0.0) with a Change Date of 2030-08-04; that conversion has been brought
forward and the BSL terms no longer apply to any version.

The engine's lineage is PAQ, ZPAQ and lpaq. Bit-history states, logistic mixing
in the stretched domain, ISSE chains and SSE/APM refinement are all from Matt
Mahoney's work. No PAQ code is used here, but the ideas are not mine.

Gleipnir 1.0.0

Gleipnir is a context-mixing lossless compressor and archiver in one C file. It
predicts each bit with 27 statistical models, blends them with a learned mixer,
and codes the result with an arithmetic coder. There is no dictionary in the
output at all.

New here? Download and install instructions are in the README — plain numbered steps for Windows and Linux.

Formerly released as Nyx; renamed to Gleipnir. The on-disk format is unchanged (archive version v2), so archives written by Nyx 1.0.0 still read.

Silesia

211,938,580 bytes, one machine, single thread, every gleipnir row round-trip
verified against its SHA-256.

                      total       ratio     bpc     comp     decomp          peak
gleipnir -9           35,582,296       5.96x   1.343     597s       672s       1055 MB
gleipnir -7           36,493,092       5.81x   1.377     451s       467s        922 MB
gleipnir -5           38,273,415       5.54x   1.445     337s       342s        483 MB
zpaq -m5         39,113,069       5.42x   1.476     559s       582s      839 MB *
gleipnir -3           39,510,297       5.36x   1.491     248s       248s        305 MB
lpaq1 -6         43,006,234       4.93x   1.623     173s       186s      199 MB *
gleipnir -1           43,207,157       4.91x   1.631     139s       139s        196 MB
xz -9e           48,456,100       4.37x   1.829     114s         2s      509 MB *

Codecs marked * do not report their own peak RSS, so those figures are sampled
at 50 ms and are a lower bound. Do not read them as equals. The peak column is
compression; decompression peaks 37-40 MB higher at every preset.

Two things worth saying first

Decompression costs the same as compression, because the decoder runs the same
model. That is inherent to context mixing and it is why this is not a
general-purpose tool.

-7 is unstable in time. Across sessions it has read anywhere from 404 to 492
seconds with byte-identical output, so its speed claim carries a range of
1.21-1.28x rather than a single number.

Integrity

The gate is that for any input, damaged or not, Gleipnir either exits 0 with
byte-exact output or exits non-zero with a diagnostic.

• 648 round trips, all exact
• 60 round trips across 6 thread counts, all exact
• 300 randomized corruption trials across 7 damage models: no crash, no hang,
  and no case of exit 0 with wrong output

CI builds and fuzzes on Linux and Windows, and verifies that an archive written
on Linux reads on Windows.

Downloads

gleipnir-1.0.0-setup.exe      226eec2f45a1b7c6d86aa2e9e1d3349f317bcf479b232d70f72492e0c1080f4e
gleipnir.exe                  2359a1060c216e677a4a06dbd23e33fec6583bfed8806a1e4d2b8460af55ef68
gleipnir-windows-x86_64.zip   84d45231f42cf2060f769ea17b32d02d8ba74963dfc7c1caaff1c312b9bdd51d
gleipnir-linux-x86_64         3bcc359036cdfb87467845b5e696af05fd419dca52f5e57963148b164ef198d6

Every binary here is the unmodified artifact from the CI run for this commit, so each hash is checkable against that run.

• gleipnir-1.0.0-setup.exe — Windows one-click installer (recommended). Double-click, click through, done: it adds gleipnir to your PATH and adds Compress with Gleipnir to the File Explorer right-click menu (and Extract / Verify / List on .gl files). Per-user, so no administrator prompt. Not code-signed, so SmartScreen warns the first time: More info → Run anyway.
• gleipnir.exe — the whole program in one file, no installer. Statically linked, KERNEL32 and the UCRT only, so it runs on Windows 10 and later with nothing else.
• gleipnir-windows-x86_64.zip — the same gleipnir.exe plus a script install (install.ps1) for people who would rather not run an unsigned installer.
• gleipnir-linux-x86_64 — statically linked, so it runs on any x86-64 Linux with nothing to install — no glibc version requirement and no libz.so.1 on the system. chmod +x it and go.

The binaries are built -march=x86-64-v2 (SSE4.2/POPCNT, any x86-64 CPU from roughly 2009 on), never -march=native.

Licence

GPL-3.0-or-later. Gleipnir was first published under the Business Source License 1.1
with a Change Date of 2030-08-04; that conversion has been brought forward.

The engine's lineage is PAQ, ZPAQ and lpaq. Bit-history states, logistic mixing
in the stretched domain, ISSE chains and SSE/APM refinement are all from Matt
Mahoney's work. No PAQ code is used here, but the ideas are not mine.