gzipt — gzip as a language model

gzipt generates text using gzip as its only model. No neural network, no training, no parameters. You prime it with a corpus, and it continues a prompt by searching for the byte sequences that compress best, because what compresses well is what the model predicts. Below is an example output:

gzipt --corpus data/tinyshakespeare.txt --prompt $'MENENIUS:\n' --length 200

MENENIUS:
'Though all at once canq

MARCIUS:
Pray now, nocamest thou to a morsel .

LARTIUS:
Hence, and
I' the end admire, where G
again; and after it ag .

LARTIUS:
Hence, and
I' the end ad

LARTIUS:
fame and

This is somewhat of a cherry picked example (it is normally slightly worse than this) but isn't it cool that we can do this at all!

Usage

To download the Shakespeare dataset I used:

# Download the dataset (Shakespeare text)
wget https://github.com/nathan-barry/tiny-diffusion/releases/download/v2.0.0/data.txt

Below are the default values for the CLI arguments.

gzipt \
  --corpus FILE \         # primes gzip's window with context
  --prompt "text" \       # promt to continue
  --length 200 \          # bytes to generate
  --horizon 24 \          # beam depth: bytes looked ahead and committed per span
  --beam-width 32 \       # partial continuations kept each step
  --temperature 0.5 \ 
  --tail 80 \             # generated bytes kept in scoring context (anti-copy)
  --window 30000 \        # corpus bytes shown to gzip (<= 32768)
  --workers 8             # threads for scoring (zlib releases the GIL)

Other compressors

--algo swaps the compressor that is the model (zlib default, plus bz2, lzma, zstd, brotli). Only zlib can clone its encoder state, so the others recompress each candidate and want a smaller --window. In the default byte-level mode they also tend to degenerate: a one-byte difference is below their compressed-length granularity, so the beam goes blind and repeats the cheapest filler byte.

Secondary mode: --mode spans

A second generator that re-ranks whole continuation spans pulled from the corpus (via an n-gram lookup) instead of inventing one byte at a time. Scoring multi-byte spans clears that granularity floor, so the coarse compressors produce coherent text:

gzipt --corpus data/tinyshakespeare.txt --prompt $'MENENIUS:\n' \
  --mode spans --algo bz2 --span-len 8

MENENIUS:
I tell you, friends, most charitable care
Have the patricians of you. For your wants,
Your suffering in this dearth, you

--span-len is the copy/recombine knob (higher = longer verbatim quotes). The trade-off is that the compressor only ranks real corpus text rather than generating it, so the result copies more and emerges less — which is why the byte-level mode above stays the main event.