fesh

fesh is an experiment asking AI (ChatGPT Pro and Gemini) blindly to see if they can make a compression library that is more efficient than xz. I had no idea how any of this works.

fesh is a specialized compression pre-processor for x86_64 ELF binaries. It leverages native binary structure to vastly improve traditional LZMA (XZ) dictionary chains.

By deterministically lifting structural mechanics (e.g. Near Branches, RIP-relative addressing, and ELF Relocation structures) into absolute, fixed-width delta domains, fesh achieves zero-metadata exact reversibility while compressing executable artifacts deeper than standard xz -9e and xz --x86.

Architecture

The core engine driving fesh has four main pillars:

1. Big-Endian Image-Relative MoE Mapping: It disassembles .text locally and overwrites relative offsets (disp32) with absolute Virtual Addresses globally, then normalizes those addresses relative to the image_base of the ELF segment. fesh uses a Mixture of Experts (MoE) evaluation gate to convert and test the resulting addresses dynamically into standard Little-Endian or reversed Big-Endian layouts. This leverages LZMA's anchor chaining when high-order stability zeroes are front-loaded against opcodes. It extends this mapping to .eh_frame_hdr headers and Jump Table boundaries inside .rodata.
2. 16-Stream Entropy Separation: It separates the transformed execution skeleton into disjoint semantic pipes (e.g., Code, Strings, .eh_frame, .rela, .dynamic, Jump Tables). LZMA models each boundary independently in parallel. Parameter vectors assign lzma_literal_context_bits = 0 for these structures, and empty streams are excluded via a RAW method flag.
3. In-Place ZigZag Struct Deltas: Complex ELF table structures (like .rela.dyn, .symtab, .relr, and .dynamic) undergo in-place column-wise delta mathematics based on the ELF spec (r_offset, r_addend, st_size). ZigZag encoders are used to prevent signed 64-bit deltas from bleeding 0xFF trails across the sequence.
4. Field-Endian Pre-Transpose: fesh forces each data column into Big-Endian representations before executing the final byte shuffle, grouping zero-padding bytes of 64-bit fields together.

Build

Built entirely in Rust for aggressive multithreaded performance (via rayon).

cd fesh_comp
cargo build --release

Usage

# Compress
./target/release/fesh_comp compress <input_elf> <output.fes>

# Decompress
./target/release/fesh_comp decompress <input.fes> <output_elf>

100-Package Benchmark

The following benchmarks were generated by downloading 103 application binaries from Alpine Repositories across 6 major compression configurations (GZIP, Brotli -11, ZSTD -19, XZ -9e, XZ -9e + BCJ, and fesh).

Every benchmark enforces decompression validation to ensure exact reproduction.

fesh had the smallest size in 103 out of 103 tests.

Full benchmark details are in BENCHMARK.md.

(Note: Binaries under 50KB were excluded. fesh executes within ~150ms per binary via Rayon).