API Reference

API Reference

moezip offers three distinct integration interfaces depending on your programming language and runtime environment:

1. Native Python Module (import moezip): Installable via PyPI or compiled with PyBind11.
2. C-API Shared Library (moezip.dll / libmoezip.so): Exports C-linkage functions for ctypes, C, C++, Rust, or Go.
3. Native C++ Headers: Direct C++20 header inclusion for native C++ projects.

---

1. Native Python Extension (import moezip)

Install directly from PyPI:

pip install moezip

When installed via pip or built with PyBind11, the C++ engine exposes thread-safe, high-level functions directly to Python. Embedded assets load quietly and automatically on first call.

moezip.init(vocab_path: str = "", router_path: str = "", quiet: bool = True) -> None

Initializes or re-loads search engine assets. Calling this manually is optional—compress() and decompress() auto-initialize quietly if not called.

• Parameters:
  • vocab_path (str) – Path to a custom vocabulary text file, or "" for embedded defaults.
  • router_path (str) – Path to a custom router JSON matrix file, or "" for embedded defaults.
  • quiet (bool) – Silences C++ initialization warning logs when True (default: True).

moezip.compress(text: str) -> bytes

Compresses a UTF-8 string into a binary bytes buffer.

• Parameters: text (str) – Raw input string.
• Returns: bytes – Compressed binary data payload.
• Raises: std::runtime_error if encoding fails.

moezip.decompress(packed_bytes: bytes) -> str

Decompresses a moezip binary bytes buffer back into its exact UTF-8 string representation.

• Parameters: packed_bytes (bytes) – Compressed binary data buffer.
• Returns: str – Restored UTF-8 text string.
• Raises: std::runtime_error if payload structure is invalid or corrupt.

moezip.train(corpus_or_path: str, output_filepath: str = "router_stateless_v4.json", quiet: bool = False) -> None

Trains the $32 \times 32$ router state matrix on specialized text to optimize compression ratios. Accepts either a raw text string or a path to a corpus file on disk. Updates the live in-memory transition matrix and saves the trained weights to JSON.

• Parameters:
  • corpus_or_path (str) – Raw text string or file path to a text corpus on disk.
  • output_filepath (str) – Target JSON file path for saving trained transition matrix weights.
  • quiet (bool) – Silences training metrics printed to stderr when True.

Python Example

import moezip

# 1. Direct compression (quiet auto-initialization)
data = "Sample string to compress using embedded assets."
compressed = moezip.compress(data)
restored = moezip.decompress(compressed)

assert data == restored

# 2. Train router matrix on custom domain text
sample_corpus = "Medical text dataset for domain optimization..."
moezip.train(sample_corpus, output_filepath="medical_router.json")

# 3. Re-initialize engine with custom trained router matrix
moezip.init(router_path="medical_router.json", quiet=True)

---

2. C-API Shared Library (moezip.dll / libmoezip.so)

For integration via Python ctypes, C, C++, Rust, or Go, moezip exports C-linkage functions (extern "C") through api.cpp.

Functions

void init_engine(const char* vocab_path, const char* router_path)

Pre-loads search tables and state transition matrices into memory.

• vocab_path: File path to custom vocabulary text file, or NULL / "" to load embedded C++ binary assets.
• router_path: File path to custom router JSON file, or NULL / "" to load embedded C++ binary assets.

int compress_text(const char* text, uint8_t* out_buf, int max_out_len)

Compresses a null-terminated UTF-8 text string into an output byte buffer.

• Returns: Integer length of written bytes in out_buf, or -1 if output buffer size is insufficient.

int decompress_bytes(const uint8_t* in_buf, int in_len, char* out_buf, int max_out_len)

Decompresses raw binary bytes back into a null-terminated UTF-8 character buffer.

• Returns: Integer character length written to out_buf, or -1 if output buffer size is insufficient.

Python ctypes Integration Example

import os
import sys
import ctypes

# Select platform shared library
lib_name = "moezip.dll" if sys.platform == "win32" else "libmoezip.so"
lib_dir = "build/Release" if sys.platform == "win32" else "build_linux"
lib_path = os.path.abspath(os.path.join(lib_dir, lib_name))

moezip = ctypes.CDLL(lib_path)

# Configure C function signatures
moezip.init_engine.argtypes = [ctypes.c_char_p, ctypes.c_char_p]
moezip.compress_text.argtypes = [ctypes.c_char_p, ctypes.POINTER(ctypes.c_uint8), ctypes.c_int]
moezip.compress_text.restype = ctypes.c_int
moezip.decompress_bytes.argtypes = [ctypes.POINTER(ctypes.c_uint8), ctypes.c_int, ctypes.c_char_p, ctypes.c_int]
moezip.decompress_bytes.restype = ctypes.c_int

# Initialize engine with embedded binary assets (pass empty strings)
moezip.init_engine(b"", b"")

def compress(text: str) -> bytes:
    raw_bytes = text.encode('utf-8')
    buf_size = len(raw_bytes) * 2 + 2048
    out_buf = (ctypes.c_uint8 * buf_size)()
    comp_len = moezip.compress_text(raw_bytes, out_buf, buf_size)
    if comp_len <= 0:
        raise RuntimeError("Compression error")
    return bytes(out_buf[:comp_len])

def decompress(comp_bytes: bytes) -> str:
    in_buf = (ctypes.c_uint8 * len(comp_bytes)).from_buffer_copy(comp_bytes)
    buf_size = len(comp_bytes) * 10 + 4096
    out_buf = ctypes.create_string_buffer(buf_size)
    decomp_len = moezip.decompress_bytes(in_buf, len(comp_bytes), out_buf, buf_size)
    if decomp_len <= 0:
        raise RuntimeError("Decompression error")
    return out_buf.value.decode('utf-8')

---

3. Native C++ Integration

For native C++20 projects, include the core headers directly into your build system.

Core Headers

• #include "vocab.hpp": Vocabulary partitioning and dictionary data structures.
• #include "router.hpp": $32 \times 32$ Markov transition matrix and predictor routines.
• #include "codec.hpp": Primary adaptive compression and decompression entry points.

Primary C++ Functions

// Load vocabulary assets (or embedded defaults if empty string passed)
VocabData vd = load_and_partition_wordlist("");

// Load router transition matrix (or embedded defaults if empty string passed)
TransitionMatrix matrix = load_router("", vd.expert_count);

// Compress text to binary byte vector
std::vector<uint8_t> packed = compress_adaptive_moe(
    text, vd.vocab_to_idx, matrix, vd.expert_count, vd.expert_size
);

// Decompress binary byte vector back to string
std::string restored = decompress_adaptive_moe(
    packed, vd.vocab, vd.vocab_to_idx, matrix, vd.expert_count, vd.expert_size
);

Native C++ Code Example

#include <iostream>
#include <string>
#include <vector>
#include "vocab.hpp"
#include "router.hpp"
#include "codec.hpp"

int main() {
    // Initialize search engine assets
    VocabData vd = load_and_partition_wordlist("");
    TransitionMatrix matrix = load_router("", vd.expert_count);

    std::string input = "Sample text for native C++ compression test.";

    // Compress
    auto compressed = compress_adaptive_moe(
        input, vd.vocab_to_idx, matrix, vd.expert_count, vd.expert_size
    );

    // Decompress
    std::string decompressed = decompress_adaptive_moe(
        compressed, vd.vocab, vd.vocab_to_idx, matrix, vd.expert_count, vd.expert_size
    );

    std::cout << "Original size  : " << input.size() << " bytes\n";
    std::cout << "Compressed size: " << compressed.size() << " bytes\n";
    std::cout << "Restored string: " << decompressed << "\n";

    return 0;
}