Skip to content

Releases

v0.1.0

Choose a tag to compare

@oguzeldereli oguzeldereli released this 09 Nov 13:25

First release. Ternary language models on a home computer, one command line (ttf):

  • Run the released models: ttf engine build (llama.cpp + the TTF patches) and ttf chat / ttf serve.
    • The 27B (8.2 GB) fits on a 12 GB GPU: 29 tok/s decode, 748 tok/s prefill on an RTX 3090 Ti.
    • Both released models (TQ1_P weights) run in the --q38 engine (ttf engine build --q38); for Flash-Next the experts go to host memory as needed.
  • Train from scratch (ttf train, the model size picked from GPU memory), convert Qwen3.5 / 3.8 models
    (ttf convert), fine-tune on your own text (ttf finetune), prepare data from text / Markdown / JSONL
    (ttf data).
  • Export to GGUF (ttf export), exactly: llama.cpp's loss matches the trainer's.
    • Weight types: TQ2_0 / TQ1_0 / Q2_K, plus TQ1_P (1.6 bits) for Flash-Next.
  • llama.cpp patches:
    • llama.cpp/: 11 patches on llama.cpp 0ef6f06d5;
    • llama.cpp/q38/: 8 patches on ca89968a2.
    • What they add: TTF side tensors and adapters applied in build_lora_mm (every architecture), CUDA TQ2_0 / TQ1_0
      matrix-vector and MMQ kernels, and the fused scale / shift / bias epilogues.

Running models needs only pip install -e . from a clone. Training needs the [train] extra and an NVIDIA GPU.

License: PolyForm Noncommercial 1.0.0 (code and patches). llama.cpp is MIT. The released weights keep Apache-2.0.