Getting Started

Getting Started

Use Ornith 1.5 8-bit first. It completed every 8-bit coding/tooling qualification cell in the current matrix, and the fastest correct configuration used standard responses with Thinking off. Choose 4-bit when its lower storage use and higher raw decode speed matter more than that result.

Tip

Prefer one command? tools/install_tinytitan.sh does steps 1–3 for you: it checks the Mac, downloads the published arm64 executables and verifies their published checksum, offers to download a model, and installs a tinytitan command that starts the server — then offers to start it, so you finish with the base URL a client points at. It needs no Xcode, Homebrew, git or Node. With --web it also sets up the browser chat window (see step 5). From a clone:

tools/install_tinytitan.sh

or without a clone already on disk:

bash -c "$(curl -fsSL https://raw.githubusercontent.com/Pummelchen/TinyTitan/main/tools/install_tinytitan.sh)"

Use that form rather than curl … | bash: a pipe makes the script's stdin the pipe, so it cannot ask anything and takes the default at every step — including the model download. It will not set up the browser window, because that prompt needs a terminal; pass --web if you want it without one.

It is safe to re-run, and it never deletes a model. The rest of this page is the same installation by hand, one step at a time, for when you want to see or control each part.

1. Check the Mac

You need Apple Silicon, macOS 26+, and enough free internal SSD space. The one-command install needs nothing else; building from source (step 2) also needs Swift 6.4+.

Model	Installed size	Status
Ornith 1.5 8-bit	about 36.9 GB	Default, text-only
Ornith 1.5 4-bit	about 19.5 GB	Supported, smaller and faster decode
Qwen 3.6 4-bit	about 19.5 GB	Supported
Qwen 3.6 8-bit	about 37.8 GB	Supported, slower
KAT-Coder-V2.5-Dev 4-bit	about 20 GB	Supported: Kwaipilot's Qwen 3.6 fine-tune for agentic coding; verified on the real install
KAT-Coder-V2.5-Dev 8-bit	about 38 GB	Supported; same answers, slower
Qwen 3.5 2B / 4B / 9B	about 2.2–11.0 GB	4-bit and 8-bit; GPU by default, CPU on request

Note

The dense Qwen 3.5 2B is a light worker, not a small agent. Measured: it got a two-part instruction (copy a six-character code, add two three-digit numbers) wrong — or dropped the code — in three to five of eight attempts with thinking off, and six of eight at the model's own sampling. Thinking does not fix it: at 8-bit with thinking on it answered five of eight but took 42 to 253 seconds per reply and produced nothing at all on the other three, spending the whole budget inside the reasoning block (with thinking off those answers took about three seconds). Decoded greedily it never answers with thinking on — that path loops until the budget runs out. Use it for chat, summaries and recall beside a larger model; use a 4B/9B, or a 35B for real work, when the output has to be right.

Before loading a model, confirm that memory pressure is acceptable and no other local-model process is running:

memory_pressure -Q
pgrep -fl 'TinyTitanServer|TinyTitanCLI|TinyTitanPackageTests|swiftpm-testing-helper|mlx_lm|mlx-lm'

Continue only when the process check prints nothing. Do not terminate a process you did not start.

2. Build from source

Skip this step if you used the one-command install above: it downloads the released binaries instead. Building by hand is the contributor path and needs Swift 6.4+ (Xcode 27 or a matching toolchain).

git clone https://github.com/Pummelchen/TinyTitan.git
cd TinyTitan
swift build -c release

3. Install a model

The installer streams verified ranges directly into the final .gturbo directory; it does not stage a second full checkpoint.

# Recommended default 8-bit model; --model ornith15-8bit is optional
swift run -c release TinyTitanRepack \
  --output models/ornith-1.5_35B_A3B_8Bit

# Smaller Ornith 1.5 text model, 4-bit
swift run -c release TinyTitanRepack \
  --model ornith15 \
  --output models/ornith-1.5_35B_A3B_4Bit

# Optional Qwen 3.6 model, 4-bit
swift run -c release TinyTitanRepack \
  --model qwen36 \
  --output models/qwen3.6_35B_A3B_4Bit

# Optional Qwen 3.6 model, 8-bit
swift run -c release TinyTitanRepack \
  --model qwen36-8bit \
  --output models/qwen3.6_35B_A3B_8Bit

If the download is interrupted, repeat the same command with --resume. Use HF_TOKEN only when Hugging Face explicitly requests authentication.

The catalog installer

tools/install_models.sh wraps these installs and knows the whole catalogue, including the models that are converted from a checkpoint rather than streamed. Running it with no argument shows what is installed and what is missing, and --help lists the sources:

tools/install_models.sh                     # what is installed
tools/install_models.sh katcoder            # KAT-Coder-V2.5-Dev 4-bit
tools/install_models.sh katcoder-8bit       # KAT-Coder-V2.5-Dev 8-bit
tools/install_models.sh katcoder both       # both widths, ONE download

Use both when you want two widths of the same model. The Qwen3.5-MoE checkpoints -- Ornith 1.5, Qwen 3.6, Qwen-AgentWorld and KAT-Coder -- convert their 4-bit and 8-bit builds in a single pass from one ~70 GB fetch, so asking for one width at a time fetches that checkpoint twice. Installing a single width still converts both and keeps the other snapshot, so adding the second width later reuses it instead of downloading again.

To discard an incomplete installation:

swift run -c release TinyTitanRepack \
  --discard-partial \
  --output models/ornith-1.5_35B_A3B_8Bit

Optional Ornith MTP sidecar

MTP is experimental and disabled by default. The official MLX text releases omit the draft layer, so prepare it from Ornith's pinned original shard. This downloads about 4.1 GiB and creates a roughly 454 MiB 4-bit sidecar:

python3 -m venv .build/ornith-mtp-venv
.build/ornith-mtp-venv/bin/pip install numpy safetensors ml_dtypes huggingface_hub

HF_HOME="$PWD/.build/hf-cache" \
  .build/ornith-mtp-venv/bin/hf download \
  ornith-ai/Ornith-1.5-35B-A3B \
  config.json model.safetensors.index.json model-00016-of-00016.safetensors \
  --revision e4dfb35a93d4b6822a811a7676f3488514abe7e2 \
  --local-dir .build/ornith-mtp-source

.build/ornith-mtp-venv/bin/python tools/prepare_ornith_mtp.py \
  --source-shard .build/ornith-mtp-source/model-00016-of-00016.safetensors \
  --source-config .build/ornith-mtp-source/config.json \
  --source-index .build/ornith-mtp-source/model.safetensors.index.json \
  --output .build/ornith-mtp-affine-4bit

swift run -c release TinyTitanRepack \
  --input-snapshot .build/ornith-mtp-affine-4bit \
  --model-id ornith-1.5-35b-a3b-mtp-4bit \
  --output models/ornith-1.5_35B_A3B_MTP_4Bit

The preparation tool verifies the pinned index, architecture, all 785 MTP tensors, shapes, and dtypes before converting. The local import uses the same hashed manifest, trusted receipt, and SSD expert layout as a normal install. Enable it only for a measured experiment:

.build/release/TinyTitanServer \
  --model models/ornith-1.5_35B_A3B_8Bit \
  --mtp-model models/ornith-1.5_35B_A3B_MTP_4Bit \
  --prompt-cache-mode off \
  --port 8081

MTP requires greedy decoding and native RoPE; it cannot be combined with YaRN or prompt-cache reuse. See Benchmarks before enabling it.

4. Verify and run

Verify without loading the model:

swift run -c release TinyTitanRepack \
  --verify-install \
  --input-gturbo models/ornith-1.5_35B_A3B_8Bit

Run a deterministic smoke test:

.build/release/TinyTitanCLI \
  --model models/ornith-1.5_35B_A3B_8Bit \
  --prompt "The capital of France is" \
  --max-new 32 \
  --temperature 0

Text is written to standard output; timing and token counts are written to standard error. The live KV cache defaults to 8-bit regardless of whether the installed weights are 4-bit or 8-bit. See Runtime Controls for 16-/8-/4-bit KV selection and optional 512K/1M YaRN context.

5. Choose an interface

• CLI and scripts: .build/release/TinyTitanCLI --help
• A prompt box in the browser: tools/server_launcher.sh --web, which starts the server and opens TinyTitan's own DeepSeek Harness on a local page already pointed at the model
• API and coding CLIs: tools/server_launcher.sh, which starts the server on its own or opens Codex, Claude Code, Qwen Code, OpenCode or Zed against it — see Local Server

tools/server_launcher.sh                                   # asks everything
tools/server_launcher.sh --client server --model ornith 4   # API only
tools/server_launcher.sh --client codex  --model ornith 4   # API + client
tools/server_launcher.sh --web --model ornith 4             # API + browser chat

The browser window is optional and set up by tools/dsh_local.sh, which installs a pinned DeepSeek Harness into ~/.tinytitan and keeps it entirely separate from any DeepSeek Harness you already run: its own home, its own npm prefix, its own pnpm store, and port 7788 (the next free port if that one is taken) rather than the stock 3080. The installer offers it as --web, or you can run tools/dsh_local.sh ensure afterwards. tools/dsh_local.sh status reports what is installed and where.

Pick the model with --model and --bits: the keys are ornith, qwen36, agentworld, katcoder, qwen38, and qwen35-2b/qwen35-4b/qwen35-9b; --bits 4|8 selects the width, and a served id from /v1/models names its own. The launcher serves OpenAI-compatible on http://127.0.0.1:8080/v1 and Anthropic Messages at http://127.0.0.1:8080, and accepts any API key because it never authenticates.

If the model directory moves

The verified receipt is bound to the absolute installation path. Moving or renaming the directory causes trusted receipt invalid: model directory mismatch; it does not mean the model is corrupt. Reissue the receipt in place:

swift run -c release TinyTitanRepack \
  --verify-install \
  --input-gturbo /new/path/to/model

Never hand-edit verified-install.json.