Skip to content

1.1.0

Choose a tag to compare

View all tags
@mitmul mitmul released this 07 Oct 09:39
· 3 commits to main since this release
1.1.0
ca67953

Highlights

  • Add a native llama.cpp backend for local GGUF inference, with Metal acceleration on Apple Silicon, using the existing CLI and MCP interface. Streaming, batch, interactive, and one-shot translation are supported. (#23)
  • Add a reproducible llama.cpp build and GGUF conversion workflow. The tested runtime includes a PLaMo 2 RoPE loader fix; the recommended Q4_0-ssm-f16 recipe retains the Mamba output projections in F16. See the setup instructions before using this backend.
  • Correct MLX PLaMo 2 inference, including RoPE settings, Mamba state handling, BF16 activations, and BOS prompting. Add local model selection, additional precision modes, and validated standalone 4bit, 8bit, and BF16 model workflows. (#22, #24)
  • Refresh Python 3.10–3.14 support, pin Transformers below version 5 and MCP below version 2, suppress optional GPU dependency warnings, and improve CLI configuration and regression coverage. (#17, #18, #19, #20, #21)

Install or upgrade

python -m pip install --upgrade plamo-translate==1.1.0

Available on PyPI. The same macOS arm64 wheel and its SHA-256 checksum are attached below.

Using llama.cpp

After building the patched runtime and preparing a GGUF model as described in the setup instructions:

plamo-translate server --backend-type llama.cpp \
  --llama-server /path/to/llama-server \
  --model /path/to/plamo-2-translate-Q4_0-ssm-f16.gguf

# In another terminal:
plamo-translate --from English --to Japanese --input 'Hello, world!'

MLX remains the default backend on macOS. The wheel targets macOS 11 or later on Apple Silicon. llama.cpp binaries and model weights are prepared separately.

See the recorded llama.cpp measurements for quality and performance evidence, including the limits of the single-example evaluation.

Full changelog: 1.0.5...1.1.0

Assets 4
Loading