Skip to content

Splash 1.0.2

Latest
Latest

Choose a tag to compare

View all tags
@jianc99 jianc99 released this 21 Sep 18:36
1.0.2
e8fffde
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Splash 1.0.2 brings faster Apple9 inference, more flexible API configuration, and improvements to concurrent requests and structured output.

Performance

  • Accelerate Apple9 Q4 and MoE decode, including batch sizes 2–4 while retaining the existing batch-one policy.
  • Improve prefill tile selection on Apple9 GPUs with up to 32 GPU cores.
  • Batch compatible greedy and sampling requests, and bound prefill commands while other requests await token masks.
  • Reuse compiled image templates and JSON Schema validators. Validator caching has bounded retention; independent grammars can compile without holding the shared cache lock.

On M3 Max, the batch extension measured 53–74% higher aggregate decode throughput for 27B and 30–41% for 35B compared with the preceding batch-one-optimized implementation. These are incremental measurements, not comparisons against release 1.0.1 or guarantees for every workload. See #88 for baselines and validation details.

API and usability

  • Add score-only /v1/judgments and /v1/systemone APIs.
  • Configure API model aliases with repeatable --served-model-name options.
  • Set a server-wide default with --default-reasoning-effort or SPLASH_DEFAULT_REASONING_EFFORT; explicit request settings take priority.
  • Configure the server listen address through the launcher and inspect request latency histograms.
  • Support composed tool argument schemas and send Unicode JSON text as UTF-8.
  • Avoid false Metal command timeouts.
  • Remove surrounding blank lines from chat replies consistently during streaming and when reopening a conversation, preserving indentation and the original message content.

Validation

The integrated engine passed numerical, model/vision, HTTP, and prefill/decode checks on M3 Max and 16-core M5 Pro with both supported models, including batch sizes 1–4 and contexts through 128K. Existing real-agent client scenarios were not rerun for this release. Hardware results do not establish universal speedups across all chips and workloads.

Additional contributor validation on M4 Max (32 GPU cores) passed all 39 prefill projection comparisons with Metal shader validation enabled. This covers the target device class for the prefill policy; it is not a full M4 model release gate.

Release packages were also checked for fresh installation and upgrade from 1.0.1 on macOS 26 and 27. Packaged model requests passed on M3 Max and both 16-core and 20-core M5 Pro systems with developer tools blocked.

Upgrade

Stop running Splash servers, then:

brew update
brew upgrade incoai/tap/splash
splash --version

Restart with your usual splash serve --model owner/repo command. Existing model downloads and agent sessions are preserved. The prebuilt packages require no Xcode, Command Line Tools, or system Python.

Requires Apple GPU family 9 or newer and macOS 26.4+.

Full changelog

Assets 6
Loading