Whallm 1.1.7

What's new since 1.1.6

Whallm 1.1.7 brings DeepSeek V4.1 Flash support, a built-in Throughput benchmark, and more control over model loading, caching, and updates. This stable release includes the changes from 1.1.7-dev and 1.1.7-dev.2, plus the fixes and model options added since those previews.

Throughput benchmark

• Run benchmarks from the new Throughput page. Select an installed model and Whallm loads it automatically.
• Choose bundled, offline Code or Novel input, context lengths from 1K to 200K, and output limits of 128, 1024, or 4096 tokens.
• View first-token latency, input/output speed, time per output token, total time, throughput, peak MLX memory, and the slots used for each run.
• Copy results as plain text, JSON, or a Markdown table.
• Cancel during input processing or generation. Completed results remain available, and the model unloads when the run finishes, fails, or is cancelled.
• Benchmarks run sequentially with prompt reuse disabled. Model loading and input tokenization are excluded from timing; no extra warm-up is performed. Dry run remains exclusive to local builds and is excluded from this release.

DeepSeek V4.1 Flash

• Added text-only installation, repacking, model discovery, API access, SSD expert and Engram loading, and model-specific tool-call parsing for the pinned DeepSeek V4.1 Flash checkpoint.
• Added Off / Memory / Disk prompt-cache choices, with Memory as the default for new settings.
• Added optional DSpark installation and generation, layer-major input processing, batched expert input processing, next-layer expert prefetch, packed KV/index caches, candidate-only index scoring, and CED input processing.
• Added optional ANE input projection for DeepSeek V4 and V4.1, and an optional approximation mode for all three model families. Approximation and DSpark/MTP cannot be enabled together.
• V4.1 defaults to 1152 slots. Its displayed 501.38 GB weight size comes from the verified installation plan; metadata and network traffic are not included in that figure.

Model settings and runtime fixes

• Max tokens defaults to 8192 for every model. Qwen3.8 defaults to 3072 slots. Explicitly saved custom values remain unchanged.
• Added or exposed controls for prompt caching, prefetch workers, input batch size, LRU/LFU expert-cache eviction, and processing experts as their reads finish. Qwen also gains optional quantized QSA KV/index caches.
• Unloaded models can be configured while the server is running; updated settings take effect when the model next loads. Fixed command-line settings being ignored with a model catalog.
• Reduced overlapping expert memory during long-input processing in DeepSeek V4 and Qwen by releasing old expert slots and avoiding overlapping layer buffers, while retaining the loaded model and prompt cache.
• Added cancellation checks during input processing and shared expert reads, with cleanup before another run starts.
• Removed Qwen's text-derived four-token verification and resident-expert grouped Decode options. Prefill batching and trained DSpark/MTP remain available. Old catalogs that explicitly enable either removed option must be updated; false values remain accepted.
• Unified installation and runtime model descriptions, and moved Log Level to the Logs page.

App updates

• Added stable / dev update sources and an Automatically check for updates setting under Settings → Updates.
• Stable receives official releases; dev also receives development builds. Both can update to this stable 1.1.7 release, including from 1.1.7-dev.2.
• Added English, Simplified Chinese, and Traditional Chinese text for the new controls.

Validation and limitations

• Release checks passed 424 Python tests and 91 Swift tests. The interactive Keychain test was excluded.
• DeepSeek V4.1 remains experimental and text-only. A complete checkpoint installation, full-model generation, and full V4.1 DSpark weights have not been validated.
• New packed-cache, candidate-index, CED, DeepSeek ANE, and approximation options default to off. Some combinations are incompatible and are disabled or rejected by the app/runtime.
• Qwen quantized caches are lossy, and ANE uses FP16; these options may change generated output. No new full-model speed, peak-memory, or quality improvement is claimed for the optional acceleration features.
• Bundled inputs cover 200K tokens with all three tokenizers; this is an offline coverage check, not a verified 200K generation result. Generation may end before the selected output limit.
• This release retains MLX/MLX Metal 0.32.2.

Full changelog: v1.1.6...v1.1.7

Whallm 1.1.7-dev.2

Whallm 1.1.7-dev.2

This alpha adds a Throughput benchmark page and updates model defaults. Select dev in Settings → Updates to receive it. Stable users continue to receive official releases only.

Throughput benchmark

• Select an installed model and run the benchmark; the app loads the selected model automatically.
• Choose Code or Novel. Both are bundled and work offline: a Whallm source snapshot and Herman Melville's Moby-Dick. Each contains enough tokens for 200K inputs with all three supported tokenizers, without repeating a short excerpt.
• Select one or more context lengths from 1K to 200K, with a generation limit of 128, 1024, or 4096 tokens.
• View first-token latency, time per output token, input/output speed, total time, throughput, and peak MLX memory. Results include the slots used by that run.
• Copy results as plain text, JSON, or a Markdown table. The results table fills the available width, and settings controls align on the right.
• Cancel a running benchmark and retain completed results. Tests run one request at a time with prompt reuse disabled; model loading and input tokenization are excluded from timing, and no extra warm-up is performed.

Model settings

• Max tokens now defaults to 8192 for every model. Explicitly saved values remain unchanged; reset or edit Advanced Settings to change an existing preference.

Validation and alpha limits

• Development checks passed 412 Python tests and 82 Swift tests. Three Swift tests require unavailable model fixtures and were skipped; the interactive Keychain round-trip test was excluded.
• Packaging checks verify signatures, notarization, all three localizations, and isolated startup for both the app and an extracted ZIP copy.
• Dry run is available only in local builds. This alpha excludes the simulated-results option and its generator.
• Corpus coverage is an offline tokenizer check, not a measured 200K generation result. No new full-model speed or output-quality claim is made. Actual generation can end before the selected output limit.
• DeepSeek V4.1 remains experimental and text-only; its previous unsupported features remain unchanged.

Whallm 1.1.7-dev

What's new in 1.1.7-dev

This is a pre-release for testing. Stable update checks will continue to offer official releases only. Select dev in Settings → Updates to receive development updates.

App updates and settings

• Added stable/dev update sources, backed by a shared, signed Sparkle update feed. The dev source also accepts newer stable releases.
• Added Automatically check for updates, connected directly to Sparkle's saved preference. Turning it off still allows manual update checks.
• Moved update controls into a dedicated Updates section and aligned the source selector with the other controls on the right.
• Added English, Simplified Chinese, and Traditional Chinese labels for the new settings.

DeepSeek V4.1 Flash preview

• Added initial text-only support for the pinned DeepSeek V4.1 Flash checkpoint across installation planning, repacking, model discovery, the API catalog, SSD expert/Engram loading, and tool-call parsing.
• Fixed V4.1 installation dispatch in the command-line repacker.
• Set the Advanced Settings slot default and recommendation to 1152. Existing saved slot settings are preserved.
• The model list now displays the verified planned weight size of 501.38 GB without fetching an installation plan just to display the size. This is a storage estimate from the repack plan, not measured download traffic.

Runtime controls

• Added Off / Memory / Disk prompt-cache choices where supported, with Memory as the default. V4.1 prompt-cache reuse remains disabled.
• Fixed explicit command-line runtime settings being ignored when starting the server with a model catalog.
• Added controls for prefetch workers, MoE prefill batch size, and LRU/LFU expert-cache eviction.
• Added an optional DeepSeek approximation setting, disabled by default, and Qwen automatic sampling, enabled by default, with manual temperature/top-p/top-k controls when automatic sampling is off.
• Qwen's option to verify up to four tokens together now defaults to off. Explicitly saved choices are preserved.
• Moved Log Level to the Logs page.
• Unified model installation and runtime support around shared model descriptions so the app and bundled runtime use the same model defaults and supported features.

Validation and known limits

• Development validation passed 403 Python tests and 77 Swift tests; three Swift tests were skipped because installed-model fixtures were unavailable.
• Update-setting tests cover saved source selection, Sparkle preference synchronization, development-build version ordering, and localized labels. The Updates layout and controls were also checked in the app.
• Packaged checks cover strict code-signature verification, all three localizations, and startup without access to the project's build directory for both the app and an extracted ZIP copy.
• V4.1 remains experimental: a complete checkpoint installation and full-model generation have not been validated. Vision, MTP/DSpark, prompt-cache reuse, and layer-major prefill are not supported for V4.1 in this release. No V4.1 throughput, memory-use, or output-quality claim is made.
• Selecting stable after installing a dev build does not downgrade the app. A newer stable build can replace it; build 1.1.7d1 sorts before the future stable 1.1.7 build.
• This release retains MLX/MLX Metal 0.32.2. It does not switch the mlx-lm dependency to the planned owned fork, and it does not claim new full-model performance improvements.

Whallm 1.1.6

What's new in 1.1.6

Fix for crashes after a response finishes

• Updated MLX and MLX Metal to 0.32.2 to address the compiled-function cleanup crash investigated in issue #7. The upstream fix acquires the Python GIL when releasing compiled-function data during worker-thread teardown.
• Retains the sampling fix introduced in 1.1.5. This update does not add per-request mx.clear_streams() cleanup, which caused later requests to fail in our tests.
• Existing installed models can be reused; no download or repacking is required for this dependency update.

Model Advanced Settings

• Added Recent Expert Cache, enabled by default in the app, to retain recently used experts within the existing cache capacity.
• Added a default-on Qwen short-block generation option that proposes continuations from previously seen text and verifies them with the main model, allowing up to four tokens to be handled together when eligible.
• Saved settings migrate to the new defaults while explicitly disabled values remain disabled. Changes take effect when the model is next loaded.
• Unsupported configurations, insufficient state-copy budget, and unhelpful proposals fall back to ordinary generation. Qwen short-block generation is not used with MTP or DSpark.

Validation and limits

• MLX 0.32.2 passed 375 development Python tests and 371 packaged-runtime tests. Four research-grader tests were excluded from the packaged run because their sandbox denies the bundled Python path; they passed in the development environment. The initial failed packaged attempt is retained in the validation records.
• Local packaged checks covered 100 worker-thread lifecycles, six consecutive DeepSeek requests with matching output, actual prefix-cache reuse, disconnect recovery, and four Qwen sampling requests.
• These are bounded tests on an M2 Max. Confirmation on the issue reporter's M1 Ultra and broader workload coverage remain open. No universal speedup or memory reduction is claimed.
• Earlier short-block output/state parity and memory measurements used MLX 0.32.1; the complete real-model short-block/MTP and long-context matrix has not been rerun on 0.32.2.

Whallm 1.1.5

What's new in 1.1.5

This release includes changes since 1.1.4 and is built from commit 13d44524c9c12a71f26bee79b33bc345a5163d7f.

Fixes

• Fixed random sampling in server background threads by updating MLX and MLX Metal to 0.32.1. This addresses the stale random state found while investigating repeated Qwen output in issue #6. Packaging now rejects an outdated MLX environment.
• Fixed Codex Responses connections timing out while waiting for model output. The server sends a progress event every 10 seconds while generation is running.
• Fixed generation continuing after Ctrl+C or a client disconnect. Cancellation now reaches queued requests and Qwen input processing and generation, releases resources, and allows subsequent requests to proceed. Incomplete generation is not saved as a reusable cache result.
• Reduced repeated macOS Keychain prompts caused by App state recreation and unrelated settings changes.
• Strengthened packaged App startup checks for English, Simplified Chinese, and Traditional Chinese. Both the App and the extracted ZIP must initialize translations without access to the project's build directory or the Swift fallback resource bundle.
• Added localized local-network permission descriptions and validation.

Qwen and API changes

• Added a default-on Prefill acceleration setting for Qwen, controlling the grouped-expert input processing path. The setting is unavailable when MTP is enabled or layer-major Prefill is disabled.
• /api/status now requires the configured API key, like other protected API endpoints. Clients using this endpoint should send their authorization header.
• Added benchmark and research records. The experimental grouped Decode path remains disabled by default; this release does not claim a general speed improvement from that experiment.

Validation and limits

• Release preparation: Python 338 tests passed; Swift 65 tests passed, with four existing Keychain-interactive tests excluded.

• Real Codex CLI and Qwen checks recorded in the repository cover slow tool responses, Ctrl+C, disconnects during output, and recovery on subsequent requests.

• The sampling fix passed the recorded long-text termination and repetition checks. These checks do not guarantee overall writing quality or eliminate every possible repetition case.

• Cancellation checks are present in the MTP path, but cancellation has not been validated with a real MTP run.

• Signed release artifacts must pass Apple notarization, signature, localization, and isolated startup checks before publication and again after downloading from GitHub.

Whallm 1.1.4

What's new in 1.1.4

This release includes all changes since the last published version, 1.1.2.

Codex and Qwen reliability

• Added Codex model metadata and stable aliases for long namespace tool names.
• Qwen now calls a Codex shell tool before claiming to inspect a project. Required
  tool results are validated and retried once when necessary.
• Responses tool streams now use the complete parsed result as their single source
  of truth. This fixes blank Codex replies caused by valid Qwen tool calls being
  rejected by a second streaming parser.
• Fixed mutable Qwen prompt-cache snapshots. Normal persistent prompt cache now
  uses format 5 and rejects older affected entries.

App and server

• Added Debug, Info, and Error server log levels. Debug records the complete JSON
  body of every parsed request; it may contain prompts, paths, and tool results.
• Advanced settings and aliases can now be changed for models that are not loaded.
  Changes are sent to the running server and apply on the next model load.
• Qwen now defaults to 4,096 Active Parameters Cache slots. MTP remains disabled by
  default.
• Added resumable installation and repair for the optional Qwen MTP sidecar.
• Reduced unnecessary Chat UI updates while streaming long responses.

Runtime

• Added the packaged private ANE Prefill projection path for Qwen, with automatic
  fallback to the original GPU path when the private interface is unavailable.
• Added DeepSeek approximate expert routing as the normal API and CLI default.
  Qwen and DeepSeek with DSpark continue to use exact routing.
• Added reproducible expert-cache, routing, ANE, MTP, and SSD benchmark artifacts.
  Experimental paths that did not pass their gates remain disabled.

Benchmark comparison with v1.1.0

Measured on the same MacBook Pro with Apple M5 Pro, 64 GB unified memory, and
1 TB storage. Both versions used the same mixed SPEED-Bench dataset files,
input sizes, /v1/completions endpoint, and 64-token output limit.

Values in parentheses show improvement over v1.1.0. Positive values are
improvements; negative values are regressions. Lower is better for total time,
TTFT, and memory; higher is better for Prefill and Decode throughput.

DeepSeek V4 Flash 0731

Input tokens	P95 total time	P95 TTFT	P95 Prefill	P95 Decode	Peak memory
1,024	27.10 s (+42.0%)	17.75 s (+52.4%)	59.9 tok/s (+188.7%)	7.8 tok/s (+15.6%)	32.84 GiB (−42.6%)
2,048	27.35 s (−1.0%)	17.60 s (−4.0%)	117.3 tok/s (+8.5%)	7.3 tok/s (+11.7%)	33.26 GiB (−0.2%)
8,192	50.12 s (−5.2%)	40.47 s (−6.6%)	206.3 tok/s (−1.5%)	7.4 tok/s (+10.3%)	34.50 GiB (+0.7%)
16,384	88.27 s (−2.6%)	78.61 s (−3.0%)	209.0 tok/s (−1.5%)	7.2 tok/s (+8.0%)	35.70 GiB (−0.2%)

Qwen3.8 Next Flash FP8

Input tokens	P95 total time	P95 TTFT	P95 Prefill	P95 Decode	Peak memory
1,024	22.60 s (+7.6%)	15.65 s (+9.0%)	69.2 tok/s (+15.7%)	10.4 tok/s (+11.6%)	20.92 GiB (−37.7%)
2,048	31.42 s (+22.3%)	24.90 s (+23.7%)	87.5 tok/s (+34.7%)	9.8 tok/s (+18.1%)	21.26 GiB (−29.6%)
8,192	84.88 s (+40.4%)	77.74 s (+42.2%)	111.9 tok/s (+81.4%)	10.4 tok/s (+25.0%)	21.90 GiB (−22.3%)
16,384	157.89 s (+42.9%)	150.33 s (+43.9%)	113.3 tok/s (+83.4%)	9.7 tok/s (+23.9%)	22.75 GiB (−20.3%)

The Qwen measurement used 4,096 cache slots instead of 1,152 and enabled the
25% ANE Prefill projection. This configuration used more memory, while Qwen
throughput was higher in these measurements. The DeepSeek measurement used the
v1.1.4 default approximate routing.

v1.1.4 used three runs per input size, while v1.1.0 used two. Nearest-rank P95
therefore equals the maximum in both sets. The filesystem cache was not purged.
These percentages are a descriptive release comparison, not a paired causal
measurement or a performance guarantee. See the
full benchmark
and the raw
DeepSeek
and
Qwen
artifacts.

Validation and limits

• Swift: 68 passed, 0 skipped, and 0 failed.
• Python: 294 passed, 0 skipped, and 0 failed.
• Codex CLI 0.153.0 completed both the initial required Qwen tool flow and the
  following automatic tool flow without invalid_tool_call or an empty reply.
• The App and extracted ZIP passed Developer ID signature, localization, and
  isolated startup checks. The release workflow verifies notarization before and
  after publication.
• Tool-enabled Responses are buffered until complete parsing finishes. This favors
  reliable tool calls over incremental tool-event delivery.

Whallm 1.1.2

What's new in 1.1.2

• Corrected the canonical GitHub repository name to yanun0323/Whallm.
• Updated the app, documentation, Sparkle feed, and release workflow to use the
  corrected repository URL.

This release does not change the model runtime or installed model format.

Whallm 1.1.1

What's new in 1.1.1

• Updated the app, documentation, and release workflow to use the renamed
  yanun0323/Whallm GitHub repository.
• Updated the Sparkle feed and release links to use the new repository URL.

This release does not change the model runtime or installed model format.

Whallm 1.1.0

What's new in 1.1.0

This release includes all changes since version 1.0.5.

Whallm and Qwen

• Renamed the app and release files from DeepSeekV4SSD to Whallm. The new files
  are Whallm.app and Whallm-macOS-arm64.zip.
• Added text support for the pinned Qwen3.8-Flash-Next-FP8 checkpoint alongside
  DeepSeek-V4-Flash-0731.
• The app downloads a pinned, verified MXFP4 Qwen installed model. The app does
  not convert the FP8 checkpoint on your Mac.
• dsv4-repack can also inspect the pinned Qwen checkpoint and build the MXFP4
  installed model with resumable conversion.
• Qwen supports text chat, thinking, prompt cache, and OpenAI-compatible tool
  calls. Qwen vision, video, MTP, and DSpark are not supported.
• Added Qwen download resume, file verification, and damaged-file repair.

Model management

• Added a Model page for DeepSeek and Qwen. It shows each model folder, status,
  required space, available space, download progress, verification, repair,
  DSpark controls, and advanced settings.
• Added manual Load and Unload controls. The loaded model appears in a separate
  Loaded section.
• The server can start with no installed model. It reads the model catalog at
  startup but does not load model weights.
• The first generation request loads its selected model. The server keeps one
  model runtime loaded and closes it before it loads another model.
• Model downloads can continue while the server uses another installed model.
  The app runs one model download at a time.
• Added per-model settings. Alias changes are validated and saved automatically.
• New DeepSeek downloads always include DSpark. Installing DSpark does not
  enable it, and you can remove DSpark without reinstalling the main model.
• Added a Chat model picker. Changing the model keeps the conversation. A reply
  also continues when you open another app page.
• Assistant messages keep the model name that produced each response.
• Metric shows the loading or loaded model. Metric history clears after a model
  change.

API changes

• Added fixed API model IDs and optional, case-sensitive aliases for both models.
• Generation endpoints accept an API model ID or Alias. Responses and streaming
  events return the requested name.
• GET /v1/models lists the models and aliases that were available when the
  server started.
• Added POST /api/models/load and POST /api/models/unload.
• The server processes one generation request at a time. Other generation
  requests wait until the current response stream ends.
• Removed GET /api/settings and PUT /api/settings. Both endpoints now return
  404 not_found.
• Fixed Qwen conversion for Codex Responses instructions, tool definitions, tool
  history, function arguments, and function results.

Runtime and prompt cache

• Added a configurable DeepSeek layer-major Prefill threshold. The default is
  1,024 uncached prompt tokens.
• Qwen uses a model-safe automatic memory limit and avoids loading every expert
  layer for short prompts.
• Replaced the old persistent prompt cache with format 4. The new format checks
  the full model and cache contract, uses content-addressed 128-token blocks,
  and can reuse a safe prefix when the prompt suffix changes.
• Old normal prompt-cache formats are not loaded. The runtime creates new cache
  entries automatically.
• Added reproducible diagnostics for DSpark verification, expert I/O, MTLIO,
  cache policy, and Prefill candidates. These paths remain disabled by default
  or research-only because they did not pass the required performance gates.
• Added repeatable API benchmarks for 1,024, 2,048, 8,192, and 16,384 input
  tokens for both models. These results are local measurements, not performance
  guarantees.
• Updated the README content in English, Simplified Chinese, Traditional Chinese,
  Japanese, and Korean.

Validation and known limits

• Swift: 61 passed, 0 skipped, and 0 failed.
• Python: 239 passed, 0 skipped, and 0 failed.
• The Qwen full-model checks passed for all 57 manifest files, text, thinking,
  tool calls, greedy 4K generation, prompt cache, and packaged app startup.
• The signed and notarized app and extracted ZIP passed Developer ID, Gatekeeper,
  localization, and isolated startup checks.
• The direct Qwen download path passed resume and SHA-256 tests. This validation
  did not repeat a complete 125 GB app download.
• Automated multi-model lifecycle tests use fake runtimes. This validation did
  not run a real two-model switch or measure its process memory.