MoEShift 0.1.0 Alpha 3

Guided first-run setup

MoEShift now guides you from model selection to a running local server. Run moeshift, choose a tested model or use GGUF files already on your Mac, and let the wizard prepare the experts and configuration.

MoEShift builds on llama.cpp, llama.cpp-omni and Lookahead Expert Scheduling (LES).

Included

• Interactive setup with arrow-key navigation and disk requirements for each model.
• Presets for ISTA-DASLab Qwen3.8 Flash Next GSQ-RCO IQ3_XXS and Unsloth Qwen3.6-35B-A3B UD-Q6_K_XL.
• Local GGUF reuse with automatic detection of adjacent shards.
• Downloads through hf, including optional temporary installation, or direct HTTPS, with progress and ETA.
• Automated expert preparation, verification and configuration, followed by server startup.
• Configuration recovery with a backup of the original file.
• Manual configuration with a linked setup guide.
• Animated verification and a success summary showing the API URL and model ID.

Requirements

• Apple Silicon Mac with 10 GB RAM or more.
• Fast SSD with approximately 120 GB available for Flash Next or 61 GB for Qwen3.6, including GGUFs and prepared experts.
• Access to Hugging Face for downloads, or compatible model files already available locally.

Managed downloads and experts are stored under ~/.moeshift/models/. Existing local GGUFs are used in place.

MoEShift 0.1.0 Alpha 2

Qwen3.6 MoE support

MoEShift now runs Qwen3.6-35B-A3B alongside Qwen 3.8 Flash Next locally on Apple Silicon, loading experts from SSD to reduce the amount of model data kept in memory.

Built on llama.cpp through llama.cpp-omni and its Lookahead Expert Scheduling (LES) foundations.

Included

• Optimized expert execution shared across both supported architectures.
• Adaptive prompt-processing and generation budgets, protected LRFU caching and Metal resident/missing execution for Qwen3.6.
• Qwen3.6 profile in the standard JSON configuration.
• Updated model download links, setup instructions and benchmark results.
• Extended Metal checks for Q6_K and Q8_0 expert execution.

Requirements

• Mac with Apple Silicon and at least 10 GB of RAM.
• Fast SSD with space for the original GGUF and prepared expert files.
• Original model files downloaded separately.

Tested models are Qwen 3.8 Flash Next with ISTA-DASLab GSQ-RCO IQ3_XXS and Qwen3.6-35B-A3B UD Q6_K XL.

Performance

On a Mac mini M4 Pro with 24 GB unified memory, Qwen3.6-35B-A3B UD Q6_K XL measured median generation speeds of 16.50 tokens per second with a short prompt and 14.66 with a long prompt.

Results use 512 output tokens and three measured runs after warmup, without cached prompt tokens. Peak process memory was not recorded.

The existing Flash Next comparison with Strata remains unchanged. See the README for complete results and measurement conditions.

Alpha status

Validated on M4 Pro. Seven focused native tests and nine HTTP router tests passed, alongside real-model checks for both supported models.

Existing configuration files are preserved. Copy the Qwen3.6 profile from the bundled template to enable it, then set your model and expert-directory paths.

Performance varies with hardware, prompt length and cache state. Please report installation or inference problems through the repository's issue forms.

See the README for setup instructions and the architecture document for runtime details.

MoEShift 0.1.0 Alpha 1

First alpha release

MoEShift runs Qwen 3.8 Flash Next locally on Apple Silicon, loading experts from SSD to reduce the amount of model data kept in memory.

Built on llama.cpp through llama.cpp-omni and its Lookahead Expert Scheduling (LES) foundations.

Included

• An OpenAI-compatible HTTP server on localhost.
• Expert loading overlapped with Metal computation.
• A demand-LRFU expert cache and separate prompt-processing and generation budgets.
• Growing context buffers and key-only indexer storage.
• Tools to extract and verify expert files from the original GGUF model.
• Automatic configuration creation on first launch.

Requirements

• Mac with Apple Silicon and at least 10 GB of RAM.
• Fast SSD with at least 120 GB of free space for the reference model and prepared files.
• Original model files downloaded separately.

The reference model is Qwen 3.8 Flash Next with ISTA-DASLab's GSQ-RCO IQ3_XXS quantization.

Performance

On a Mac mini M4 Pro with 24 GB unified memory, the long-prompt comparison with Strata measured 63.3 versus 129.9 seconds of inference time and 7.5 versus 14 GB peak process memory.

See the README for the complete results and measurement conditions.

Alpha status

Validated on M4 Pro. Performance varies with hardware, prompt length and cache state. Please report installation or inference problems through the repository's issue forms.

See the README for setup instructions and the architecture document for runtime details.