Repository navigation
README
Prompts, requests, profiles and selected recorded measurements accompany the wiki guide. No model weights or native expert/MTP packs are supplied. The Flash Q2/F16 draft head used in the recorded run is a local conversion, not a stock Q8 head; substitute your matching head only as a new configuration.
The dossiers contain MIT-licensed documentation extracts from llama.cpp and Strata. Retain llama-MIT-LICENSE and strata-MIT-LICENSE when copying these files. Profiles are model-specific rankings/statistics, not model weights. Do not reuse a profile with different expert geometry or model identity.
benchmark-affinity.cpp is a Linux-only benchmark interposer for the generic CPU pool. It pins pool workers to distinct physical cores within taskset's allowed CPUs. The recorded binary also had an older caller hook that does not match the published pool-run ABI, so caller pinning is not claimed. The supplied source omits that inactive hook and preserves worker placement. It adds no execution kernels or residency policy, and is not a production dependency. Recorded Flash/Ornith/repeated-Gemma runs used this helper. Nemotron/long-Gemma/DeepSeek commands below did not. Do not use it for concurrent contexts: its task-creation inventory requires isolated pool creation. Model-serving concurrency itself does not require the helper.
MANIFEST.json pins all assets except itself. GPU jobs were not run to write this guide. The original benchmark logs and complete runtime closures remain in the owner's archived results; measurements.json is a selected public export, not a substitute for every raw trace.
GenerelSchwerz llama.cpp
- Home
- Discord community
- Contributors
- Complete feature index
- Hardware setup guides
- Benchmarks
- BeeLlama Main
- Llama Main
- Llama Dev
- MoE Cache
Feature groups
- Memory placement and workspace
- Validation and diagnostics
- CUDA MoE cache and helpers
- Grouped MoE drafting
Source branches