Repository navigation
v2.5.4 — Bench/Chat UX Fixes, Non-BMI2 VQ3_0 Fix, Cross-Platform Onboarding, & Benchmark Precision
v2.5.4 — ./modchip bench & Chat UX Fixes, Non-BMI2 VQ3_0 Scale Unpack & Kernel Test Fixes, Cross-Platform Windows/macOS/Docker Onboarding, & Benchmark Table Precision
Highlights
-
./modchip benchCrash Fix & Dynamic Neutral System Prompt in./modchip chat(scripts/chat.py):- Fixed a
NameError: name 'prompt_label' is not defined(and remote-endpointUnboundLocalErroroncpu/ram) at the end ofrun_benchmark_suite()when generating the pre-filled GitHub Leaderboard submission URL in./modchip benchand/bench. - Replaced the hardcoded Italian
DeepSeek-V4/Ryzen 9 9955HXsystem prompt in./modchip chatwithbuild_chat_system_prompt(), which dynamically reflects the active served model (Nemotron,DeepSeek-V4,Qwen, etc.) and detected host hardware, responds in the user's language, and refreshes on/model.
- Fixed a
-
Non-BMI2
VQ3_0Scale Unpack Fix, AVX512F Portability,test-quantize-fnsCoverage, & Transcode Layout Parity (vq3/vq3_codebook.h,patches/lut-vq3-64-avx512-vbmi.patch,patches/dsv4-ipc-hardening.patch,vq3/vq3_enc_lib.c,vq3/test_vq3_ggml_kernels.c):- Fixed a bit-carry bug (
|=->+=) in the#elsenon-__BMI2__fallback ofggml_vq3_unpack_scales(m_all = 0x0202020202020202ULL), eliminating scale unpacking error when compiled without-mbmi2(such as MSVC or scalar non-BMI2 builds; distributed Linux-avx512and-avx2binaries use__BMI2___pdep_u64and were unaffected, and zero GGUF weights changed). - Replaced AVX512DQ
_mm512_extractf32x8_psinops.cppwith pure AVX512F_mm512_extractf64x4_pd, addedGGML_TYPE_VQ3_0thresholds totests/test-quantize-fns.cpp(verifiedPASSon both-avx2and-avx512GLIBC_2.30builds), aligneddequantize_row_q2_0_localinvq3/vq3_enc_lib.c(--aggressiveoptional transcoder) toquants.c's transposedqhlayout, and tightenedvq3/test_vq3_ggml_kernels.cto validate both near-zero cancellation and positive-signal relative error (exit 0under both-mbmi2and-mno-bmi2).
- Fixed a bit-carry bug (
-
Windows 11 Speculative Flag Detection (
Issue #33),-Target nemotronIdempotency, &-CudaISA Auto-Detection (windows/modchip_win.ps1,windows/build_patched_engine.ps1,patches/zen5-nemotron-256k-optimizations.patch,windows/modchip_dsv4.ps1):- Fixed
modchip_win.ps1matching--spec-ngram-size-ninsidellama-server.exe --helpremoval warnings before checking--spec-ngram-simple-size-n(thanks to@kevinbloin Issue #33), and removed legacy--draft-max. - Guarded POSIX
<sys/mman.h>/<fcntl.h>/<unistd.h>includes directly with#if defined(__linux__)inzen5-nemotron-256k-optimizations.patch, madebuild_patched_engine.ps1 -Target nemotronidempotent across repeated runs, and replaced unconditional-DGGML_AVX512=ONunder-Cudawith host AVX-512 VBMI auto-detection so AVX2 + RTX PCs never hitSTATUS_ILLEGAL_INSTRUCTION.
- Fixed
-
macOS (
Darwin),.modchip_workspace, Docker/dev/shmGuard, & Standalonellama_modchip.pyParity (install.sh,modchip,scripts/build_zen5_llama.sh,apple-silicon/run-server.sh,scripts/llama_modchip.py):- Replaced GNU-only
realpath -mand baresha256sumwith portable POSIX/Python +shasum -a 256helpers, persisted custom--workspacepaths via.modchip_workspaceacross./modchipand./scripts/chat.py, bypassed/dev/shmslot-snapshot allocation in64 MiBDocker containers (< 1 GiBtmpfs), routedDarwindsv4source builds topatches/apple-siliconandDarwin./modchip startto${WORKSPACE_DIR}with unified-memory--load-mode mmap, and added RAM-safe-c 32768 -ub 512,--load-mode, andDSV4_MOE_CFG_PATHdefaults to standalonellama_modchip.py --print-cmd(marking boot-time#2/#9controls asBOOTin--tui).
- Replaced GNU-only
-
Benchmark Table Separation, Documentation Precision, & 7-Repo Hugging Face Sync (
README.md,benchmarks/QUALITY_GATE_VQ3_64.md,docs/NEMOTRON_OPTIMIZATIONS.md,ARCHITECTURE.md,FAQ.md):- Explicitly separated
Raw Pure Decode (N=1)fromServed Gate Decode (Stockfish-PVS E2E)andIn-Context Refactor Burst (M=7), footnoted the13.08 GiB--all-up-expseval_nemotron_vq3_64.jsontelemetry andAbliterated-VQ3-64(401total tensors =76re-quantized +325copied), added@kevinblo'sRyzen 9 9950X3D(9.47 tok/sTop-4/8.11Top-6284B &24.92 tok/sNemotronQ4_0) leaderboard entries, and synchronized all 8v2.5.4GLIBC_2.30binaries +SHA256SUMS+ model cards across all 7Synchopate/*Hugging Face repositories (0 GGUF weight changes).
- Explicitly separated
Portable GLIBC_2.30 Linux x86_64 Binaries (SHA256SUMS)
Built inside Ubuntu 20.04 (GLIBC_2.30 max symbol, -static-libstdc++, zero libssl/libcurl dependency; requires OpenMP libgomp1 / libgomp):
*-avx512: AMD Zen 4/5 (Ryzen 7000/8000/9000,EPYC Genoa/Turin,Threadripper 7000) & Intel Ice Lake / Sapphire Rapids (avx512f+avx512vbmi;-DGGML_AVX512_BF16=OFF).*-avx2(and unsuffixed*-linux-x86_64): Pre-AVX512x86-64-v3CPUs (Haswell,Skylake,Alder/Raptor Lake,Zen 1/2/3—0%zmminstructions).
c9fe218d3be566e38c7677c903013ee6d90fa3663aa87d3bbc78c2109e8ac2c4 llama-cli-modchip-linux-x86_64
c9fe218d3be566e38c7677c903013ee6d90fa3663aa87d3bbc78c2109e8ac2c4 llama-cli-modchip-linux-x86_64-avx2
6c9317d68f47d4c9c23740ff597d56cafb83ad816f8581cb5315bc11ebb6e163 llama-cli-modchip-linux-x86_64-avx512
fa214d0f86a37934dcf9c49e32e8d40d5406e6b970d968aac1e093b952c80bdd llama-perplexity-modchip-linux-x86_64-avx2
282183f33732666ffd34ecbc5258de107267e21797f5205c70ff107bc1d81d84 llama-perplexity-modchip-linux-x86_64-avx512
09f2a1accacb1454aaf0d9a8d75dda8f7cca36df81138eb57bfee45137ef07e4 llama-server-modchip-linux-x86_64
09f2a1accacb1454aaf0d9a8d75dda8f7cca36df81138eb57bfee45137ef07e4 llama-server-modchip-linux-x86_64-avx2
7cf2563ede8670b482ccc7e108d9810023f994a6ce9a3d7bd320f8393dd26304 llama-server-modchip-linux-x86_64-avx512