Fix #547: scalar fallback for i2_s dot kernels on x86 without AVX2

[amitpatole]

Summary

Fixes #547 — i2_s inference produces garbage output (a repeating token loop such as GGGG… / !!!!…) on x86 CPUs without AVX2 (e.g. Ivy Bridge, or any build configured with -DGGML_AVX2=OFF), while standard quants like Q2_K work fine on the same hardware.

Root cause

The four ggml_vec_dot_i2_i8_s_* kernels in src/ggml-bitnet-mad.cpp are gated as:

#if defined(__AVX2__)
    ... // AVX2 implementation
#elif defined(__ARM_NEON)
    ... // NEON implementation
#endif   // <-- no #else

On an x86 target without AVX2, neither branch is compiled, so each kernel has an empty body and never writes its output s. The uninitialized dot-product results propagate into the softmax, and generation collapses to a constant token. This matches the reporter's environment exactly (AVX = 1 | AVX2 = 0 | MATMUL_INT8 = 0).

Fix

Add a portable scalar #else fallback to all four kernels (_1x1, _1xN, _Nx1, _1x4_32W) through a shared helper:

static inline int ggml_i2_s_block_dot(const uint8_t * px, const int8_t * py, int nb);

It reproduces the AVX2 ternary unpack bit-for-bit: 2-bit weight codes are extracted at bit offsets 6/4/2/0 and paired with activation sub-blocks 0/1/2/3 (32 lanes each), summed into int32. QK_I2_S is also defined for the no-SIMD case so the 128-element block layout is interpreted identically. No behavioural change on AVX2 or NEON targets — the new code only compiles when neither SIMD path is available.

Verification

1. Bit-exactness vs AVX2 (unit). tests/test-i2s-scalar-fallback.cpp re-implements the exact AVX2 kernels as ground truth and cross-checks the scalar fallback over 7163 random cases across all four kernels — 0 mismatches.

g++ -O2 -mavx2 tests/test-i2s-scalar-fallback.cpp -o /tmp/t && /tmp/t
checks=7163 fails=0 -> ALL BIT-EXACT MATCH

2. End-to-end (real model). Built with -DGGML_AVX2=OFF (AVX = 1 | AVX2 = 0 | SSSE3 = 1 | MATMUL_INT8 = 0) and ran BitNet-b1.58-2B-4T (ggml-model-i2_s.gguf):

	Output for prompt "The capital of France is"
Before (main)	The capital of France is!!!!!!!!!!!!!!!!!!!!!!!!…
After (this PR)	The capital of France is Paris. Paris is a city that is known for its rich history, culture, and architecture. It is also a major center for art, fashion, and cuisine…

Notes

• Scope is limited to the x86-without-AVX2 case reported in [Bug]: i2_s quantization produces garbage output on x86 CPUs without AVX2 (Missing fallback) #547; the AVX2 and ARM NEON paths are untouched.
• Separate from the other in-flight i2_s fixes (to_float UB in fix: wrap dequantize_row_i2_s to_float callback to fix UB on ARM #469/fix(i2_s): resolve to_float UB + guard BLAS I2_S routing #533, NEON int16 overflow in fix: prevent int16 overflow in NEON non-dotprod fallback path #459/Fix Word Salad (Tensor Corruption) on ARM Devices for i2_s Quantization #551) — this addresses a distinct, currently-unhandled build configuration.

[amitpatole]

@microsoft-github-policy-service agree