Dense Penalty Counts

Dense sampler penalty counts

The penalties sampler maintains a vocabulary-sized dense count table for valid token IDs while retaining its map for out-of-range IDs and compatibility paths. This removes repeated hash lookups from the hot penalty application loop.

Semantics

The dense table is updated together with the map as tokens enter and leave the configured penalty window. Reset and clone copy or clear both representations. Duplicate candidate IDs receive the same count, and repeat, frequency, and presence penalty formulas are unchanged.

There is no flag or public API change. This is a generic sampler optimization carried by the MoE branches; it is not specific to MoE models and does not enable backend sampling.

Correctness requires the map fallback because extensions or malformed candidate lists can contain IDs outside the vocabulary-sized array. The fast path therefore checks range before indexing rather than assuming every caller is canonical.

Branch implementations

Branch	Implementation	Coverage
moe-cache	src/llama-sampler.cpp	tests/test-sampling.cpp

Normal sampler controls and formulas remain documented by upstream llama.cpp.

Feature index