Perplexity Full Batch Capacity

Perplexity full-batch output capacity

The BeeLlama feature line changes llama-perplexity context construction so the tool declares n_batch as its maximum output-row requirement before the context is created. This preserves full-logits evaluation when phase-aware serving defaults would otherwise reserve only a serving-sized output set.

Why it matters

Perplexity requires logits for every evaluated token in the batch. A server can often request a much smaller output set, but applying that serving assumption to llama-perplexity makes the tool's capacity contract wrong. The fix is plumbing and allocation correctness; it does not define a new perplexity formula, sampling rule, or performance mode.

There is no new CLI flag. Existing upstream llama-perplexity usage remains unchanged and should be read from the upstream tool documentation.

Implementation

• Tool setup: tools/perplexity/perplexity.cpp
• Context output-capacity plumbing: src/llama-context.cpp
• Coverage: tests/test-perplexity-plumbing.cpp

This fix is currently published only on beellama/main. Review history: PR 5.

Feature index