Batched decode#3

Merged

mudler merged 11 commits into

worktree-batched-encoderfrom

Jun 1, 2026

Commits on May 31, 2026

refactor(decode): share argmax/max_prob_conf in decode_common.hpp

mudler
and
claude
committed
feat(prediction): step_batch (batched LSTM, [H,N])

mudler
and
claude
committed
docs(prediction): explain the 4H gate-slice stride in step_batch

mudler
and
claude
committed
feat(joint): step_logits_batch (batched joint, [V_plus,N])

mudler
and
claude
committed
feat(decode): transducer_greedy_batch (batched RNNT+TDT greedy, bit-exact parity)

mudler
and
claude
committed
refactor(decode): extract commit_state lambda in transducer_greedy_batch; document max_symbols assumption

mudler
and
claude
committed
feat(model): batched transducer decode in transcribe_pcm_batch[_with_timestamps]

mudler
and
claude
committed
bench: add bench-decode (batched vs serial transducer decode timing)

mudler
and
claude
committed
perf(decode): cache prediction-net g across rounds in transducer_greedy_batch (bit-exact; recovers B=1, fewer LSTM calls)

mudler
and
claude
committed
refactor(model): extract batch_enc_to_row_major helper (dedup batched-decode transpose)

mudler
and
claude
committed
docs(decode): correct header comment to describe the g_valid cache

mudler
and
claude
committed