Skip to content

v0.42.0

Choose a tag to compare

@mrothroc mrothroc released this 30 Jun 23:49
· 342 commits to main since this release

score-diffusion mode: block-causal PLL scoring

Add a score-diffusion CLI mode that scores already-tokenized candidate
sequences with a block-diffusion checkpoint's native forward, for forced-choice
ranking.

  • Each scored position is masked exactly once and evaluated under the
    prefix-plus-block attention graph used by block-diffusion training; the masked
    position's log-softmax of the true token becomes its block-causal PLL.
  • Positions are scored in memory-bounded batches (-score-position-batch, auto
    ~256 MiB of logits); chunked scoring matches single-position scoring.
  • JSONL in/out: {"id","tokens","score_from?"} to {"id","n_tokens", "score_from","logprob_sum","logprob_mean","per_token"}. Per-record
    score_from overrides the global -score-skip-first. Accepts pure and hybrid
    block-diffusion configs.

The result is a deterministic block-causal pseudo-log-likelihood for ranking, not
a normalized sequence likelihood. See cli-eval.md for flags
and JSONL schemas.

Verified: stub + MLX builds, golangci-lint clean, CGO_ENABLED=0 + full MLX suites, with chunk-vs-single-position parity (CPU oracle and MLX) plus scoring/JSONL/validation tests.