Skip to content

0.3.3

Choose a tag to compare

@github-actions github-actions released this 13 Jul 22:46
· 9 commits to main since this release
Strip tokenizer padding before inference (served scores were wrong)

A tokenizer.json with a `padding` section — which real HuggingFace repos ship,
including our own published phishing classifiers — makes the Rust tokenizer pad
every encoding to a fixed width. transformers does not pad by default. We
forwarded those [PAD] ids to the model under an all-ones attention mask, so it
attended to ~500 padding tokens as if they were text.

Impact: every score served for the published catalog models was wrong. On the
probe 'Account notice: sign in to review recent activity.' the real model
returned LABEL_0=0.0005 where HuggingFace returns 0.7474. Affects 0.2.0-0.3.2
(0.3.2 was never released).

- drop trailing pad ids (pad_id read from tokenizer.json) before building the
  inputs, reproducing the HF reference exactly (now 0.7474327 vs 0.747434)
- new tiny-pad fixture: tiny-clf with padding enabled in tokenizer.json; its
  golden scores are tiny-clf's, so any regression that lets pad ids through
  fails immediately. The existing fixtures could not catch this — their
  tokenizers declare no padding, which is why it shipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>