Skip to content

fix: constrained decoding cache bug and task rotation#199

Merged
abrichr merged 1 commit into
mainfrom
fix/constrained-decoding-and-task-rotation
Mar 28, 2026
Merged

fix: constrained decoding cache bug and task rotation#199
abrichr merged 1 commit into
mainfrom
fix/constrained-decoding-and-task-rotation

Conversation

@abrichr

@abrichr abrichr commented Mar 28, 2026

Copy link
Copy Markdown
Member

Summary

Fixes two bugs reported by customer on their first standalone trainer run:

  1. Constrained decoding not activating: Outlines RegexLogitsProcessor failed silently on DFA compilation (the (.|\n)* prefix was too complex). Failure was cached as [] which passed the is not None check, so model.generate() received an empty logits_processor list — unconstrained output. Fixed: removed prefix from regex (model must output action directly), use False sentinel for failures.

  2. Task rotation stuck on first task: _load_task_configs checked if not self._config.task_ids inside the loop — after appending the first task, the list was non-empty so remaining tasks were skipped. Fixed: check once before the loop.

Tests

21 new tests covering:

  • Action regex: 8 valid actions match, 6 invalid texts rejected
  • Cache sentinel: failure returns None (not []), success returns list
  • Regression: empty list [] no longer treated as successful cache
  • Task loading: all tasks from dir, explicit ids preserved, rotation math

All 21 pass in 0.03s, no GPU or server required.

Test plan

  • 21/21 pytest tests pass
  • Regex matches all valid action formats, rejects free-text
  • Customer re-runs with constrained_decoding=True and verifies structured output

🤖 Generated with Claude Code

Constrained decoding:
- Remove (.|\n)* prefix from action regex — Outlines can't compile
  it into a DFA efficiently. Model must output action directly.
- Fix cache sentinel: use False for failure (not []) so subsequent
  calls correctly return None instead of empty logits_processor list.
  Prior bug: [] cached as "success" → model generated unconstrained.
- Upgrade warning to error level for visibility.

Task rotation:
- Fix _load_task_configs: check `not task_ids` once BEFORE the loop
  (was checking inside loop — only first task ever appended).

Tests (21 new):
- TestActionRegex: 8 valid actions match, 6 invalid texts rejected
- TestConstrainedDecodingCache: sentinel logic, regression for [] bug
- TestTaskRotation: all tasks loaded, explicit ids preserved, rotation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant