Add Mobile300 Prime-RL token-suffix staging#864
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 08934d5fc5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| rendered = tokenizer.apply_chat_template( | ||
| messages, | ||
| tools=tools, | ||
| tokenize=False, | ||
| add_generation_prompt=False, | ||
| ) |
There was a problem hiding this comment.
Preserve per-row chat template kwargs when rendering
When a local JSONL row already carries chat_template_kwargs, this new suffix renderer calls apply_chat_template without forwarding them and _custom_trainer_token_suffix_row later drops the field from the staged row. In that scenario—for example Qwen rows that set enable_thinking=False—the staged token suffix is rendered with tokenizer defaults, so custom-trainer-token-suffix trains on a different token tail than the row/custom-trainer path requested.
Useful? React with 👍 / 👎.
Summary
Fixes the Mobile300 PR828 Prime-RL SFT compatibility path in BenchFlow by staging the dataset the way the historical custom trainer actually trained it:
tool_defs/toolsfor the compatibility profile, matching the archived PR828 rowsstackpacking gives one 8192-token row per sampleThis keeps the fix in BenchFlow’s Prime-RL wrapper and avoids patching Prime-RL internals.
Why
The Mobile300 regression was not caused by the BenchFlow result JSONL schema alone. The key mismatch was that Prime-RL was tokenizing/truncating/loss-masking rows differently from the old custom trainer, causing a behavior collapse with many zero-tool eval attempts. The
env0-mobile300-pr828compatibility profile now encodes those historical semantics explicitly.Validation
uv run pytest tests/test_train_cli.py -quv run ruff check src/benchflow/training/backends/prime_rl.py src/benchflow/cli/train.py tests/test_train_cli.pyuv run ruff format --check src/benchflow/training/backends/prime_rl.py src/benchflow/cli/train.py tests/test_train_cli.pyuv run ty check src/benchflow/training/backends/prime_rl.pySpendful validation is intentionally not in this PR; the next step is a real Mobile300 dry-run/train launch on the H100 from this branch.