Confused about the extract function in rlvr math_rule_reward_worker.

[canghongjian]

In roll/pipeline/rlvr/rewards/math_rule_reward_worker.py, which is the default reward worker class for the math dataset rlvr training, the _extract_after_last_end_think func requires the response should not contain the beginning think tag <think>, otherwise the reward will be 0. It's quite unreasonable, and there is no clues in the math training dataset data/math_deepmath_deal.jsonl that response should not have the beginning think tag. This will cause an invalid rlvr training, and I suggest using the last_boxed_only_string func, which you can refer to slime.

[wwxFromTju]

Hey, thanks for your suggestion! Actually, we don't enforce any specific reward calculation method. If you'd like to reuse existing reward evaluation code, you can simply replace the _extract_after_last_end_think definition here.

Additionally, we encourage users to implement custom reward workers. You can refer to the examples in this directory for guidance. To route to a different reward worker, you only need to modify the worker_cls field in your config file. For reference, see this example:

worker_cls: roll.pipeline.rlvr.rewards.ifeval_rule_reward_worker.GeneralRuleRewardWorker