[About Credit Assignment] How to implement an action-dependent reward function?

[Kwen-Chen]

The credit assignment in the sample code applies a uniform policy to all states. However, I'd like to assign rewards differently based on the specific action taken. What would be a general approach to implement this?

[ultmaster]

1. Use the @reward decorator to create spans of intermediate rewards for different actions. See examples/calc_x and tests/test_trace.py for example.
2. Copy agentlightning/verl out (yes you hear me right), and customize the logic here to whatever you want.

agent-lightning/agentlightning/verl/daemon.py

Line 418 in 4d98e85

reward_list.append(sample_info["reward"])

We are planning to support a config to customize this, but the credit assignment can be a complex logic in most cases, while verl only supports a decorative config. I suspect the final design might only relief users from the "copy" step.