[Cleanup] [Policy] Use vLLM SamplingParams instead of custom class

[joecummings]

Context

We previously had a custom class called SamplingConfig that would selectively map variables to vLLM's SamplingParams. While this was nice for conversion to/fro dictionary structs, it was an additional redirect for the user to understand what knobs were actually available for sampling.

Changes

• Deleted SamplingConfig, instead relying directly on SamplingParams.
• Added a debug log of the resolved SamplingParams
• Updated naming in all configs, tests, etc
• Added a link to the vLLM sampling params in every config
• Deleted guided decoding work b/c that parameter was actually deprecated and unused outside of Judges. I think this will be formally figured out in the Judge work.

Testing

• WandB Link for Qwen runs showing that it still works the same way: https://wandb.ai/jcummings/grpo-training/runs/8jzcn46h?nw=nwuserjcummings
• Passes the updated unit tests

[codecov-commenter]

Codecov Report

❌ Patch coverage is 4.34783% with 22 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@41215d3). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
tests/unit_tests/test_policy_config.py	0.00%	13 Missing ⚠️
src/forge/actors/policy.py	10.00%	9 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #377   +/-   ##
=======================================
  Coverage        ?   64.36%           
=======================================
  Files           ?       79           
  Lines           ?     7728           
  Branches        ?        0           
=======================================
  Hits            ?     4974           
  Misses          ?     2754           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Jack-Khuu]

Oh this is a fun one, taking a look

Deleted guided decoding work b/c that parameter was actually deprecated and unused outside of Judges. I think this will be formally figured out in the Judge work.

The field is being renamed, but the underlying impl is mostly the same (i think they called it out ~2 weeks ago)

[Jack-Khuu]

This passes the gut test

[Jack-Khuu]

Suggested change

	if (prompt := cfg.get("prompt")) is None:
	gd = cfg.policy.get("sampling_config", {}).get("guided_decoding", False)
	prompt = "What is 3+5?" if gd else "Tell me a joke"
	prompt = "Tell me a joke"
	prompt := cfg.get("prompt", "Tell me a joke")

[joecummings]

Ugh can't apply suggestion on deleted lines :/

[joecummings]

This passes the gut test

Maslow's hierarchy of tests goes unit tests, integration tests, gut tests

[Jack-Khuu]

Note for whoever in the future needs to use rich fields of vllm/v1 in yaml:

from_optional doesn't play well with rich fields (nesting)