[RLlib] Fixed 'rollout_fragment_length' in pong-example by setting it to 'auto'.

[simonsays1980]

Why are these changes needed?

Pong example did not run due to a rollout_fragment_length that did not fit the train_batch_size. By setting the rollout_fragment_length to auto the rollout_fragment_length adapts to the train_batch_size.

Related issue number

Closes #38968

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[ArturNiederfahrenhorst]

Great! Thanks for the fix

[simonsays1980]

@ArturNiederfahrenhorst Looking at the failed tests I think that the test is running on the wrong cluster: it requests a GPU (in the YAML) but there is none.

Shall we add "gpu" to the tags of the Pong test?

[sven1977]

LGTM. Thanks! :)

[ArturNiederfahrenhorst]

@sven1977 I was the one who added the test. The reason we stumbled across this issue was that we don't test this code today - the issue was raised by a user who read the docs. In fact, all of our fine-tuned examples listed under doc/source/rllib/rllib-algorithms.rst are not tested.
Imho, they should be release tests given that RLlib's most used Algorithm is PPO by a good margin and that we refer to these as examples of it in our documentation.

[ArturNiederfahrenhorst]

I agree that the non-gpu learning test is not a good place, but instead, there should be another place where we execute this test. After all, if we can't prove learning on it, it should not be referred to as a fine-tuned example

[ArturNiederfahrenhorst]

I've opened an issue #39639