[RLlib] Metrics do-over 06: Remove render_images/with_render_images from MAEps and -runner and deprecate env_render config option.

[sven1977]

Metrics do-over 06:

• Remove render_images/with_render_images from MAEps and MAEnvRunner
• deprecate render_env algo config option (in favor of new WandB and MetricsLogger based examples).

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[simonsays1980]

LGTM. A single question in regard to simplifying rendering for users.

[simonsays1980]

I like that this goes away from the MAE. On the opposite: the user has now to take care of rendering in a custom Callback, doesn't she? Could we create maybe a callback that is appended as soon as a user wants to render?

[sven1977]

Valid point. My take on this is that RLlib with its distributed nature is too complicated to make this a simple config flag (like we tried this on the old stack). With the custom callback example, the user has all the possibilities now to write their own filtering logic: Do I only want eval episodes? Do I want all EnvRunners to log videos or just index=2?
Do I want all episodes or just the best performing ones? Etc..

[sven1977]

Also, I wanted to go away from the notion that rendering always means having a live video play out in front of you. Videos are huge things, they should be logged safely to something like WandB and not written en masse to local disk or displayed on a screen and then disappear again.