[RLlib] Cleanup all env-checking functionality from RLlib (except MultiAgentEnv).

[sven1977]

Cleanup all env-checking functionality from RLlib.

• Except for checking MultiAgentEnv implementations by users (to be debated whether we need these checks or not).
• We are moving toward new API stack: No BaseEnv, VectorEnv, RemoteEnv, etc.. Only pure gym.Envs -> RLlib should not be concerned with checking these as gym comes with its own pre-check functionalities.
• We are leaving the MultiAgentEnv check in place and activate it inside MultiAgentEnvRunner for the time being. However, we will also simplify this API in the near future (e.g. get rid of action_space_sample, observation_space_sample, and other methods that seem superfluous or only there, b/c RLlib's old BaseEnv required these).

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[simonsays1980]

LGTM. Might need in some cases the disable_env_checker in the env_config - also good for examples to show users how to do it (even though obvious for pros :)).

[simonsays1980]

I wonder, if we need instead to pass in disable_env_checker to the env_config such that gym.make does not check (e.g. in MultiAGentEnv?

[sven1977]

Agreed to have some of these in our examples. Still, we should probably step back and watch from the sidelines as this is a purely gymnasium issue. Similar to us also NOT checking, whether users write a functioning torch.nn.Module or configure their custom torch.optim.Adam correctly.