[RLlib][docs] Cherry-pick-learner-docs-263

[avnishn]

Signed-off-by: Avnish avnishnarayan@gmail.com

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[zhe-thoughts]

Clean docs change

[kouroshHakha]

🚀

[avnishn]

something weird going on with gym/mujoco. Is this expected? do we need to make a fix on the release branch?

[ArturNiederfahrenhorst]

@avnishn Please talk to @can-anyscale . There was an issue on master when we merged Sven's Gym PR. CI would be green and release tests would be red. We applied two forward fixing PRs I believe. @can-anyscale has the most context here def

[avnishn]

@kouroshHakha can you merge this as is? The tests are failing because of a gym issue that this pr did not cause.

[can-anyscale]

Is the release branch also failing in the same way or it only fails in this PR?

[kouroshHakha]

@rickyyx should do the pick. Are we gonna pick those changes to make this release branch green? @can-anyscale ??

[can-anyscale]

Ok, looking at CI jobs, they are pulling from the base master which is broken, which is why all of these tests are failing. This is an CI infrastructure bug/issue.

I think this is fine to merge.