[RLlib] R2D2 Implementation.

[sven1977]

An R2D2 implementation, based on DQN (paper: https://openreview.net/pdf?id=r1lyTjAqYX).

• Tf and Torch policies.

• Runs and passes the "compilation" tests in all frameworks.

• Learns CartPole in both tf and torch.

• Rudimentary documentation.

• Supports: Burn-in.

• TODO (follow up PRs):

• • Support for non-zero init states (init states will be updated after each loss call inside the buffer).
• • Run as APEX (as described in the paper).
• • More benchmarking.
• • Test R2D2 with prio. replay buffer (expected to work, but not tested properly yet).

Why are these changes needed?

Related issue number

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[sven1977]

@michaelzhiluo this is ready for review again ^^