[RLlib] Add support for Int-Box action spaces. #15012

sven1977 · 2021-03-30T09:49:53Z

Add support for Int-Box action spaces.

+test case (test_supported_spaces.py)
Box(dtype=int32|64) action spaces can have arbitrary shapes, but must have uniform low/high values.
RLlib will use the existing MultiCategorical to performa sampling, logp, entropy, etc.. calculations.

Why are these changes needed?

I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

rllib/models/torch/torch_action_dist.py

wip.

38034a6

sven1977 requested a review from michaelzhiluo March 30, 2021 09:50

sven1977 assigned michaelzhiluo Mar 30, 2021

wip.

6c773bb

michaelzhiluo reviewed Mar 30, 2021

View reviewed changes

rllib/models/torch/torch_action_dist.py Show resolved Hide resolved

fixes and LINT.

0f8603e

sven1977 added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Mar 30, 2021

sven1977 added 2 commits March 31, 2021 10:04

wip.

2235224

fix

7cdfcbd

michaelzhiluo approved these changes Apr 11, 2021

View reviewed changes

sven1977 merged commit b267f1f into ray-project:master Apr 11, 2021

sven1977 deleted the support_int_action_boxes branch April 11, 2021 19:03