[RLlib] Add coin game env. Matrix social dilemma env. With tests and examples.

[Manuscrit]

@sven1977

Why are these changes needed?

Add coin game env. Matrix social dilemma env. With tests and examples.

Related issue number

Checks

• I've run scripts/format.sh to lint the changes in this PR. => Some stuff are much less readable now ;)
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• [X](only for the new tests in this PR) I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[sven1977]

Thank you for this massive contribution! This looks really great!
Just a few changes required:

• Could you run the LINTer on your repo (some lines are too long)?
• • $ cd ray
• • $ ci/travis/format.sh
• Could you move the test cases into a new examples/env/tests dir?
• Could you rename ipd_env.py into iterated_prisoners_dilemma_env.py?
• Could you move the reward-uncertainty class in examples/utils/wrappers.py into rllib/env/wrappers/.?
• Could you rename examples/vectorized_coin_game.py into examples/coin_game.py?
• Could you rename test_coin_game.py into test_non_vectorized_coin_game.py?

[Manuscrit]

About these 2 changes:

• Could you rename examples/vectorized_coin_game.py into examples/coin_game.py?
• Could you rename test_coin_game.py into test_non_vectorized_coin_game.py?

Currently the existing files are:

• coin_game_env.py
• vectorized_coin_game_env.py
• test_coin_game_env.py
• test_vectorized_coin_game_env.py

The two changes mentioned should produce the following:

• coin_game_env.py
• coin_game_env.py
• test_non_vectorized_coin_game_env.py
• test_vectorized_coin_game_env.py

Is that what you had in mind? Then do you mean to merge vectorized_coin_game_env.py and coin_game_env.py in a single file?
Like that:

• coin_game_env.py
• test_non_vectorized_coin_game_env.py
• test_vectorized_coin_game_env.py

If not, then the following seems like a better refactoring to me:

• coin_game_non_vectorized_env.py
• coin_game_vectorized_env.py
• test_coin_game_non_vectorized_env.py
• test_coin_game_vectorized_env.py

What do you think?

[sven1977]

Sounds great, let's rename these according to your suggestion.
Thanks!

[Manuscrit]

I committed the changes @sven1977

[sven1977]

Thanks @Manuscrit for this great contribution!
One more thing before we can merge: Could you check the LINT output again? There seems to be 3 list comprehensions that need to be fixed.

[sven1977]

@Manuscrit no worries, I already fixed the LINT. Just waiting for all tests to pass now. ...