New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Continuous Mountain Car Environment same as Gym implementation #1394

Merged
merged 5 commits into from May 20, 2018

Conversation

Projects
None yet
3 participants
@sshkhr
Contributor

sshkhr commented May 15, 2018

This PR make the contiunous Mountain Car Environment streamlined with the implementation in OpenAI Gym (https://github.com/openai/gym/blob/master/gym/envs/classic_control/continuous_mountain_car.py) . The changes are :

  • Added a goal position
  • The terminal state is reached when car crosses goal position (earlier it was set as when car reaches maximum allowed position)
  • The reward for reaching the terminal state is 100 - cost of action (earlier it was just 100)
  • The next state calculation is more configurable via introducing the power member variable (This was requested by @zoq in #1368 but not addressed)

I have also added my name to list of contributors following #1388

@@ -96,13 +96,17 @@ class ContinuousMountainCar
* @param velocityMax Maximum legal velocity.
*/
ContinuousMountainCar(const double positionMin = -1.2,
const double positionMax = 0.5,
const double positionMax = 0.6,

This comment has been minimized.

@zoq

zoq May 16, 2018

Member

Can you add a comment for each new parameter? Also I think you did some good additions to this file, so feel free to add yourself as another author.

return 100.0;
return -pow(action.action[0], 2)*0.1;
reward = 100.0;
reward -= std::pow(action.action[0], 2) * 0.1;

This comment has been minimized.

@zoq

zoq May 16, 2018

Member

Don't think this makes a huge difference, but I guess it's a good idea to get the same results as the gym env.

sshkhr added some commits May 16, 2018

@rcurtin

No comments from my side; thanks for the contribution! :)

@zoq

This comment has been minimized.

Member

zoq commented May 17, 2018

@mlpack-jenkins test this please

@zoq zoq merged commit 095f784 into mlpack:master May 20, 2018

4 of 5 checks passed

continuous-integration/travis-ci/pr The Travis CI build could not complete due to an error
Details
Memory Checks
Details
Static Code Analysis Checks Build finished.
Details
Style Checks Build finished.
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
@zoq

This comment has been minimized.

Member

zoq commented May 20, 2018

Thanks again for the contribution 👍

@sshkhr sshkhr deleted the sshkhr:rl2 branch May 20, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment