Skip to content
Branch: master
Find file History
sunil19m and vrkhare Removing outdated reward function (#829)
* With many changes to the SimulationApplication the notebook version of the DeepRacer became outdated.
This did not work with new SimApp application

Changes made to the notebook to work
1. The notebook uses the custom docker instead of default sagemaker docker
2. The files are made in consistence with the AwsSilverstoneMarkov Package. Going forward change to these files must be tested in the notebook aswell.
3. S3FullAccess role was required to sagemaker role
4. Fixed the issue with the evaluation worker import issues
5. Metrics now uses the json file instead of the .csv file to show training rewards
6. Added all the instructions to clean up the job
7. More instructions are added on how to use the github README.md

Testing:
Tested it murasuni account, science-team account and burner account. It works.

* Notebook support to new SimulationApplication

Minor review comment fixes
1. /common was symbolic link. Kept it the same and made modification to the ../common (original file)
2. Removed all the commented code in the Dockerfile

Testing:
Tested it murasuni account, science-team account and burner account. It works.

* Upgrading pandas and futures to fix the support for rl-coach

* Removing outdated reward function

1. These reward functions were used during the reInvent-2018 and are no more supported. Hence these reward functions are removed.
2. Adding the 3 reward functions present in the console to this notebook
3. Removing some of the unnecessary comments

Tested:
Tested locally
Latest commit cbb6796 Aug 1, 2019

README.md

DeepRacer notebook using Amazon SageMaker RL and AWS RoboMaker services

This folder contains examples of how to use RL to train an autonomous deepracer. This is a jailbreaker for the AWS DeepRacer. This gives a glimse of architecture used to get the DeepRacer working.

Contents

  • deepracer_rl.ipynb: notebook for training autonomous race car.

  • Dockerfile: Custom docker instead of using SageMaker default docker

  • src/

    • training_worker.py: Main entrypoint for starting distributed training job
    • markov/: Helper files for S3 upload/download
    • presets/default.py: Preset (configuration) for DeepRacer
    • rewards/default.py: Custom reward function
    • environments/deepracer_racetrack_env.py: Gym environment file for DeepRacer
    • actions/model_metadata_10_state.json: JSON file to customize your action space & the speed
    • lib/: redis configuration file and ppo_head.py customized tensorflow file copied to sagemaker container.
  • common/: helper function to build docker files.

How to use the notebook

  1. Login to your AWS account - SageMaker service (SageMaker Link)
  2. On the left tab select Notebook instances
  3. Select Create notebook instance
  4. Fill up the notebook instance name. In the Additional configuration select atleast 25GB. This is because docker gets installed and takes up space.
  5. Create a new IAM role. Give root permission
  6. Select the git repository and clone this repository.
  7. Then click create notebook instance button at the button
  8. This takes like 2 min to create your notebook instance. Then click on the newly created instance and click on the juypter notebook.
  9. You will see all the github files and now run deepracer_rl.ipynb
  10. Run clean robomaker & sagemaker commands in the script only when you are done with training.
You can’t perform that action at this time.