Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
043d6f7
spell checker on in docs
Aug 29, 2018
a2d9364
spell check on ml-agents folder
Aug 29, 2018
36ff3cb
updated capitlizations
Aug 29, 2018
fe32c27
capitalization fix for brain
Aug 29, 2018
62dbe93
rigidbody capitalization fix
Aug 29, 2018
301515d
academy capitalization fixes
Aug 29, 2018
bf47b49
agents capitalization
Aug 29, 2018
f4fff1e
capitlization for learning env
Aug 29, 2018
0f3602a
more agents capitlization
Aug 29, 2018
875f503
updated s3 links
Aug 29, 2018
bb57a1e
Fixes to readme on ml-agents interface and training
Aug 29, 2018
0c995b9
minor capitalization typo
Aug 29, 2018
f2f49cd
added toolkit
Aug 29, 2018
3f663eb
added ml-agents SDK readme
Aug 29, 2018
77e8ebd
typo
Aug 29, 2018
9ce0e8e
updated installation with the correct directory
Aug 29, 2018
5bab1f7
python / pip to python3 / pip3
Aug 29, 2018
169f757
Merge branch 'release-v0.5' into develop-doc-check-0.5
shihzy Aug 29, 2018
8b75882
remove agents capitalization
Aug 29, 2018
98895a1
Merge branch 'develop-doc-check-0.5' of https://github.com/Unity-Tech…
Aug 29, 2018
4418812
test
Aug 29, 2018
c3fc9b5
revert additional agents capitalization
Aug 29, 2018
03dd749
conflict fix for agents capitalization
shihzy Aug 29, 2018
37b397c
fix for pip3 in windows
Aug 29, 2018
a787697
typo
shihzy Aug 29, 2018
22f3507
rigidbody typo
Aug 29, 2018
4b4e2df
capitalization fix for agent
Aug 29, 2018
ada9870
Rename Readme.md to README.md
shihzy Aug 29, 2018
89ac27c
added Python package
shihzy Aug 29, 2018
fa5622d
Helping clean documentation.
Aug 29, 2018
4b9a7b9
Merge branch 'develop-doc-check-0.5' of github.com:Unity-Technologies…
Aug 29, 2018
275783d
updated install guide to include link to examples
shihzy Aug 30, 2018
d92b9bd
updated installation to include a check
shihzy Aug 30, 2018
40abcb0
updated basic guide to include more details
shihzy Aug 30, 2018
4e803f8
fixed reference to trainer-config-file in example
shihzy Aug 30, 2018
1c7bf12
typo
shihzy Aug 30, 2018
5e36f52
fixed consistency
shihzy Aug 30, 2018
29797e4
Merge branch 'release-v0.5' into develop-doc-check-0.5
shihzy Aug 30, 2018
ea225e0
Merge branch 'release-v0.5' into develop-doc-check-0.5
Aug 30, 2018
55b0514
updated installation
shihzy Aug 30, 2018
c78fe8e
updated basic guide with new python package change
shihzy Aug 30, 2018
30c38ad
Moving mlagents package README.md.
Aug 30, 2018
1cde2a1
Woops, wrong place.
Aug 30, 2018
a24d603
Updating mlagents API readme.
Aug 30, 2018
52d9596
Updated the new Python API with the new changes
shihzy Aug 30, 2018
030e586
update to match readme in package. removed instal
shihzy Aug 30, 2018
e5ee369
update to match python-api.md
shihzy Aug 30, 2018
26d5c1d
Just fixing line lengths.
Aug 30, 2018
2fa1cea
More line length fixes.
Aug 31, 2018
f36b3e8
Missed a line.
Aug 31, 2018
84c9024
Merge branch 'release-v0.5' into develop-doc-check-0.5
Sep 4, 2018
c1e3002
Linting errors.
Sep 4, 2018
0781d88
round of agent capitalizations
Sep 5, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,8 @@ members of the project's leadership.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct/
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 1.4, available at
https://www.contributor-covenant.org/version/1/4/code-of-conduct/

[homepage]: https://www.contributor-covenant.org
60 changes: 31 additions & 29 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,56 @@
# Contribution Guidelines

Thank you for your interest in contributing to the ML-Agents toolkit! We are incredibly
excited to see how members of our community will use and extend the ML-Agents toolkit.
To facilitate your contributions, we've outlined a brief set of guidelines
to ensure that your extensions can be easily integrated.
Thank you for your interest in contributing to the ML-Agents toolkit! We are
incredibly excited to see how members of our community will use and extend the
ML-Agents toolkit. To facilitate your contributions, we've outlined a brief set
of guidelines to ensure that your extensions can be easily integrated.

### Communication
## Communication

First, please read through our [code of conduct](CODE_OF_CONDUCT.md),
as we expect all our contributors to follow it.
First, please read through our [code of conduct](CODE_OF_CONDUCT.md), as we
expect all our contributors to follow it.

Second, before starting on a project that you intend to contribute
to the ML-Agents toolkit (whether environments or modifications to the codebase),
we **strongly** recommend posting on our
[Issues page](https://github.com/Unity-Technologies/ml-agents/issues) and
briefly outlining the changes you plan to make. This will enable us to provide
some context that may be helpful for you. This could range from advice and
feedback on how to optimally perform your changes or reasons for not doing it.
Second, before starting on a project that you intend to contribute to the
ML-Agents toolkit (whether environments or modifications to the codebase), we
**strongly** recommend posting on our
[Issues page](https://github.com/Unity-Technologies/ml-agents/issues)
and briefly outlining the changes you plan to make. This will enable us to
provide some context that may be helpful for you. This could range from advice
and feedback on how to optimally perform your changes or reasons for not doing
it.

Lastly, if you're looking for input on what to contribute, feel free to
reach out to us directly at ml-agents@unity3d.com and/or browse the GitHub
issues with the `contributions welcome` label.

### Git Branches
## Git Branches

Starting with v0.3, we adopted the
Starting with v0.3, we adopted the
[Gitflow Workflow](http://nvie.com/posts/a-successful-git-branching-model/).
Consequently, the `master` branch corresponds to the latest release of
Consequently, the `master` branch corresponds to the latest release of
the project, while the `develop` branch corresponds to the most recent, stable,
version of the project.

Thus, when adding to the project, **please branch off `develop`**
and make sure that your Pull Request (PR) contains the following:

* Detailed description of the changes performed
* Corresponding changes to documentation, unit tests and sample environments
(if applicable)
* Corresponding changes to documentation, unit tests and sample environments (if
applicable)
* Summary of the tests performed to validate your changes
* Issue numbers that the PR resolves (if any)

### Environments
## Environments

We are also actively open to adding community contributed environments as
examples, as long as they are small, simple, demonstrate a unique feature of
the platform, and provide a unique non-trivial challenge to modern
We are also actively open to adding community contributed environments as
examples, as long as they are small, simple, demonstrate a unique feature of
the platform, and provide a unique non-trivial challenge to modern
machine learning algorithms. Feel free to submit these environments with a
PR explaining the nature of the environment and task.
PR explaining the nature of the environment and task.

### Style Guide
## Style Guide

When performing changes to the codebase, ensure that you follow the style
guide of the file you're modifying. For Python, we follow
[PEP 8](https://www.python.org/dev/peps/pep-0008/). For C#, we will soon be
adding a formal style guide for our repository.
When performing changes to the codebase, ensure that you follow the style guide
of the file you're modifying. For Python, we follow
[PEP 8](https://www.python.org/dev/peps/pep-0008/).
For C#, we will soon be adding a formal style guide for our repository.
1 change: 1 addition & 0 deletions MLAgentsSDK/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# ML-Agents SDK
94 changes: 50 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,75 +4,81 @@

# Unity ML-Agents Toolkit (Beta)

**The Unity Machine Learning Agents Toolkit** (ML-Agents) is an open-source Unity plugin
that enables games and simulations to serve as environments for training
intelligent agents. Agents can be trained using reinforcement learning,
imitation learning, neuroevolution, or other machine learning methods through
a simple-to-use Python API. We also provide implementations (based on
TensorFlow) of state-of-the-art algorithms to enable game developers
and hobbyists to easily train intelligent agents for 2D, 3D and VR/AR games.
These trained agents can be used for multiple purposes, including
controlling NPC behavior (in a variety of settings such as multi-agent and
adversarial), automated testing of game builds and evaluating different game
design decisions pre-release. The ML-Agents toolkit is mutually beneficial for both game
developers and AI researchers as it provides a central platform where advances
in AI can be evaluated on Unity’s rich environments and then made accessible
to the wider research and game developer communities.
**The Unity Machine Learning Agents Toolkit** (ML-Agents) is an open-source
Unity plugin that enables games and simulations to serve as environments for
training intelligent agents. Agents can be trained using reinforcement learning,
imitation learning, neuroevolution, or other machine learning methods through a
simple-to-use Python API. We also provide implementations (based on TensorFlow)
of state-of-the-art algorithms to enable game developers and hobbyists to easily
train intelligent agents for 2D, 3D and VR/AR games. These trained agents can be
used for multiple purposes, including controlling NPC behavior (in a variety of
settings such as multi-agent and adversarial), automated testing of game builds
and evaluating different game design decisions pre-release. The ML-Agents
toolkit is mutually beneficial for both game developers and AI researchers as it
provides a central platform where advances in AI can be evaluated on Unity’s
rich environments and then made accessible to the wider research and game
developer communities.

## Features

* Unity environment control from Python
* 10+ sample Unity environments
* Support for multiple environment configurations and training scenarios
* Train memory-enhanced Agents using deep reinforcement learning
* Train memory-enhanced agents using deep reinforcement learning
* Easily definable Curriculum Learning scenarios
* Broadcasting of Agent behavior for supervised learning
* Broadcasting of agent behavior for supervised learning
* Built-in support for Imitation Learning
* Flexible Agent control with On Demand Decision Making
* Flexible agent control with On Demand Decision Making
* Visualizing network outputs within the environment
* Simplified set-up with Docker

## Documentation

* For more information, in addition to installation and usage
instructions, see our [documentation home](docs/Readme.md).
* If you have
used a version of the ML-Agents toolkit prior to v0.4, we strongly recommend
our [guide on migrating from earlier versions](docs/Migrating.md).
* For more information, in addition to installation and usage instructions, see
our [documentation home](docs/Readme.md).
* If you have used a version of the ML-Agents toolkit prior to v0.4, we strongly
recommend our [guide on migrating from earlier versions](docs/Migrating.md).

## References

We have published a series of blog posts that are relevant for ML-Agents:
- Overviewing reinforcement learning concepts
([multi-armed bandit](https://blogs.unity3d.com/2017/06/26/unity-ai-themed-blog-entries/)
and [Q-learning](https://blogs.unity3d.com/2017/08/22/unity-ai-reinforcement-learning-with-q-learning/))
- [Using Machine Learning Agents in a real game: a beginner’s guide](https://blogs.unity3d.com/2017/12/11/using-machine-learning-agents-in-a-real-game-a-beginners-guide/)
- [Post](https://blogs.unity3d.com/2018/02/28/introducing-the-winners-of-the-first-ml-agents-challenge/) announcing the winners of our
[first ML-Agents Challenge](https://connect.unity.com/challenges/ml-agents-1)
- [Post](https://blogs.unity3d.com/2018/01/23/designing-safer-cities-through-simulations/)
overviewing how Unity can be leveraged as a simulator to design safer cities.

* Overviewing reinforcement learning concepts
([multi-armed bandit](https://blogs.unity3d.com/2017/06/26/unity-ai-themed-blog-entries/)
and
[Q-learning](https://blogs.unity3d.com/2017/08/22/unity-ai-reinforcement-learning-with-q-learning/))
* [Using Machine Learning Agents in a real game: a beginner’s guide](https://blogs.unity3d.com/2017/12/11/using-machine-learning-agents-in-a-real-game-a-beginners-guide/)
* [Post](https://blogs.unity3d.com/2018/02/28/introducing-the-winners-of-the-first-ml-agents-challenge/)
announcing the winners of our
[first ML-Agents Challenge](https://connect.unity.com/challenges/ml-agents-1)
* [Post](https://blogs.unity3d.com/2018/01/23/designing-safer-cities-through-simulations/)
overviewing how Unity can be leveraged as a simulator to design safer cities.

In addition to our own documentation, here are some additional, relevant articles:
- [Unity AI - Unity 3D Artificial Intelligence](https://www.youtube.com/watch?v=bqsfkGbBU6k)
- [A Game Developer Learns Machine Learning](https://mikecann.co.uk/machine-learning/a-game-developer-learns-machine-learning-intent/)
- [Explore Unity Technologies ML-Agents Exclusively on Intel Architecture](https://software.intel.com/en-us/articles/explore-unity-technologies-ml-agents-exclusively-on-intel-architecture)

* [Unity AI - Unity 3D Artificial Intelligence](https://www.youtube.com/watch?v=bqsfkGbBU6k)
* [A Game Developer Learns Machine Learning](https://mikecann.co.uk/machine-learning/a-game-developer-learns-machine-learning-intent/)
* [Explore Unity Technologies ML-Agents Exclusively on Intel Architecture](https://software.intel.com/en-us/articles/explore-unity-technologies-ml-agents-exclusively-on-intel-architecture)

## Community and Feedback

The ML-Agents toolkit is an open-source project and we encourage and welcome contributions.
If you wish to contribute, be sure to review our
[contribution guidelines](CONTRIBUTING.md) and
The ML-Agents toolkit is an open-source project and we encourage and welcome
contributions. If you wish to contribute, be sure to review our
[contribution guidelines](CONTRIBUTING.md) and
[code of conduct](CODE_OF_CONDUCT.md).

You can connect with us and the broader community
through Unity Connect and GitHub:

* Join our
[Unity Machine Learning Channel](https://connect.unity.com/messages/c/035fba4f88400000)
to connect with others using the ML-Agents toolkit and Unity developers enthusiastic
about machine learning. We use that channel to surface updates
regarding the ML-Agents toolkit (and, more broadly, machine learning in games).
* If you run into any problems using the ML-Agents toolkit,
[submit an issue](https://github.com/Unity-Technologies/ml-agents/issues) and
make sure to include as much detail as possible.
[Unity Machine Learning Channel](https://connect.unity.com/messages/c/035fba4f88400000)
to connect with others using the ML-Agents toolkit and Unity developers
enthusiastic about machine learning. We use that channel to surface updates
regarding the ML-Agents toolkit (and, more broadly, machine learning in
games).
* If you run into any problems using the ML-Agents toolkit,
[submit an issue](https://github.com/Unity-Technologies/ml-agents/issues) and
make sure to include as much detail as possible.

For any other questions or feedback, connect directly with the ML-Agents
team at ml-agents@unity3d.com.
Expand All @@ -86,7 +92,7 @@ of the documentation to one language (Chinese), but we hope to continue
translating more pages and to other languages. Consequently,
we welcome any enhancements and improvements from the community.

- [Chinese](docs/localized/zh-CN/)
* [Chinese](docs/localized/zh-CN/)

## License

Expand Down
2 changes: 1 addition & 1 deletion docs/API-Reference.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# API Reference

Our developer-facing C# classes (Academy, Agent, Decision and Monitor) have been
documented to be compatabile with
documented to be compatible with
[Doxygen](http://www.stack.nl/~dimitri/doxygen/) for auto-generating HTML
documentation.

Expand Down
2 changes: 1 addition & 1 deletion docs/Background-TensorFlow.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ to TensorFlow-related tools that we leverage within the ML-Agents toolkit.
performing computations using data flow graphs, the underlying representation of
deep learning models. It facilitates training and inference on CPUs and GPUs in
a desktop, server, or mobile device. Within the ML-Agents toolkit, when you
train the behavior of an Agent, the output is a TensorFlow model (.bytes) file
train the behavior of an agent, the output is a TensorFlow model (.bytes) file
that you can then embed within an Internal Brain. Unless you implement a new
algorithm, the use of TensorFlow is mostly abstracted away and behind the
scenes.
Expand Down
32 changes: 19 additions & 13 deletions docs/Basic-Guide.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Basic Guide

This guide will show you how to use a pretrained model in an example Unity
This guide will show you how to use a pre-trained model in an example Unity
environment, and show you how to train the model yourself.

If you are not familiar with the [Unity Engine](https://unity3d.com/unity), we
Expand All @@ -13,7 +13,7 @@ the basic concepts of Unity.
In order to use the ML-Agents toolkit within Unity, you need to change some
Unity settings first. Also [TensorFlowSharp
plugin](https://s3.amazonaws.com/unity-ml-agents/0.4/TFSharpPlugin.unitypackage)
is needed for you to use pretrained model within Unity, which is based on the
is needed for you to use pre-trained model within Unity, which is based on the
[TensorFlowSharp repo](https://github.com/migueldeicaza/TensorFlowSharp).

1. Launch Unity
Expand Down Expand Up @@ -70,14 +70,14 @@ if you want to [use an executable](Learning-Environment-Executable.md) or to
`None` if you want to interact with the current scene in the Unity Editor.

More information and documentation is provided in the
[Python API](../ml-agents/README.md) page.
[Python API](Python-API.md) page.

## Training the Brain with Reinforcement Learning

### Setting the Brain to External

Since we are going to build this environment to conduct training, we need to set
the brain used by the agents to **External**. This allows the agents to
the Brain used by the Agents to **External**. This allows the Agents to
communicate with the external training process when making their decisions.

1. In the **Scene** window, click the triangle icon next to the Ball3DAcademy
Expand All @@ -90,17 +90,23 @@ communicate with the external training process when making their decisions.
### Training the environment

1. Open a command or terminal window.
2. Nagivate to the folder where you installed the ML-Agents toolkit.
2. Navigate to the folder where you cloned the ML-Agents toolkit repository.
**Note**: If you followed the default [installation](Installation.md), then
you should be able to run `mlagents-learn` from any directory.
3. Run `mlagents-learn <trainer-config-path> --run-id=<run-identifier> --train`
Where:
where:
- `<trainer-config-path>` is the relative or absolute filepath of the
trainer configuration. The defaults used by environments in the ML-Agents
SDK can be found in `config/trainer_config.yaml`.
trainer configuration. The defaults used by example environments included
in `MLAgentsSDK` can be found in `config/trainer_config.yaml`.
- `<run-identifier>` is a string used to separate the results of different
training runs
- And the `--train` tells `mlagents-learn` to run a training session (rather
- `--train` tells `mlagents-learn` to run a training session (rather
than inference)
4. When the message _"Start training by pressing the Play button in the Unity
4. If you cloned the ML-Agents repo, then you can simply run
```sh
mlagents-learn config/trainer_config.yaml --run-id=firstRun --train
```
5. When the message _"Start training by pressing the Play button in the Unity
Editor"_ is displayed on the screen, you can press the :arrow_forward: button
in Unity to start training in the Editor.

Expand Down Expand Up @@ -143,6 +149,7 @@ INFO:mlagents.learn:{'--curriculum': 'None',
'--train': True,
'--worker-id': '0',
'<trainer-config-path>': 'config/trainer_config.yaml'}
INFO:mlagents.envs:Start training by pressing the Play button in the Unity Editor.
```

**Note**: If you're using Anaconda, don't forget to activate the ml-agents
Expand All @@ -152,7 +159,6 @@ If `mlagents-learn` runs correctly and starts training, you should see something
like this:

```console
INFO:mlagents.envs:Start training by pressing the Play button in the Unity Editor.
INFO:mlagents.envs:
'Ball3DAcademy' started successfully!
Unity Academy name: Ball3DAcademy
Expand Down Expand Up @@ -208,7 +214,7 @@ You can press Ctrl+C to stop the training, and your trained model will be at
`models/<run-identifier>/editor_<academy_name>_<run-identifier>.bytes` where
`<academy_name>` is the name of the Academy GameObject in the current scene.
This file corresponds to your model's latest checkpoint. You can now embed this
trained model into your internal brain by following the steps below, which is
trained model into your Internal Brain by following the steps below, which is
similar to the steps described
[above](#play-an-example-environment-using-pretrained-model).

Expand All @@ -229,7 +235,7 @@ similar to the steps described
page.
- For a more detailed walk-through of our 3D Balance Ball environment, check out
the [Getting Started](Getting-Started-with-Balance-Ball.md) page.
- For a "Hello World" introduction to creating your own learning environment,
- For a "Hello World" introduction to creating your own Learning Environment,
check out the [Making a New Learning
Environment](Learning-Environment-Create-New.md) page.
- For a series of Youtube video tutorials, checkout the
Expand Down
Loading