Skip to content

Release v0.6 docs #1325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 13, 2018
Merged

Release v0.6 docs #1325

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fc821c1
Fix Typo #1323
vincentpierre Oct 12, 2018
0c3721f
First update to the docs
vincentpierre Oct 12, 2018
9a66789
Addressed comments
vincentpierre Oct 12, 2018
d6ad25d
remove references to TF#
vincentpierre Oct 11, 2018
2b4f34c
Replaced the references to TF# with new document.
vincentpierre Oct 12, 2018
51714b0
Edditied the FAQ
vincentpierre Oct 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 5 additions & 14 deletions docs/Background-TensorFlow.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ to TensorFlow-related tools that we leverage within the ML-Agents toolkit.
performing computations using data flow graphs, the underlying representation of
deep learning models. It facilitates training and inference on CPUs and GPUs in
a desktop, server, or mobile device. Within the ML-Agents toolkit, when you
train the behavior of an agent, the output is a TensorFlow model (.bytes) file
train the behavior of an agent, the output is a TensorFlow model (.tf) file
that you can then embed within a Learning Brain. Unless you implement a new
algorithm, the use of TensorFlow is mostly abstracted away and behind the
scenes.
Expand All @@ -36,18 +36,9 @@ documentation, but, in the meantime, if you are unfamiliar with TensorBoard we
recommend this
[tutorial](https://github.com/dandelionmane/tf-dev-summit-tensorboard-tutorial).

## TensorFlowSharp
## Tensorflow Model Inference

One of the drawbacks of TensorFlow is that it does not provide a native C# API.
This means that the Learning Brain is not natively supported since Unity scripts
are written in C#. Consequently, to enable the Learning Brain, we leverage a
third-party library
[TensorFlowSharp](https://github.com/migueldeicaza/TensorFlowSharp) which
provides .NET bindings to TensorFlow. Thus, when a Unity environment that
contains a Learning Brain is built, inference is performed via TensorFlowSharp.
We provide an additional in-depth overview of how to leverage
[TensorFlowSharp within Unity](Using-TensorFlow-Sharp-in-Unity.md)
which will become more
relevant once you install and start training behaviors within the ML-Agents
toolkit. Given the reliance on TensorFlowSharp, the Learning Brain is currently
marked as experimental.
We have are using the [Unity Machine Learning Inference SDK](Inference-Engine.md) to
run the models inside of Unity. In order to use it, you will need to have an
appropriate backend downloaded.
73 changes: 29 additions & 44 deletions docs/Basic-Guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,9 @@ the basic concepts of Unity.
## Setting up the ML-Agents Toolkit within Unity

In order to use the ML-Agents toolkit within Unity, you need to change some
Unity settings first. Also [TensorFlowSharp
plugin](https://s3.amazonaws.com/unity-ml-agents/0.5/TFSharpPlugin.unitypackage)
is needed for you to use pre-trained model within Unity, which is based on the
[TensorFlowSharp repo](https://github.com/migueldeicaza/TensorFlowSharp).
Unity settings first. Youy will also need to have appropriate inference backends
installed in order to run your models inside of Unity. See [here](Inference-Engine.md)
for more information.

1. Launch Unity
2. On the Projects dialog, choose the **Open** option at the top of the window.
Expand All @@ -26,38 +25,23 @@ is needed for you to use pre-trained model within Unity, which is based on the
1. Option the **Other Settings** section.
2. Select **Scripting Runtime Version** to **Experimental (.NET 4.6
Equivalent or .NET 4.x Equivalent)**
3. In **Scripting Defined Symbols**, add the flag `ENABLE_TENSORFLOW`. After
typing in the flag name, press Enter.
6. Go to **File** > **Save Project**

![Project Settings](images/project-settings.png)

[Download](https://s3.amazonaws.com/unity-ml-agents/0.5/TFSharpPlugin.unitypackage)
the TensorFlowSharp plugin. Then import it into Unity by double clicking the
downloaded file. You can check if it was successfully imported by checking the
TensorFlow files in the Project window under **Assets** > **ML-Agents** >
**Plugins** > **Computer**.

**Note**: If you don't see anything under **Assets**, drag the
`UnitySDK/Assets/ML-Agents` folder under **Assets** within Project window.

![Imported TensorFlowsharp](images/imported-tensorflowsharp.png)

## Running a Pre-trained Model

1. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Scenes` folder
and open the `3DBall` scene file.
2. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Prefabs` folder
and select the `Game/Platform` prefab.
3. In the `Ball 3D Agent` Component: Drag the **Ball3DBrain** located into
3. In the `Ball 3D Agent` Component: Drag the **3DBallLearning** located into
`Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property of
the `Ball 3D Agent`.
4. Make sure that all of the Agents in the Scene now have **Ball3DBrain** as `Brain`.
4. Make sure that all of the Agents in the Scene now have **3DBallLearning** as `Brain`.
__Note__ : You can modify multiple game objects in a scene by selecting them all at once using the search bar in the Scene Hierarchy.
5. In the **Project** window, locate the `Assets/ML-Agents/Examples/3DBall/TFModels`
folder.
6. Drag the `3DBall` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels`
folder to the **Model** field of the **Ball3DBrain**.
folder to the **Model** field of the **3DBallLearning**.
7. Click the **Play** button and you will see the platforms balance the balls
using the pretrained model.

Expand All @@ -83,10 +67,11 @@ Since we are going to build this environment to conduct training, we need to add
the Brain to the training session. This allows the Agents linked to that Brain
to communicate with the external training process when making their decisions.

1. Assign the **Ball3DBrain** to the agents you would like to train.
1. Assign the **3DBallLearning** to the agents you would like to train and the **3DBallPlayer** Brain to the agents you want to control manually.
__Note:__ You can only perform training with an `Learning Brain`.
2. Select the **Ball3DAcademy** GameObject and add the **Ball3DBrain**
to the Broadcast Hub and toggle the `Control` checkbox.
2. Select the **Ball3DAcademy** GameObject and make sure the **3DBallLearning** Brain
is in the Broadcast Hub. In order to train, you need to toggle the
`Control` checkbox.

![Set Brain to External](images/mlagents-SetBrainToTrain.png)

Expand Down Expand Up @@ -168,17 +153,17 @@ INFO:mlagents.envs:
'Ball3DAcademy' started successfully!
Unity Academy name: Ball3DAcademy
Number of Brains: 1
Number of External Brains : 1
Number of Training Brains : 1
Reset Parameters :

Unity brain name: Ball3DBrain
Unity brain name: 3DBallLearning
Number of Visual Observations (per agent): 0
Vector Observation space size (per agent): 8
Number of stacked Vector Observation: 1
Vector Action space type: continuous
Vector Action space size (per agent): [2]
Vector Action descriptions: ,
INFO:mlagents.envs:Hyperparameters for the PPO Trainer of brain Ball3DBrain:
INFO:mlagents.envs:Hyperparameters for the PPO Trainer of brain 3DBallLearning:
batch_size: 64
beta: 0.001
buffer_size: 12000
Expand All @@ -200,24 +185,24 @@ INFO:mlagents.envs:Hyperparameters for the PPO Trainer of brain Ball3DBrain:
use_curiosity: False
curiosity_strength: 0.01
curiosity_enc_size: 128
model_path: ./models/first-run-0/Ball3DBrain
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 1000. Mean Reward: 1.242. Std of Reward: 0.746. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 2000. Mean Reward: 1.319. Std of Reward: 0.693. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 3000. Mean Reward: 1.804. Std of Reward: 1.056. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 4000. Mean Reward: 2.151. Std of Reward: 1.432. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 5000. Mean Reward: 3.175. Std of Reward: 2.250. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 6000. Mean Reward: 4.898. Std of Reward: 4.019. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 7000. Mean Reward: 6.716. Std of Reward: 5.125. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 8000. Mean Reward: 12.124. Std of Reward: 11.929. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 9000. Mean Reward: 18.151. Std of Reward: 16.871. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 10000. Mean Reward: 27.284. Std of Reward: 28.667. Training.
model_path: ./models/first-run-0/3DBallLearning
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 1000. Mean Reward: 1.242. Std of Reward: 0.746. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 2000. Mean Reward: 1.319. Std of Reward: 0.693. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 3000. Mean Reward: 1.804. Std of Reward: 1.056. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 4000. Mean Reward: 2.151. Std of Reward: 1.432. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 5000. Mean Reward: 3.175. Std of Reward: 2.250. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 6000. Mean Reward: 4.898. Std of Reward: 4.019. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 7000. Mean Reward: 6.716. Std of Reward: 5.125. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 8000. Mean Reward: 12.124. Std of Reward: 11.929. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 9000. Mean Reward: 18.151. Std of Reward: 16.871. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 10000. Mean Reward: 27.284. Std of Reward: 28.667. Training.
```

### After training

You can press Ctrl+C to stop the training, and your trained model will be at
`models/<run-identifier>/editor_<academy_name>_<run-identifier>.bytes` where
`<academy_name>` is the name of the Academy GameObject in the current scene.
`models/<run-identifier>/<brain_name>.tf` where
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we no longer put editor before the .tf file name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use the brain_name directly

`<brain_name>` is the name of the Brain corresponding to the model.
(**Note:** There is a known bug on Windows that causes the saving of the model to
fail when you early terminate the training, it's recommended to wait until Step
has reached the max_steps parameter you set in trainer_config.yaml.) This file
Expand All @@ -229,9 +214,9 @@ the steps described
1. Move your model file into
`UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/`.
2. Open the Unity Editor, and select the **3DBall** scene as described above.
3. Select the **Ball3DBrain** Learning Brain from the Scene hierarchy.
5. Drag the `<env_name>_<run-identifier>.bytes` file from the Project window of
the Editor to the **Graph Model** placeholder in the **Ball3DBrain**
3. Select the **3DBallLearning** Learning Brain from the Scene hierarchy.
5. Drag the `<brain_name>.tf` file from the Project window of
the Editor to the **Model** placeholder in the **3DBallLearning**
inspector window.
6. Press the :arrow_forward: button at the top of the Editor.

Expand Down
32 changes: 4 additions & 28 deletions docs/FAQ.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,36 +13,12 @@ This is because .NET 3.5 doesn't support method Clear() for StringBuilder, refer
to [Setting Up The ML-Agents Toolkit Within
Unity](Installation.md#setting-up-ml-agent-within-unity) for solution.

## TensorFlowSharp flag not turned on
## Cannot drag Model into Learning Brain

If you have already imported the TensorFlowSharp plugin, but haven't set
ENABLE_TENSORFLOW flag for your scripting define symbols, you will see the
following error message:
You migh not have the appropriate backend required to import the model. Refer to the
[Inference Engine](Inference-Engine.md) for more information on how to import backends
and reimport the asset.

```console
You need to install and enable the TensorFlowSharp plugin in order to use the Learning Brain.
```

This error message occurs because the TensorFlowSharp plugin won't be usage
without the ENABLE_TENSORFLOW flag, refer to [Setting Up The ML-Agents Toolkit
Within Unity](Installation.md#setting-up-ml-agent-within-unity) for solution.

## Instance of CoreBrainInternal couldn't be created

If you try to use ML-Agents in Unity versions 2017.1 - 2017.3, you might
encounter an error that looks like this:

```console
Instance of CoreBrainInternal couldn't be created. The the script
class needs to derive from ScriptableObject.
UnityEngine.ScriptableObject:CreateInstance(String)
```

You can fix the error by removing `CoreBrain` from CoreBrainInternal.cs:16,
clicking on your Brain Gameobject to let the scene recompile all the changed
C# scripts, then adding the `CoreBrain` back. Make sure your brain is in
Internal mode, your TensorFlowSharp plugin is imported and the
ENABLE_TENSORFLOW flag is set. This fix is only valid locally and unstable.

## Environment Permission Error

Expand Down
16 changes: 6 additions & 10 deletions docs/Getting-Started-with-Balance-Ball.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ environment around the Agents.
### Brain

Brains are assets that exist in your project folder. The Ball3DAgents are connected
to a brain, for example : the **Ball3DBrain**.
to a brain, for example : the **3DBallLearning**.
A Brain doesn't store any information about an Agent, it just
routes the Agent's collected observations to the decision making process and
returns the chosen action to the Agent. Thus, all Agents can share the same
Expand Down Expand Up @@ -273,18 +273,14 @@ Once the training process completes, and the training process saves the model
use it with Agents having a **Learning Brain**.
__Note:__ Do not just close the Unity Window once the `Saved Model` message appears.
Either wait for the training process to close the window or press Ctrl+C at the
command-line prompt. If you simply close the window manually, the .bytes file
command-line prompt. If you simply close the window manually, the `.tf` file
containing the trained model is not exported into the ml-agents folder.

### Setting up TensorFlowSharp Support
### Setting up Inference Support

Because TensorFlowSharp support is still experimental, it is disabled by
default. In order to enable it, you must follow these steps. Please note that
the `Learning` Brain mode will only be available once completing these steps.

To set up the TensorFlowSharp Support, follow [Setting up ML-Agents Toolkit
within Unity](Basic-Guide.md#setting-up-ml-agents-within-unity) section. of the
Basic Guide page.
In order to run neural network models inside of Unity, you will need to setup the
Inference Engine with an appropriate backend. See [here](Inference-Engine.md) for more
information.

### Embedding the trained model into Unity

Expand Down
16 changes: 8 additions & 8 deletions docs/Learning-Environment-Design-Learning-Brains.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,18 +43,18 @@ model.
To use a graph model:

1. Select the **Learning Brain** asset in the **Project** window of the Unity Editor.
**Note:** In order to see the **Learning** Brain Type option, you must
[enable TensorFlowSharp](Using-TensorFlow-Sharp-in-Unity.md).
3. Import the `environment_run-id.bytes` file produced by the PPO training
program. (Where `environment_run-id` is the name of the model file, which is
**Note:** In order to use the **Learning** Brain, you have appropriate backend for the
Inference Engine. See [here](Inference-Engine.md).
2. Import the `model_name` file produced by the PPO training
program. (Where `model_name` is the name of the model file, which is
constructed from the name of your Unity environment executable and the run-id
value you assigned when running the training process.)

You can
[import assets into Unity](https://docs.unity3d.com/Manual/ImportingAssets.html)
in various ways. The easiest way is to simply drag the file into the
**Project** window and drop it into an appropriate folder.
4. Once the `environment.bytes` file is imported, drag it from the **Project**
3. Once the `model_name.tf` file is imported, drag it from the **Project**
window to the **Model** field of the Brain component.

If you are using a model produced by the ML-Agents `mlagents-learn` command, use
Expand All @@ -64,10 +64,10 @@ the default values for the other Learning Brain parameters.

The default values of the TensorFlow graph parameters work with the model
produced by the PPO and BC training code in the ML-Agents SDK. To use a default
ML-Agents model, the only parameter that you need to set is the `Graph Model`,
which must be set to the .bytes file containing the trained model itself.
ML-Agents model, the only parameter that you need to set is the `Model`,
which must be set to the `.tf` file containing the trained model itself.

* `Model` : This must be the `bytes` file corresponding to the pre-trained
* `Model` : This must be the `.tf` file corresponding to the pre-trained
TensorFlow graph. (You must first drag this file into your Project window
and then from the Resources folder into the inspector)

Loading