Migrating from ML-Agents toolkit v0.5 to v0.6
Brains are now Scriptable Objects instead of MonoBehaviors.
You can no longer modify the type of a Brain. If you want to switch between
LearningBrainfor multiple agents, you will need to assign a new Brain to each agent separately. Note: You can pass the same Brain to multiple agents in a scene by leveraging Unity's prefab system or look for all the agents in a scene using the search bar of the
Hierarchywindow with the word
We replaced the Internal and External Brain with Learning Brain. When you need to train a model, you need to drag it into the
Broadcast Hubinside the
Academyand check the
We removed the
Broadcastcheckbox of the Brain, to use the broadcast functionality, you need to drag the Brain into the
When training multiple Brains at the same time, each model is now stored into a separate model file rather than in the same file under different graph scopes.
The Learning Brain graph scope, placeholder names, output names and custom placeholders can no longer be modified.
Steps to Migrate
- To update a scene from v0.5 to v0.6, you must:
- Remove the
BrainGameObjects in the scene. (Delete all of the Brain GameObjects under Academy in the scene.)
- Create new
BrainScriptable Objects using
Assets -> Create -> ML-Agentsfor each type of the Brain you plan to use, and put the created files under a folder called Brains within your project.
- Edit their
Brain Parametersto be the same as the parameters used in the
- Agents have a
Brainfield in the Inspector, you need to drag the appropriate Brain ScriptableObject in it.
- The Academy has a
Broadcast Hubfield in the inspector, which is list of brains used in the scene. To train or control your Brain from the
mlagents-learnPython script, you need to drag the relevant
LearningBrainScriptableObjects used in your scene into entries into this list.
- Remove the
Migrating from ML-Agents toolkit v0.4 to v0.5
- The Unity project
unity-environmenthas been renamed
pythonfolder has been renamed to
ml-agents. It now contains two packages,
mlagents.envcan be used to interact directly with a Unity environment, while
mlagents.trainerscontains the classes for training agents.
- The supported Unity version has changed from
2017.1 or laterto
2017.4 or later. 2017.4 is an LTS (Long Term Support) version that helps us maintain good quality and support. Earlier versions of Unity might still work, but you may encounter an error listed here.
- Discrete Actions now use branches. You can now specify concurrent discrete actions. You will need to update the Brain Parameters in the Brain Inspector in all your environments that use discrete actions. Refer to the discrete action documentation for more information.
In order to run a training session, you can now use the command
python3 learn.pyafter installing the
mlagentspackages. This change is documented here. For example, if we previously ran
python3 learn.py 3DBall --train
pythonsubdirectory (which is changed to
ml-agentssubdirectory in v0.5), we now run
mlagents-learn config/trainer_config.yaml --env=3DBall --train
from the root directory where we installed the ML-Agents Toolkit.
It is now required to specify the path to the yaml trainer configuration file when running
mlagents-learn. For an example trainer configuration file, see trainer_config.yaml. An example of passing a trainer configuration to
mlagents-learnis shown above.
The environment name is now passed through the
Curriculum learning has been changed. Refer to the curriculum learning documentation for detailed information. In summary:
- Curriculum files for the same environment must now be placed into a folder. Each curriculum file should be named after the Brain whose curriculum it specifies.
min_lesson_lengthnow specifies the minimum number of episodes in a lesson and affects reward thresholding.
- It is no longer necessary to specify the
Max Stepsof the Academy to use curriculum learning.
Migrating from ML-Agents toolkit v0.3 to v0.4
using MLAgents;needs to be added in all of the C# scripts that use ML-Agents.
- We've changed some of the Python packages dependencies in requirement.txt
file. Make sure to run
pip3 install -e .within your
ml-agents/pythonfolder to update your Python packages.
Migrating from ML-Agents toolkit v0.2 to v0.3
There are a large number of new features and improvements in the ML-Agents toolkit v0.3 which change both the training process and Unity API in ways which will cause incompatibilities with environments made using older versions. This page is designed to highlight those changes for users familiar with v0.1 or v0.2 in order to ensure a smooth transition.
- The ML-Agents toolkit is no longer compatible with Python 2.
- The training script
PPO.ipynbPython notebook have been replaced with a single
learn.pyscript as the launching point for training with ML-Agents. For more information on using
learn.py, see here.
- Hyperparameters for training Brains are now stored in the
trainer_config.yamlfile. For more information on using this file, see here.
- Modifications to an Agent's rewards must now be done using either
- Setting an Agent to done now requires the use of the
CollectStates()has been replaced by
CollectObservations(), which now no longer returns a list of floats.
- To collect observations, call
CollectObservations(). Note that you can call
AddVectorObs()with floats, integers, lists and arrays of floats, Vector3 and Quaternions.
AgentStep()has been replaced by
WaitTime()has been removed.
Frame Skipfield of the Academy is replaced by the Agent's
Decision Frequencyfield, enabling the Agent to make decisions at different frequencies.
- The names of the inputs in the Internal Brain have been changed. You must
visual_observation. In addition, you must remove the
In order to more closely align with the terminology used in the Reinforcement Learning field, and to be more descriptive, we have changed the names of some of the concepts used in ML-Agents. The changes are highlighted in the table below.
|Old - v0.2 and earlier||New - v0.3 and later|