Releases: Unity-Technologies/ml-agents
Releases · Unity-Technologies/ml-agents
ML-Agents Release 1
Package Versions
As part of ML-Agents Release 1, we will be versioning the different packages that make up the release.
NOTE: It is strongly recommended that you use packages from the same release together for the best experience.
Package | Version |
---|---|
com.unity.ml-agents (C#) | v1.0.0 |
ml-agents (Python) | v0.16.0 |
ml-agents-envs (Python) | v0.16.0 |
gym-unity (Python) | v0.16.0 |
Communicator (C#/Python) | v1.0.0 |
Major Changes
com.unity.ml-agents (C#)
- The
MLAgents
C# namespace was renamed toUnity.MLAgents
, and other nested namespaces were similarly renamed. (#3843) - The offset logic was removed from
DecisionRequester
. (#3716) - The signature of
Agent.Heuristic()
was changed to take a float array as a parameter, instead of returning the array. This was done to prevent a common source of error where users would return arrays of the wrong size. (#3765) - The communication API version has been bumped up to 1.0.0 and will use Semantic Versioning to do compatibility checks for communication between Unity and the Python process. (#3760)
- The obsolete
Agent
methodsGiveModel
,Done
,InitializeAgent
,AgentAction
andAgentReset
have been removed. (#3770) - The SideChannel API has changed:
- Introduced the
SideChannelManager
to register, unregister and access side channels. (#3807) Academy.FloatProperties
was replaced byAcademy.EnvironmentParameters
. See the Migration Guide for more details on upgrading. (#3807)SideChannel.OnMessageReceived
is now a protected method (was public). (#3807)- SideChannel IncomingMessages methods now take an optional default argument, which is used when trying to read more data than the message contains. (#3751)
- Added a feature to allow sending stats from C# environments to TensorBoard (and other python StatsWriters). To do this from your code, use
Academy.Instance.StatsRecorder.Add(key, value)
. (#3660)
- Introduced the
CameraSensorComponent.m_Grayscale
andRenderTextureSensorComponent.m_Grayscale
were changed frompublic
toprivate
. These can still be accessed via their corresponding properties. (#3808)- Public fields and properties on several classes were renamed to follow Unity's C# style conventions. All public fields and properties now use "PascalCase" instead of "camelCase"; for example,
Agent.maxStep
was renamed toAgent.MaxStep
. For a full list of changes, see the pull request. (#3828) WriteAdapter
was renamed toObservationWriter
. If you have a customISensor
implementation, you will need to change the signature of itsWrite()
method. (#3834)- The Barracuda dependency was upgraded to 0.7.0-preview (which has breaking namespace and assembly name changes). (#3875)
ml-agents / ml-agents-envs / gym-unity (Python)
- The
--load
and--train
command-line flags have been deprecated. Training now happens by default, and use--resume
to resume training instead of--load
. (#3705) - The Jupyter notebooks have been removed from the repository. (#3704)
- The multi-agent gym option was removed from the gym wrapper. For multi-agent scenarios, use the Low Level Python API. (#3681)
- The low level Python API has changed. You can look at the document Low Level Python API documentation for more information. If you use
mlagents-learn
for training, this should be a transparent change. (#3681) - Added ability to start training (initialize model weights) from a previous run ID. (#3710)
- The GhostTrainer has been extended to support asymmetric games and the asymmetric example environment Strikers Vs. Goalie has been added. (#3653)
- The
UnityEnv
class from thegym-unity
package was renamedUnityToGymWrapper
and no longer creates theUnityEnvironment
. Instead, theUnityEnvironment
must be passed as input to the constructor ofUnityToGymWrapper
(#3812)
Minor Changes
com.unity.ml-agents (C#)
- Added new 3-joint Worm ragdoll environment. (#3798)
StackingSensor
was changed frominternal
visibility topublic
. (#3701)- The internal event
Academy.AgentSetStatus
was renamed toAcademy.AgentPreStep
and made public. (#3716) - Academy.InferenceSeed property was added. This is used to initialize the random number generator in ModelRunner, and is incremented for each ModelRunner. (#3823)
Agent.GetObservations()
was added, which returns a read-only view of the observations added inCollectObservations()
. (#3825)UnityRLCapabilities
was added to help inform users when RL features are mismatched between C# and Python packages. (#3831)
ml-agents / ml-agents-envs / gym-unity (Python)
- Format of console output has changed slightly and now matches the name of the model/summary directory. (#3630, #3616)
- Renamed 'Generalization' feature to 'Environment Parameter Randomization'. (#3646)
- Timer files now contain a dictionary of metadata, including things like the package version numbers. (#3758)
- The way that UnityEnvironment decides the port was changed. If no port is specified, the behavior will depend on the
file_name
parameter. If it isNone
, 5004 (the editor port) will be used; otherwise 5005 (the base environment port) will be used. (#3673) - Running
mlagents-learn
with the same--run-id
twice will no longer overwrite the existing files. (#3705) - Model updates can now happen asynchronously with environment steps for better performance. (#3690)
num_updates
andtrain_interval
for SAC were replaced withsteps_per_update
. (#3690)- The maximum compatible version of tensorflow was changed to allow tensorflow 2.1 and 2.2. This will allow use with python 3.8 using tensorflow 2.2.0rc3. (#3830)
mlagents-learn
will no longer set the width and height of the executable window to 84x84 when no width nor height arguments are given. (#3867)
Bug Fixes
com.unity.ml-agents (C#)
- Fixed a display bug when viewing Demonstration files in the inspector. The shapes of the observations in the file now display correctly. (#3771)
ml-agents / ml-agents-envs / gym-unity (Python)
- Fixed an issue where exceptions from environments provided a return code of 0. (#3680)
ML-Agents Beta 0.15.1
Bug Fixes
- Raise the wall in the CrawlerStatic scene to prevent the Agent from falling off. (#3650)
- Fixed an issue where specifying
vis_encode_type
was required only for SAC. (#3677) - Fixed the reported entropy values for continuous actions. (#3684)
- Fixed an issue where switching models using
SetModel()
during training would use an excessive amount of memory. (#3664) - Environment subprocesses now close immediately on timeout or wrong API version. (#3679)
- Fixed an issue in the gym wrapper that would raise an exception if an Agent called EndEpisode multiple times in the same step. (#3700)
- Fixed an issue where logging output was not visible; logging levels are now set consistently. (#3703)
ML-Agents Beta 0.15.0
Major Changes
Agent.CollectObservations
now takes a VectorSensor argument. (#3352, #3389)- Added
Agent.CollectDiscreteActionMasks
virtual method with aDiscreteActionMasker
argument to specify which discrete actions are unavailable to the Agent. (#3525) - We consolidated our API for
DiscreteActionMasker
.SetMask
takes two arguments : the branch index and the list of masked actions for that branch. (#3525) - Beta support for ONNX export was added. If the
tf2onnx
python package is installed, models will be saved to.onnx
as well as.nn
format. Note that Barracuda 0.6.0 or later is required to import the.onnx
files properly. (#3101) - Multi-GPU training and the
--multi-gpu
option has been removed temporarily. (#3345) - All Sensor related code has been moved to the namespace
MLAgents.Sensors
. (#3519) - All SideChannel related code has been moved to the namespace
MLAgents.SideChannels
. (#3533) - The following methods in the
Agent
class have been deprecated and will be removed in a later release (#3557):InitializeAgent()
was renamed toInitialize()
AgentAction()
was renamed toOnActionReceived()
AgentReset()
was renamed toOnEpisodeBegin()
Done()
was renamed toEndEpisode()
GiveModel()
was renamed toSetModel()
Minor Changes
- Monitor.cs was moved to Examples. (#3372)
- Automatic stepping for Academy is now controlled from the AutomaticSteppingEnabled property. (#3376)
- The GetEpisodeCount, GetStepCount, GetTotalStepCount and methods of Academy were changed to EpisodeCount, StepCount, TotalStepCount properties respectively. (#3376)
- Several classes were changed from public to internal visibility. (#3390)
- Academy.RegisterSideChannel and UnregisterSideChannel methods were added. (#3391)
- A tutorial on adding custom SideChannels was added. (#3391)
- The stepping logic for the Agent and the Academy has been simplified. (#3448)
- Update Barracuda to 0.6.1-preview. (#3584)
- The interface for
RayPerceptionSensor.PerceiveStatic()
was changed to take an input class and write to an output class, and the method was renamed toPerceive()
. (#3527) - The checkpoint file suffix was changed from
.cptk
to.ckpt
. (#3470) - The command-line argument used to determine the port that an environment will listen on was changed from
--port
to--mlagents-port
. (#3477) DemonstrationRecorder
can now record observations outside of the editor. (#3354)DemonstrationRecorder
now has an optional path for the demonstrations. This will default toApplication.dataPath
if not set. (#3354)DemonstrationStore
was changed to accept aStream
for its constructor, and was renamed toDemonstrationWriter
. (#3354)- The method
GetStepCount()
on the Agent class has been replaced with the property getterStepCount
. (#3476) RayPerceptionSensorComponent
and related classes now display the debug gizmos whenever the Agent is selected (not just Play mode). (#3484)- Most fields on
RayPerceptionSensorComponent
can now be changed while the editor is in Play mode. The exceptions to this are fields that affect the number of observations. (#3484) - Most fields on
CameraSensorComponent
andRenderTextureSensorComponent
were changed to private and replaced by properties with the same name. (#3564) - Unused static methods from the
Utilities
class (ShiftLeft, ReplaceRange, AddRangeNoAlloc, and GetSensorFloatObservationSize) were removed. (#3495) - The
Agent
class is no longer abstract. (#3528) - SensorBase was moved out of the package and into the Examples directory. (#3528)
AgentInfo.actionMasks
has been renamed toAgentInfo.discreteActionMasks
. (#3539)DecisionRequester
has been made internal (you can still use the DecisionRequesterComponent from the inspector).RepeatAction
was renamed toTakeActionsBetweenDecisions
for clarity. (#3555)- The
IFloatProperties
interface has been removed. (#3570) - Improved inference performance for models with multiple action branches. (#3598)
- The interfaces to the
SideChannel
classes (on C# and python) have changed to use newIncomingMessage
andOutgoingMessage
classes. These should make reading and writing data to the channel easier. (#3596)
Bug Fixes and Improvements
- Various documentation changes and additions. (#3603, #3574, #3577, #3568, #3567, #3551, #3582)
- Fixed logging issue with SAC. (#3606)
- Fixed an issue when using GAIL with less than
batch_size
number of demonstrations. (#3591) - Made Unity lifecycle methods protected (
OnEnable
,OnDisable
). (#3590) - SideChannel messages were lost if sent before the first call to reset from Python. (#3573)
- Updated the ExpertPyramid.demo example demonstration file. (#3613)
- Updated project version for example environments to 2018.4.18f1. (#3618)
- Changed the
Product Name
in the example environments to remove spaces, so that the default build executable file doesn't contain spaces. (#3612)
ML-Agents Beta 0.14.1
Bug Fixes
- Fixed an issue which caused self-play training sessions to consume a lot of memory. (#3451)
- Fixed an IndexError when using GAIL or behavioral cloning with demonstrations recorded with 0.14.0 or later (#3464)
- Updated the
gail_config.yaml
to work with per-Agent steps (#3475) - Fixed demonstration recording of experiences when the Agent is done. (#3463)
- Fixed a bug with the rewards of multiple Agents in the gym interface (#3471, #3496)
ML-Agents Beta 0.14.0
Major Changes
- A new self-play mechanism for training agents in adversarial scenarios was added (#3194)
- Tennis and Soccer environments were refactored to enable training with self-play (#3194, #3331)
- UnitySDK folder was split into a Unity Package (com.unity.ml-agents) and our examples were moved to the Project folder (#3267)
- Academy is now a singleton and is no longer abstract (#3210, #3184)
- In order to reduce the size of the API, several classes and methods were marked as internal or private.
- Some public fields on the Agent were trimmed (#3342, #3353, #3269)
- Decision Period and on-demand decision checkboxes were removed from the Agent. on-demand decision is now the default (#3243)
- Calling Done() on the Agent will reset it immediately and call the AgentReset virtual method (#3291, #3242)
- The "Reset on Done" setting in AgentParameters was removed; this is now always true. AgentOnDone virtual method on the Agent was removed (#3311, #3222)
- Trainer steps are now counted per-Agent, not per-environment as in previous versions. For instance, if you have 10 Agents in the scene, 20 environment steps now correspond to 200 steps as printed in the terminal and in Tensorboard (#3113)
Minor Changes
- Barracuda was updated to 0.5.0-preview (#3329)
--num-runs
option was removed from mlagents-learn (#3155)- Curriculum config files are now YAML formatted and all curricula for a training run are combined into a single file (#3186)
- ML-Agents components, such as BehaviorParameters and various Sensor implementations, now appear in the Components menu (#3231)
- Exceptions are now raised in Unity (in debug mode only) if NaN observations or rewards are passed (#3221)
- RayPerception MonoBehavior, which was previously deprecated, was removed (#3304)
- Uncompressed visual (i.e. 3d float arrays) observations are now supported. CameraSensorComponent and RenderTextureSensor now have an option to write uncompressed observations (#3148)
- Agent’s handling of observations during training was improved so that an extra copy of the observations is - no longer maintained (#3229)
- Error message for missing trainer config files was improved to include the absolute path (#3230)
- Support for 2017.4 LTS was dropped (#3121, #3168)
- Some documentation improvements were made (#3296, #3292, #3295, #3281)
Bug Fixes
- Numpy warning when stats don’t exist (#3251)
- A bug that caused RayPerceptionSensor to behave inconsistently with transforms that have non-1 scale was fixed (#3321)
- Some small bugfixes to tensorflow_to_barracuda.py were backported from the barracuda release (#3341)
- Base port in the jupyter notebook example was updated to use the same port that the editor uses (#3283)
ML-Agents Beta 0.13.1
ML-Agents Beta 0.13.0
Major Changes
- The low level Python API has changed (#3022). See the Low Level Python API documentation for more information.
- Parameters such as resolution and time scale are no longer configured in the Academy; these are now passed on the mlagents-learn command line. See the Migration Guide for more details. (#2956)
- Offline Behavioral Cloning training was removed. To learn from demonstrations, use the GAIL and Behavioral Cloning features with either PPO or SAC (#2969)
- Agents can now use sensors of child GameObjects as observations (#3095)
- The RayPerceptionSensor now supports a layerMask option that is used in raycasting. (#3111 )
- The official minimum version of Unity supported by ML-Agents is now 2018.4 LTS.
For instructions on how to migrate from 0.12.0 to 0.13.0 see the Migration Guide.
Minor Fixes and Improvements
- A separation of statistics collection from the trainer with the StatsReporter and StatsWriter classes (#3076, #3108)
- mlagents.envs was renamed to mlagents_envs
- A new AgentProcessor class and Trajectory abstraction is now live. Trainers now ingest trajectories which are assembled by the AgentProcessor. (#3067)
- A bug that could cause the Academy to call a disabled agent’s _AgentReset() method is fixed (#3072)
- Better error handling when the trainer configuration doesn’t contain a “default” entry (#3063)
- Better error handling when there is a mismatch between the metacurriculum configuration and brains being trained (#3034)
- A bug that prevented agents with different decision intervals from learning in the same scene is fixed (#3181)
ML-Agents Beta 0.12.1
Fixes & Performance Improvements
- Fixed bug that could degrade training for recurrent networks (#3066). If you are on 0.12.0 and using LSTMs to train your agents, we recommend you upgrade to 0.12.1 and retrain your model.
- Fixed bug in the environment setup in a Jupyter notebook (#3068)
- Added extra logging on API versions in port numbers, to help debugging connection issues (#3069)
- Minor documentation fixes (#3070)
ML-Agents Beta 0.12.0
Major Changes
- TensorFlow 2.0.0 is now supported. This is done through the
tf.compat.v1
interface (#2665) - Vector observations from the Agent are now sent using the ISensor interface. (#2813)
- Demonstration files recorded using previous versions are not compatible with 0.12.0
- Custom observations, custom actions, text observations, and text actions were removed. (#2839)
- Raycast and spherecast observations are now done through a RayPerceptionSensor, and should be configured by adding a RayPerceptionSensorComponent3D to an Agent. See the Migration Guide for more details. (#2874)
- RayPerceptionSensor adds more configuration such as toggling between raycasts and spherecasts, and better debug display.
- RayPerception3D and RayPerception2D are deprecated and will be removed in the next release.
For instructions on how to migrate from 0.11.0 to 0.12.0 see the Migration Guide.
Minor Fixes and Improvements
- A bug in how RayPerception3D used the vertical offset parameters was fixed. This may cause a behavior change in your observations; see the Migration Guide for more details. (#2823)
- A bug which swapped the height and width dimensions in visual observation was fixed. (#2919)
- Memories and previous actions are no longer passed between the environment and the trainer. These are now saved by the trainer and at inference. This affects the low-level API but existing models are still compatible. (#2795, #2913)
- Unity Inference Engine (Barracuda) was upgraded to 0.3.2, and it is now installed via the Unity Package Manager. Please see the Barracuda release notes for the list of changes.
- An issue with setting num_batches_per_epoch past the number of batches in the buffer while using recurrent and BC was fixed (#2965)
- Addressed an issue with setting pretraining steps to 0 (continuous pretraining) (#2977)
ML-Agents Beta 0.11.0
Major Changes
- The BroadcastHub has been deprecated. If there is a training Python process, all LearningBrains in the scene will automatically be trained. If there is no Python process, inference will be used.
- The Brain ScriptableObjects have been removed. The Brain Parameters are now on the Agent and are referred to as Behavior Parameters. Make sure the Behavior Parameters is attached to the Agent GameObject.
- The setup for visual observations has changed significantly.
- Camera resolutions are no longer stored in the Brain Parameters.
- AgentParameters no longer stores lists of Cameras and RenderTextures
- To add visual observations to an Agent, you must now attach a CameraSensorComponent or RenderTextureComponent to the agent. The corresponding Camera or RenderTexture can be added to these in the editor, and the resolution and color/grayscale is configured on the component itself.
- The definition of the gRPC service has changed.
- The online BC training feature has been removed.
For instructions on how to migrate from 0.10.x to 0.11.0 see the Migration Guide.
See the updated Getting Started and Basic Guide for more information.
Minor Fixes and Improvements
- Fixed an exception being thrown when dragging between two screens with different DPIs (#2782)
- Improved hyperparameters for the GridWorld environment using SAC (#2776)
- Fixed crash when SAC is used with Curiosity and Continuous Actions (#2740)
- Sped up processing large vector observations (#2717)
- Fixed bug causing excessive memory usage during inference (#2722)
- Added support for multiple training areas in the GridWorld environment (#2721)
- RayPerception.Perceive() was made abstract and RayPerception2D now correctly overrides it. (#2788)