From df9f6c804b7d28efcbd83c7aac33d81f3e1bfb0f Mon Sep 17 00:00:00 2001 From: Arthur Juliani Date: Wed, 14 Mar 2018 13:23:43 -0700 Subject: [PATCH 1/3] Add additional line on stacked observations --- docs/Learning-Environment-Design-Brains.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Learning-Environment-Design-Brains.md b/docs/Learning-Environment-Design-Brains.md index b073b4e98c..b7458a0b5c 100644 --- a/docs/Learning-Environment-Design-Brains.md +++ b/docs/Learning-Environment-Design-Brains.md @@ -23,7 +23,7 @@ The Brain Inspector window in the Unity Editor displays the properties assigned * `Vector Observation` * `Space Type` - Corresponds to whether the observation vector contains a single integer (Discrete) or a series of real-valued floats (Continuous). * `Space Size` - Length of vector observation for brain (In _Continuous_ space type). Or number of possible values (in _Discrete_ space type). - * `Stacked Vectors` - The number of previous vector observations that will be stacked before being sent to the brain. + * `Stacked Vectors` - The number of previous vector observations that will be stacked before being sent to the brain. This results in the effective size of the vector observation being passed to the brain being: _Space Size_ x _Stacked Vectors_. * `Visual Observations` - Describes height, width, and whether to grayscale visual observations for the Brain. * `Vector Action` * `Space Type` - Corresponds to whether action vector contains a single integer (Discrete) or a series of real-valued floats (Continuous). From bc9b1e4710b3b404a1477073b0a4451837efa3d7 Mon Sep 17 00:00:00 2001 From: Arthur Juliani Date: Wed, 14 Mar 2018 13:29:47 -0700 Subject: [PATCH 2/3] Add mention of stacked vectors in another place --- docs/Learning-Environment-Best-Practices.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/Learning-Environment-Best-Practices.md b/docs/Learning-Environment-Best-Practices.md index dc444c2668..f3da5e452e 100644 --- a/docs/Learning-Environment-Best-Practices.md +++ b/docs/Learning-Environment-Best-Practices.md @@ -15,6 +15,7 @@ complexity over time. This can either be done manually, or via Curriculum Learni ## Vector Observations * Vector Observations should include all variables relevant to allowing the agent to take the optimally informed decision. +* In cases where Vector Observations need to be remembered or compared over time, increase the `Stacked Vectors` value to allow the agent to keep track of multiple observations into the past. * Categorical variables such as type of object (Sword, Shield, Bow) should be encoded in one-hot fashion (i.e. `3` -> `0, 0, 1`). * Besides encoding non-numeric values, all inputs should be normalized to be in the range 0 to +1 (or -1 to 1). For example, the `x` position information of an agent where the maximum possible value is `maxValue` should be recorded as `AddVectorObs(transform.position.x / maxValue);` rather than `AddVectorObs(transform.position.x);`. See the equation below for one approach of normalization. * Positional information of relevant GameObjects should be encoded in relative coordinates wherever possible. This is often relative to the agent position. From ce1e622dc031b1fa3306b1e3caefc58f29e6f774 Mon Sep 17 00:00:00 2001 From: Arthur Juliani Date: Wed, 14 Mar 2018 14:47:18 -0700 Subject: [PATCH 3/3] Adjust wording --- docs/Learning-Environment-Design-Brains.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Learning-Environment-Design-Brains.md b/docs/Learning-Environment-Design-Brains.md index b7458a0b5c..649429f886 100644 --- a/docs/Learning-Environment-Design-Brains.md +++ b/docs/Learning-Environment-Design-Brains.md @@ -23,7 +23,7 @@ The Brain Inspector window in the Unity Editor displays the properties assigned * `Vector Observation` * `Space Type` - Corresponds to whether the observation vector contains a single integer (Discrete) or a series of real-valued floats (Continuous). * `Space Size` - Length of vector observation for brain (In _Continuous_ space type). Or number of possible values (in _Discrete_ space type). - * `Stacked Vectors` - The number of previous vector observations that will be stacked before being sent to the brain. This results in the effective size of the vector observation being passed to the brain being: _Space Size_ x _Stacked Vectors_. + * `Stacked Vectors` - The number of previous vector observations that will be stacked and used collectively for decision making. This results in the effective size of the vector observation being passed to the brain being: _Space Size_ x _Stacked Vectors_. * `Visual Observations` - Describes height, width, and whether to grayscale visual observations for the Brain. * `Vector Action` * `Space Type` - Corresponds to whether action vector contains a single integer (Discrete) or a series of real-valued floats (Continuous).