What is the Brain stacked vectors value?

[GeriBP]

I've found this on the docummentation:

Stacked Vectors - The number of previous vector observations that will be stacked and used collectively for decision making. This results in the effective size of the vector observation being passed to the brain being: Space Size x Stacked Vectors.

But I still don't get it.

Thanks in advance

[ruanrothmann]

(speaking under correction) Let's say your observation space is 3 big e.g. an x,y and z value. if you set stacked vectors to 2, it means that your observation space is now actually 6 vectors: the current x,y,z as well as the previous update step's x,y,z. In this example, it might allow the agent to make inferences based on the velocity of what is being observed.

[GeriBP]

@ruanrothmann Cool, makes sense. Thanks :)

[tschmidt64]

Would this example be any different from manually calculating the velocity in code and manually encoding that as a separate piece of state, as opposed to just stacking position vectors?

[awjuliani]

Hi @tschmidt64,

In the case of that example it wouldn't necessarily be different that just providing the velocity. Where it becomes more useful is situations where there is more important information about what has happened in the past, or happened over time which you want to keep track of, but don't necessarily have a good representation for. Imagine an FPS where an agent might need to keep track of where it last saw an enemy that is now out of view.

[GeriBP]

@awjuliani Oh. And now that I think about it how do we choose which values to stack. Lets say we pass in 9 parameters and stack 3. The first 3 would be the ones getting stacked right?

[awjuliani]

@GeriBP,

They all get stacked. The "Number of stacked" parameter changes how many sets of observations into the past you'd like to stack. Increasing this allows the agent to "see" further into the past.

[GeriBP]

@awjuliani Nice. Got it.

[q7777777hk]

@awjuliani

What is the difference between

1. setting "Number of stacked"to X
2. enabling recurrent, then set sequence_length to X ?

[awjuliani]

Hi @q7777777hk,

The difference is that in the first case the network used to control the agent will be a simple feed-forward network which is fed the last X observations. In the second case a RNN will be created, and only the most recent observation will be fed into the network.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.