More information about the LongerExplorationPolicy

[dynamik1703]

Hey VinF,

do you have more information about the LongerExplorationPolicy?

I'm wondering whether this policy is suitable for my environment. How should the length parameter be chosen?

Thanks!

Best wishes

[VinF]

Hi dynamik,

Basically the idea is that if you have a pure random exploration, you will end up with all possible ordered sequences that have uniform probabilities. E.g., if a set of two possible actions {1,2} and two time steps, the sequences {11,12,21,22} have all the same probability 0.25 of being tried out.
For the LongerExplorationPolicy, the unordered sequences have uniform probabilities. So in the exemple, {11} has 0.33 probability, {22} has 0.33 and {12, 21} have together 0.33 (0.17 each). That can be useful in environment such as grid world where the order of the actions does not matter in most situations.

The length parameter should be chosen depending on your environment. Usually you'll have to try a few possibilities empirically and see what works.

Best,
Vincent

[dynamik1703]

Hi VinF,

thanks for your quick response!

How do you evaluate the Ornstein-Uhlenbeck-Process in comparison?

Best,
Roman

[VinF]

Hi Roman,
Indeed, you could possibly find parallels with the nomenclature in the domain of stochastic processes depending on the setting considered.