Add alternation environment #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a simple environment, which is easy to solve. It provides a quick means of sanity checking learning algorithms making sure they learn anything. The simplicity of the environment makes it easier to understand any obvious problems an algorithm might have. Additionally, it is quick to execute. A random agent will always perform poorly compared to an agent successfully learning the one winning strategy of this environment.
SwamyDev
commented
Oct 21, 2019
tests/test_alternation.py
Outdated
|
|
||
| def test_agent_starts_randomly_left_or_right(sample_reset): | ||
| avg_obs = sample_reset(100) | ||
| assert left(avg_obs) == approx(0.5, abs=1) and right(avg_obs) == approx(0.5, abs=1) |
There was a problem hiding this comment.
the absolute approx is quite high
tests/test_alternation.py
Outdated
Comment on lines
41
to
49
| @pytest.fixture | ||
| def sample_signal(): | ||
| def sample_signal_func(func, n): | ||
| total = 0 | ||
| for _ in range(n): | ||
| total += unpack_reward(func()) | ||
| return total / n | ||
|
|
||
| return sample_signal_func |
There was a problem hiding this comment.
It seems this is not used anymore
README.md
Outdated
|
|
||
|  | ||
|
|
||
| The agent receives a normally distributed reward of 1 when switching from one state to the other, and a normally distributes penalty of -1 when staying in its current state. The environment terminates after a fixed amount of steps. This setup doesn't have any local minima the agent could reach. Hence most agents should be able to learn the optimal policy quickly. However, a random agent only achieves, on average, a total reward around zero. It makes this environment well suited for sanity checking algorithms making sure that they learn at all. By providing such a simple setup, it is also easier to comprehend any obvious problems an algorithm might have. |
There was a problem hiding this comment.
Well, it kinda has a local minimum, like if it alternates 1 time and then stops or alternates 2 times and then stops, but the reward scales linearly - the more alternation the higher the reward. I should rephrase this to highlight this function
Fix readme, clarifying the purpose of the alternation environment. Remove superfluous code and make one of the statistical tests more robust.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a simple environment, which is easy to solve. It provides a
quick means of sanity checking learning algorithms making sure they
learn anything. The simplicity of the environment makes it easier to
understand any obvious problems an algorithm might have. Additionally,
it is quick to execute. A random agent will always perform poorly
compared to an agent successfully learning the one winning strategy of
this environment.