| Title | Action Type | Action Shape | Action Values | Observation Shape | Observation Values | Average Total Reward | Import |
|---|---|---|---|---|---|---|---|
| Copy | Discrete | (3,) | [(0, 1),(0,1),(0,base-1)] | (1,) | (0,base) | from gym.envs.algorithmic import copy_ |
This task involves copying content from the input tape to the output tape. This task was originally used in the paper Learning Simple Algorithms from Examples.
The model has to learn:
- correspondence between input and output symbols.
- executing the move right action on input tape.
The agent take a 3-element vector for actions.
The action space is (x, w, v), where:
xis used for left/right movement. It can take values (0,1).wis used for writing to output tape or not. It can take values (0,1).ris used for selecting the value to be written on output tape.
The observation space size is (1,) .
Rewards:
Rewards are issued similar to other Algorithmic Environments. Reward schedule:
- write a correct character: +1
- write a wrong character: -.5
- run out the clock: -1
- otherwise: 0
gym.make('Copy-v0', base=5, chars=True)
base: Number of distinct characters to read/write.
chars: If True, use uppercase alphabet. Otherwise, digits. Only affects rendering.
- v0: Initial versions release (1.0.0)