Multi Scale Actor Critic Morris Maze Model
This models a Morris Maze task with a fixed platform. It implements a multi-scale actor critic architecture.
The XML file is in multiscalemodel/src/edu/usf/ratsim/experiment/xml/morris/multiscale/actorcritic/virtual.xml.
Some important parameters are:
- numCCLayers: Number of conjunctive cell layers
- numCCCellsPerLayer: Number of cells per layer
- votesConnProbs: Layer connection probabilities to action selection for gradient connections
- valueConnProbs: Layer connection probabilities to value estimation for gradient connections
- minPCRadius: Minimum conjuntive layer cell radius (meters)
- maxPCRadius: Maximum conjuntive layer cell radius (meters)
- votesNormalizer, valueNormalizer: Normalizes activity created by the conjunctive cells
- minHDRadius: Minimum angular radius of conjuntive cells (radians)
- maxHDRadius: Maximum angular radius of conjuntive cells (radians)
- foodReward: Reward value for food
- nonFoodReward: Reward value given when it doesnt eat
- rlDiscountFactor: Lambda discount factor for the learning algorithm
- alpha: Alpha for the learning algorithm
- explorationReward: Value assigned to exploration action-values
- explorationHalfLifeVal: Half life value for decay exploration
Training: naive subjects are placed in the maze. Each episode starts from the same position and ends when the animat gets to the platform (or at timeout).
Recall: the platform stays in the same place, fewer episodes are executed to assess the rat knowledge of the maze.
Learning: executes both trials.
Control: only executes the recall trial, serving as a negative control.
For one individual, the Training/plots/ folder withing the logs folder should contain the runtimesEpisodebox.Training.pdf plot. It should look as follows:
That folder also contains value and path plots. Each dot represents the location of a conjunctive cell and its color represents its value. The plots should look like:
In the Recall/plots folder, a runtimes plots can also be found. It should look like:
If all individuals are executed, the plots should look like below. Notice that only recall has both groups, as the Control group does not run the Training trial.
