Options Framework, Intra Option Learning, SMDP formulations
Implementation of paper Between MDPs and SMDPs. http://webdocs.cs.ualberta.ca/~sutton/papers/SPS-aij.pdf options vs no options comparison on maze task with bottleneck state. Task is to reach goal location, agent with option has the subgoal as reaching the bottlneck state. executeoption.m runs the agent with the option.
Comparisons.jpg shows the difference in convergence rates.