Robust point-based value iteration algorithm for partially observable Markov decision processes (POMDPs), including standard reward and belief-reward versions. This application is built as solver for robust POMDPs.
This application is built for Julia 0.6. If not already installed, the application can be cloned using
Pkg.clone("https://github.com/ajkeith/RobustValueIteration")To solve robust POMDP models, RPOMDPs, RPOMDPToolbox, RPOMDPModels, and SimpleProbabilitySets also need to be cloned
Pkg.clone("https://github.com/ajkeith/RPOMDPs.jl/tree/ajk/robust")
Pkg.clone("https://github.com/ajkeith/RPOMDPToolbox.jl")
Pkg.clone("https://github.com/ajkeith/RPOMDPModels.jl")
Pkg.clone("https://github.com/ajkeith/SimpleProbabilitySets.jl")This solver can be used with the robust POMDP applications. See RPOMDPs.jl, RPOMDPToolbox.jl, RPOMDPModels.jl, and SimpleProbabilitySets.jl for details on writing robust POMDP problems and associated tools.
using RobustValueIteration
using RPOMDPModels, RPOMDPs, RPOMDPToolbox, SimpleProbabilitySets
rpomdp = RockRIPOMDP()
b = [psample(zeros(4), ones(4)) for i = 1:10]
solver = RPBVISolver(beliefpoints = b, max_iterations = 10)
policy = RobustValueIteration.solve(solver, rpomdp)Scripts for generating results and figures are available at RobustInfoPOMDP.
The standard-reward robust point-based value iteration algorithm implements the pseudo code from Osogami (2015). The robust POMDP environment is a direct extension of POMDPs.jl, POMDPModels.jl, and POMDPModelTools.jl to the robust setting.
If this code is useful to you, please star this package and consider citing the following papers.
Egorov, M., Sunberg, Z. N., Balaban, E., Wheeler, T. A., Gupta, J. K., & Kochenderfer, M. J. (2017). POMDPs.jl: A framework for sequential decision making under uncertainty. Journal of Machine Learning Research, 18(26), 1–5.
Osogami, T. (2015). Robust partially observable Markov decision process. In International Conference on Machine Learning (pp. 106–115).