Skip to content

MODIA framework for (PO)MDPs written in Julia, as a part of the POMDPs.jl ecosystem.

License

Notifications You must be signed in to change notification settings

sisl/POMDPModia.jl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

POMDPModia

This package provides a core interface for creating MODIA objects comprising Markov decision processes (MDPs) and/or partially observable Markov decision processes (POMDPs). The package is written in the Julia Programming Language.

! TODO: Describe MODIA architecture in more detail here.

The POMDPModia.jl package is highly compatible and dependent on the other packages in the JuliaPOMDP ecosystem.

Dependencies

The (PO)MDP definitions used in POMDPModia.jl are instantiate through the POMDPs.jl package. Through the POMDPs.jl interface, problems are defined and solved using a large variety of tools available:

Package Build Coverage
POMDPs Build Status Coverage Status
QuickPOMDPs Build Status Coverage Status
POMDPModelTools Build Status Coverage Status
BeliefUpdaters Build Status Coverage Status
POMDPPolicies Build Status Coverage Status
POMDPSimulators Build Status Coverage Status
POMDPModels Build Status Coverage Status
POMDPTesting Build Status Coverage Status

Also, see the supported solvers for the summary of solvers available, and the solvers documentation for examples and writing custom solvers.

Quick Start

In this short example, a MODIA will be created for various amounts of three different TigerPOMDP characteristics.

using POMDPModels: TigerPOMDP
using QMDP: QMDPSolver

# Define the DPs, DCs, and SSF for a MODIA object
tiger_problem1 = TigerPOMDP(-1.0, -100.0, 10.0, 0.90, 0.90);
tiger_problem2 = TigerPOMDP(-2.0, -50.0, 17.0, 0.80, 0.60);
tiger_problem3 = TigerPOMDP(-3.0, -75.0, 8.0, 0.85, 0.75);
DPs = [tiger_problem1, tiger_problem2, tiger_problem3];
DCs = [4, 1, 2];
SSF = Base.minimum;

# Create a MODIA, where all POMDP beliefs are initialized uniformly
modia = MODIA(DPs, DCs, SSF)
initialize_beliefs!(modia, BeliefUpdaters.uniform_belief);

# Compute optimal alpha vectors (policies) offline, and a define belief updater
solver = QMDPSolver();
policies = POMDPs.solve(solver, modia); 
bu = BeliefUpdaters.DiscreteUpdater(modia);

# Simulate the MODIA object for 5 timesteps, retrieve discounted rewards
sim = POMDPSimulators.RolloutSimulator(max_steps=5)
initial_states = convert(Array{Bool},(rand(initialstate(modia))))
r_totals = POMDPs.simulate(sim, modia, policies, bu, modia.beliefs, initial_states)

Citations

If you have found the MODIA architecture to be useful, consider citing the paper:

@inproceedings{ijcai2017-664,
  author    = {Kyle Hollins Wray and Stefan J. Witwicki and Shlomo Zilberstein},
  title     = {Online Decision-Making for Scalable Autonomous Systems},
  booktitle = {Proceedings of the Twenty-Sixth International Joint Conference on
               Artificial Intelligence, {IJCAI-17}},
  pages     = {4768-4774},
  year      = {2017},
  doi       = {10.24963/ijcai.2017/664},
  url       = {https://doi.org/10.24963/ijcai.2017/664},
} 

About

MODIA framework for (PO)MDPs written in Julia, as a part of the POMDPs.jl ecosystem.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages