Note: this repository is depricated. For a a package with more features, and support for both MDPs and POMDPs see POMDPs.jl.
This package provides a basic interface for working with discrete Markov decision processes (MDPs).
The basic types are
DiscreteMDPSolverSimulatorPolicyMDPFilePolicyFile
solve(solver::Solver, mdp::DiscreteMDP)simulate(simulator::Simulator, mdp::DiscreteMDP, policy::Policy)
load(file::PolicyFile)load(file::MDPFile)save(file::PolicyFile, policy::Policy)save(file::MDPFile, mdp::DiscreteMDP)
states(mdp::DiscreteMDP)returns something that might be iterable (e.g., 1:100)actions(mdp::DiscreteMDP)returns something that might be iterable (e.g., 1:100)actions(mdp::DiscreteMDP, state)returns something that might be iterable (e.g., 1:100)reward(mdp::DiscreteMDP, state, action)returns rewardtransition(mdp::DiscreteMDP, state, action, nextState)returns probabilitynextStates(mdp::DiscreteMDP, state, action)returns arrays of state indices and their corresponding probabilities
action(p::Policy, state)retuns the action according to policy pvalue(p::Policy, state, action)returns the expected value for a (s,a) pairvalue(p::Policy, state)returns the optimal expeted value for state s