MDP/POMDP support

[MaximeBouton]

The vanilla solver supports both MDPs and POMDPs. Should the Sparse solver support POMDPs as well?
The current way that one can use it to solve a POMDP is as follows:

mdp = UnderlyingMDP(pomdp)
solve(SparseValueIterationSolver(), mdp)

[zsunberg]

Seems better to use UnderlyingMDP because then it's not an expectation for all MDP solver writers to also support POMDPs, but it is kind of annoying to have to find it.

One option is to make both solvers accept pomdps but throw a warning that says to use UnderlyingMDP.

In the end, I think it's probably just most convenient to have both solvers accept both just for convenience.

[MaximeBouton]

I agree that in general MDP solvers should not be required to support POMDPs.

The main reason why Discrete VI supports POMDPs is because it is used in QMDP.jl I think. An alternative is to have a SparseQMDP solver. Then in QMDP.jl it should use UnderlyingMDP.

[zsunberg]

So it seems like the best thing to do is

@deprecate solve(s::ValueIterationSolver, m::POMDP) solve(s, UnderlyingMDP(m))

and throw a similar warning for SparseValueIterationSolver,

in QMDP, we should really have a solver field that can be any solver, and then use value(policy, s, a) to get the q values, and it should feed that solver an UnderlyingMDP.

[MaximeBouton]

Fixed JuliaPOMDP/QMDP.jl#15 and #32