State-dependent action spaces in POMDP are not transfered to underlying MDP

[AlexBork]

The call of an underlying MDP for a POMDP does not preserve state-dependent action spaces, i.e. calling actions(UnderlyingMDP(pomdp), s) for a POMDP with state-dependent action space yields a wrong result.

Adding

POMDPs.actions(mdp::UnderlyingMDP{P, S, A}, s::S) where {P,S,A} = actions(mdp.pomdp, s)

to ModelTools/underlying_mdp.jl fixes the problem.
I propose adding the fix to the main repository.

[zsunberg]

Hi @AlexBork , thanks for reporting. One issue is that actions(m::POMDP, s) is ill-defined because the agent can't typically tell which state it is in; Thus, actions(m::POMDP, s) instead we use actions(m::POMDP, b). So your proposal might lead to errors if the person who defined the POMDP expects a belief instead of a state.

Another option would be for you to define

POMDPs.actions(::UnderlyingMDP{<:YourPOMDP}, s)

What do you think about that?

If you still think that adding the method you proposed to POMDPTools is a better solution, I'd be happy to accept a PR!

[zsunberg]

Actually I guess that actions(m::POMDP, s) is advertised as part of the interface (

POMDPs.jl/src/space.jl

Line 19 in 52ee24c

actions(m::Union{MDP,POMDP}, s)

), so we should make your proposed change.

Can you submit a PR (this would be great practice if you haven't done it before!), or should I make the change?

[AlexBork]

Hey @zsunberg, thank you very much for the swift answer. I will submit a PR containing the change.