The alternative to gen that keeps me up at night

[zsunberg]

I'm not writing this issue because I necessarily think we should do it; I just want to document it somewhere.

This is the alternative to gen that keeps me up at night. I do not know how to decide if it is better than gen or not.

Instead of gen, we could have an "implicit distribution" type which contains a function that allows rand to sample from it, i.e.

struct Implicit{A<:Tuple, F<:Function}
    args::A
    sampler::F
end

Base.rand(rng::AbstractRNG, d::Implicit) = d.sampler(d.args..., rng)

and it would be used like this:

function transition(m::MyPOMDP, s, a)
    return Implicit(m, s, a) do m, s, a, rng
        return s + a + randn(rng)
    end
end

A big plus is that it reduces the surface area of POMDPs.jl. There are fewer functions exported, and every problem would have a transition function.

I am not sure if it would be possible to support a syntax like

POMDPs.gen(::MyPOMDP, s, a, rng) = (sp=s+a+randn(rng), r=s^2)

to support cases where there is an external simulator that produces states and rewards in one shot though.

[zsunberg]

I guess this would only replace the DDNNode version of gen, so maybe they could live in harmony.

[lassepe]

I would be all for this if we make something like a ConditionalDistributions.jl package. However, if we end up exporting Implicit from POMDPs.jl this does not really help to reduce the surface area, in particular if we still have gen for some of the cases.

Edit:
Thinking more about this, I start to really like this idea. If we rename the whole "generative" interface to be the "implicit" interface, this sounds like a very consistent way of describing things.

[zsunberg]

This has been implemented