The @provide/provided mechanism for optional interface functions.

[zsunberg]

What do we think of the @provide mechanism? That is the experimental/risky part of this package. Does anyone foresee big problems. Feel free to play around with it and see if it is intuitive.

[rejuvyesh]

Would this be enough? I think ChainRules.jl also does something like that.

[zsunberg]

https://discourse.julialang.org/t/mechanism-for-optional-interface-functions-is-this-a-bad-idea/41466

[zsunberg]

Hmm... yeah @rejuvyesh , we could just tell people to use static_hasmethod. It makes me a bit nervous to rely on that since they didn't put it in julia even though it apparently works. I couldn't find what you were talking about in ChainRules. If you remember where it is, let me know.

Is there value in having @provide as a more explicit signal that the environment is opting in to that optional part of the interface?

---

Edit: I don't think static_hasmethod will be adequate because it will make generic wrapping very hard: #1 (comment)

[rejuvyesh]

I can't find it either. I think I saw some discussions in Julia slack's #autodiff channel, but can't find it anymore.

[rejuvyesh]

I would let others pitch in. Depends on how the errors look like.

[zsunberg]

One difference is that someone could make provided return differently based on the values of the arguments instead of just the types (though I'm not sure whether we want that).

[darsnack]

According to this blog post by the author of Tricks.jl, throwing that method error might be an anti-pattern?

When an algorithm or user tries to call a method, if the environment doesn't provide it, then a method error will be thrown by Julia. So, I see the advantages of having some kind of @provide logic as:

• pretty printing the error message
• allowing someone to check easily all the methods that are provided (as opposed to hitting the default error message, fixing, then iterating over and over again)

The second bullet is the more compelling reason to me to include all the logic for @provide. Though I wonder if making provided as something to override in the interface would achieve the same functionality?

[zsunberg]

Very good thoughts. @provide/provided avoids the antipattern mentioned in that blog because it does not override the MethodError that gets thrown if you try to call a missing method.

allowing someone to check easily all the methods that are provided (as opposed to hitting the default error message, fixing, then iterating over and over again)

We have this in POMDPs: https://juliapomdp.github.io/POMDPs.jl/stable/requirements/. The main difficulty is that an algorithm-writer has to manually declare all of the requirements up front, and this may be somewhat error-prone, especially if they have to declare the types of all the arguments. (The other option is to write a very heavy-duty analyzer that looks for the interface throughout the code, and this seems difficult)

Though I wonder if making provided as something to override in the interface would achieve the same functionality?

Yes, the user can absolutely do that. @provide just syntactic sugar to do it easily in one line with the function definition.

all the logic for @provide

There is actually not too much logic (and it should be straightforward to support kwargs and where as well)

IMO the most important reason for this is actually to enable efficient behavior for different ways that problems might be implemented (IIUC this is the reason for Base.IteratorSize).

For example, in the future, we might have both

CommonRLInterface.setstate!(env, s)

and

CommonRLInterface.clone(env)

in the interface. For some envs, it might be very difficult to clone, and for others, it might be very difficult to set the state, so they might only be able to implement one.

If someone implementing MCTS wants to handle both cases elegantly and with high performance, you need either provided or static_hasmethod.

[zsunberg]

Actually, it seems very plausible that someone might want to make provided depend on the values of the arguments, e.g.

provided(CommonRLInterface.actionmask, env::PythonGymWrapper) = hasattr(env.pyenv, "actionmask")

I think that is a good enough argument to prefer provided to static_hasmethod.

[zsunberg]

Regarding the antipattern from the blog, though, it will be very tempting for someone to write:

if !provided(clone, env)
    error("You need to provide clone!")
end

and this will be terrible because it will silence the method error. 😿 😿 . The same person might write something worse without provided though. One possible mitigation would be to add something like require(clone, env) that shows a message and throws the MethodError if the method is not provided.

[jbrea]

That's a neat idea with @require. I like that it's really up to the author of a solver to define what should be provided by an environment and that one can print informative error messages to the authors of environments.

An alternative strategy could be to provide testing utilities that can be used by authors of solvers and environments, but are not in the way when actually running a solver on an environment. I am thinking of something similar to an annotated methodswith function that throws warnings and errors (or prints a table), e.g.

function check_interface_methods(env_type, action_type = Int)
    # mandatory interface
    hasmethod(step!, (env_type, action_type)) || @error "step! error message"
    hasmethod(reset!, (env_type,)) || @error "reset! error message"
    ....
    # optional interface
    hasmethod(clone, (env_type,)) || @warn "clone warning message"
    ....
end

Solver authors could also implement a method like

required_interface(::typeof(solver)) = (clone,)

and we change check_interface_methods such that one can call it with an optional solver argument that changes the type of messages of the optional interface functions from warning to error for the required methods of that solver.

The solver itself would never do any checks and simply throw MethodErrors, if it encounters an environment that doesn't implement the required optional interface methods.

What I like about this idea is that the check_interface_method would make clear which optional interface functions we consider, there is nothing that would hinder performance at runtime, but the developers can use such utilities for testing their implementations and debugging.

What do you think about this?

[darsnack]

Yeah something closer to what @jbrea described is what I was thinking. Except instead of required_interface(::typeof(solver)) = (clone,), the environment write would do provided_interface(::typeof(MyEnv)) = (clone,). But based on @zsunberg 's comments, I see that someone could write

if myfunc ∉ provided_interface(::typeof(env))
    error("Cannot find $myfunc!")
end

I thinking shifting the dynamic from what the environment provides to what the solver requires would circumvent the temptation for anyone to use these functions for any other purpose than to check if a solver and environment are compatible.

---

Though I would also add that if clone is the only optional interface we can think of, then it is easy enough to provide clone(env::CommonEnv) = deepcopy(env), and allow authors to override this method with more efficient versions. Do we have a compelling list of functions that would make up the optional interface?

[zsunberg]

Ok, seems like there are two side questions being raised:

1. requirements specification - breaking out into Requirements specification #7
2. default implementations - breaking out into Candidate Principle: CommonRLInterface does NOT provide any default implementations! #6

The question we need to answer in this thread could be stated as "Do we want a traits-like mechanism for telling what parts of the interface have been implemented"? The main purpose of this would be for dealing with multiple options for similar behavior (e.g. the setstate!and/or clone example above. My vote is: yes we do want this trait-like mechanism, and provided is the best way to implement it.