Introduce @impl attribute and improve defoverridable

[josevalim]

One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function definitions. This is what allowed our GenServer definition to be as simple as:

defmodule MyServer do
  use GenServer
end

Implementation-wise, use GenServer defines functions such as:

def terminate(reason, state) do
  :ok
end

and then mark them as overridable:

defoverridable terminate: 2

The usage of defoverridable/1 and @optional_callbacks have one major downside: the lack of warnings for implementation mismatches. For example, imagine that instead of defining handle_call/3, you accidentally define a non-callback handle_call/2. Because handle_call/3 is optional, Elixir won't emit any warnings, so it may take a while for developers to understand why their handle_call/2 callback is not being invoked.

We plan to solve this issue by introducing the @impl true annotation that will check the following function is the implementation of a behaviour. Therefore, if someone writes a code like this:

@impl true
def handle_call(message, state) do
  ...
end

The Elixir compiler will warn that the current module has no behaviour that requires the handle_call/2 function to be implemented, forcing the developer to correctly define a handle_call/3 function. This is a fantastic tool that will not only help the compiler to emit warnings but will also make the code more readable, as any developer that later uses the codebase will understand the purpose of such function is to be a callback implementation.

The @impl annotation is optional. When @impl true is given, we will also add @doc false unless documentation has been given. We will also support a module name to be given. If @impl ... is set once (the setting must be done outside of a quote expression so we need to check the context of the module attribute accodingly), all other callbacks in that module must have @impl declared, otherwise a warning will be emitted.

When a module name is given, Elixir will check the following function is an implementation of a callback in the given behaviour:

@impl GenServer
def handle_call(message, from, state) do
  ...
end

defoverridable with behaviours

While @impl will give more confidence and assistance to developers, it is only useful if developers are defining behaviours for their contracts. Elixir has always advocated that a behaviour must always be defined when a set of functions is marked as overridable but it has never provided any convenience or mechanism to enforce such rules.

Therefore we propose the addition of defoverridable BehaviourName, which will make all of the callbacks in the given behaviour overridable. This will help reduce the duplication between behaviour and defoverridable definitions and push the community towards best practice. Therefore, instead of:

defmodule GenServer do
  defmacro __using__(_) do
    quote do
      @behaviour GenServer
      def init(...) do ... end
      def terminate(..., ...) do ... end
      def code_change(..., ..., ...) do ... end
      defoverridable init: 1, terminate: 2, code_change: 3
    end
  end
end

We propose:

defmodule GenServer do
  defmacro __using__(_) do
    quote do
      @behaviour GenServer
      def init(...) do ... end
      def terminate(..., ...) do ... end
      def code_change(..., ..., ...) do ... end
      defoverridable GenServer
    end
  end
end

By promoting new defoverridable API above, we hope library developers will consistently define behaviours for their overridable functions, also enabling developers to use the @impl true annotation to guarantee the proper callbacks are being implemented.

The existing defoverridable API will continue to work as today and won't be deprecated.

PS: Notice defoverridable always comes after the function definitions, currently and as well as in this proposal. This is required because Elixir functions have multiple clauses and if the defoverridable came before, we would be unable to know in some cases when the overridable function definition ends and when the user overriding starts. By having defoverridable at the end, this boundary is explicit.

[samsondav]

I started looking into this, but I wonder if this @impl is compatible with the existing @impl we already use for Protocols: https://github.com/elixir-lang/elixir/blame/master/lib/elixir/lib/protocol.ex#L573

[josevalim]

Yes, we will probably have to rename that variable to something else.

[samsondav]

@josevalim you mean rename the existing protocol @impl? May I suggest we use a new name instead? Perhaps something like:

@override true
def handle_call(message, state) do
  ...
end

@override GenServer
def handle_call(message, from, state) do
  ...
end

This seems clearer to me. It's naming is consistent with defoverridable and it avoids having to change unrelated existing code.

[josevalim]

@samphilipd rename the existing @impl in defprotocol. @overrides doesn't make much sense because you can use @impl GenServer without overriding any callback. It is about behaviours and not defoverridable.