Abstract def implementations should inherit return type restrictions by default

[straight-shoota]

The compiler enforces abstract def implementations to specify a return type restriction if the abstract def has one as well:

module Foo
  abstract def next : Int32
end
 
class Bar
  include Foo
  
  def next # Error: this method overrides Foo#next() which has an explicit return type of Int32.
    1
  end
end
 
Foo.new.next

I think implementations could by default inherit the restriction of the abstract def. This makes it easier to write abstract def implementations, and in particular allows introducing return type restrictions to existing abstract defs without breaking all implementations (see #10903).

[Blacksmoke16]

The compiler should probably also see that it doesn't have a return type and inherit that from the abstract def when generating docs for that method.

[HertzDevil]

As a general reminder, a def might implement more than one abstract def at the same time:

abstract class A
  abstract def foo(x, y : Int32) : Enumerable(Int32)
  abstract def foo(x : Int32, y) : Iterable(Int32)
end

class B < A
  def foo(x, y) : Array(Int32) # okay
    [1]
  end
end

So if we omit the return type restriction in B#foo but put the deduced type in the docs, it will be a huge union of every generic instance that includes both Enumerable(Int32) and Iterable(Int32).

[straight-shoota]

@HertzDevil Implementing different abstract defs with a single method is probably very rare. Do you know any example for that?
I'd be fine with having ugly autogenerated return type documentation for that. The fix is easy: Just add an explicit restriction to the implementing method.

[HertzDevil]

Another reason for enforcing return type restrictions in implementations and not inheriting them from the abstract def(s) is here: #7956 (comment)

[HertzDevil]

The problem with autogenerated return types in docs is that they only cover the existing subtypes (possibly generic instantiations) coming from the library itself, so if an intersection expands into the union of its subtypes, it could as well be a single type or even NoReturn, both of which are very misleading as surely user code can add new subtypes.

If the solution is to add a return type restriction then there is little point in allowing them to be inherited in the first place. So I disagree allowing docs to automatically generate return type restrictions based on overridden abstract defs, unless #2404 is implemented.

[straight-shoota]

I really don't think docs is very relevant here. The important thing is to make sure the implementation uses the correct return type as per the abstract def contract.

[straight-shoota]

If the solution is to add a return type restriction then there is little point in allowing them to be inherited in the first place.

We can't just start enforcing return type restrictions on implementation methods. That's a breaking change and needs to wait for 2.0 (We could add deprecation warnings before that, though). And I think that should be the ultimate goal we're aiming for.

I would consider automatically inheriting return type restrictions from the abstract def as a useful short-term action. If an implementation has an incorrect implicit return type, that's a bug and the compiler can discover that. That's not a breaking change because it only fails to compile code that was already broken anyways, it just makes it obvious and helps identify bugs. This can happen way before we can make return type restrictions mandatory in 2.0.

[HertzDevil]

So are you asking to reapply #7999 and #9810 within every major release? Because that's what a "short-term action" sounds like to me. If later we discover during 2.x that some abstract defs in the standard library have incorrect restrictions, we would repeat this procedure until 3.0 drops. I don't think that's a healthy process.

Instead, here are two ways to add return type restrictions to abstract defs in the middle of a major release. The first one is non-binding:

module Iterator(T)
  {% begin %}
    # New in 1.2: This method must return a `T | Iterator::Stop`.
    abstract def next {% if compare_versions(::Crystal::VERSION, "2.0.0") >= 0 %} : T | Iterator::Stop {% end %}
  {% end %}
end

The second is to create a new annotation for this:

module Iterator(T)
  @[NonStrict(:abstract_def_return_type, "1.2")]
  abatract def next : T | Iterator::Stop
end

such that if Crystal::VERSION ~> "1.2", then failure to specify the return type restriction will be met with a warning instead of a hard error. With this I don't think the benefits of relaxing the return type requirement on implementations outweigh the problems it creates. (This is also applicable to a lot more compiler features and even user shards where semantic versioning must be honored.)

[straight-shoota]

Yeah, adding a return type conditionally or disabling a strict error might actually be a better idea than implicit inheritance 👍

[straight-shoota]

Wrapping the return type in a conditional macro looks nice for its simplicity.

But I don't think it's very useful, because it only becomes effective with the specified release. Before that, it's neither visible in the documentation nor can you check if there would be errors once it becomes active.
This could probably be dealt with, but then it loses its simplicity.