Nullness: Annotating wildcards and their bounds

[abreslav]

Extracting this from #19 for better granularity.

What we seem to agree on

• Wildcard bounds can be annotated, as in Foo<? extends @Nullabe Bar>
• There's a parallel with type parameter bounds, and it would be logical to apply the same rules

The current rule for type parameter bounds

In the context of @DefaultNullable/@DefaultNotNull if a type parameter has no explicitly annotated bound, its bound is considered to be annotated according to the specified default.

Questions

• Is each of the following forms allowed and if yes, what does it mean: Foo<@NotNull ?>, Foo<@NotNull ? extends @Nullable Bar>, Foo<@NotNull ? super @Nullable Bar>
• Does the default specified by @DefaultNullable/@DefaultNotNull apply to the wildcard itself? (If we give that any meaning while answering the previous question)
• Does the default apply to explicit bounds if they are not annotated? E.g. does Foo<? extends Bar> become effectively Foo<? extends @NotNull Bar> when in the scope of @DefaultNotNull?

Unbounded wildcard (?)

It seems that we might need to allow annotating an unbounded wildcard Foo<@NotNull ?> because its bound is not always denotable, e.g. for the case of F-bounded recursive types, e.g.:

interface C<T extends C<T>> {
    T get();
}

It looks like C<?> and C<? extends Object> may not mean the same thing here, but in fact JLS §4.5.1 says this:

The wildcard ? extends Object is equivalent to the unbounded wildcard ?.

And the following code compiles correctly:

    void test(C<?> unbounded, C<? extends Object> bounded) {
        unbounded.get().get();
        bounded.get().get();
    }

The bounds from the declaration site are implicitly applied to the wildcard even if it declares only Object as its explicit bound, but then it's hard to tell how an annotation on the Object should interact with those implicitly applied bounds. So, for now, we have it as an open question.

Bounded wildcards

A wildcard type in Java can specify either an upper bound (? extends Foo) or a lower bound (? super Foo). The bounds are normal type usages, so they can be annotated explicitly as @NotNull or @Nullable.

By analogy with type parameter bounds, it would make sense to apply defaults to unannotated wildcard bounds. E.g. Foo<? extends Bar> becomes effectively Foo<? extends @NotNull Bar> when in the scope of @DefaultNotNull.

"Write-only list of not-null Foo"

The intuition for List<? super Foo> is roughly "a write-only list of Foo". One may want to say something like "write-only list of not-null Foo", or "a list where you can add only not-null Foo's".

Observation: List<? super @NotNull Foo> does not capture this intent because it can be assigned List<@Nullable Foo> (@Nullable Foo is a supertype of @NotNull Foo).

To express this intent, one could use the fact that each wildcard implicitly has two bounds, so that ? super Foo means actually "lower bound Foo, upper bound Object(and? extends Foois "lower bounds Bottom, upper bound Foo", where Bottom is the subtype of all types, i.e. the empty type, like Kotlin'sNothing`).

So, the "write-only list where you can only add not-null Foo's" would be a list of "lower bound @NotNull Foo upper bound @NotNull Object). We could adopt a convention that this can be expressed as List<@NotNull ? super @NotNull Foo> where the annotation of the ? applies to the bound that is not explicit.

Some more examples of this convention:

Supertype	List<@NotNull Foo>	List<@Nullable Foo>
List<@NotNull ? super @NotNull Foo>	subtype	not a subtype
List<@Nullable ? super @NotNull Foo>	subtype	subtype
List<@Nullable ? super @Nullable Foo>	not a subtype	subtype
List<@NotNull ? super @Nullable Foo> *	not a subtype	subtype
--	--	--
List<@NotNull ? extends @NotNull Foo>	subtype	not a subtype
List<@Nullable ? extends @NotNull Foo> *	not a subtype	subtype
List<@Nullable ? extends @Nullable Foo>	not a subtype	subtype
List<@NotNull ? extends @Nullable Foo>	subtype	subtype

* Inconsistent bounds

Notes on the related discussions are available here.

[kevinb9n]

What we seem to agree on

• Wildcard bounds can be annotated, as in Foo<? extends @Nullabe Bar>
• There's a parallel with type parameter bounds, and it would be logical to apply the same rules

Clearly the parallel is strong for extends wildcards, but as we are discovering, super wildcards may be something quite special. [EDIT: no, by the time I finished this post I'm back to thinking they may be fairly normal-ish.]

The current rule for type parameter bounds

In the context of @DefaultNullable/@DefaultNotNull if a type parameter has no explicitly annotated bound, its bound is considered to be annotated according to the specified default.

I am pretty convinced this is the right answer.

It seems that we might need to allow annotating an unbounded wildcard Foo<@NotNull ?> because its bound is not always denotable, e.g. for the case of F-bounded recursive types, e.g.:

I said in our meeting that this means we "have to" do something, but now I'm really scratching my head as to how/why this problem will come up in real life, and would kinda like to wait until it does.

It looks like C<?> and C<? extends Object> may not mean the same thing here, but in fact JLS §4.5.1 says this:

The wildcard ? extends Object is equivalent to the unbounded wildcard ?.

And the following code compiles correctly:

Well, I learned something today. I'm not sure it used to be like this. But I guess it makes sense that there's no harm in referring to it by a more general type while in reality it will always be more specific.

The bounds from the declaration site are implicitly applied to the wildcard even if it declares only Object as its explicit bound, but then it's hard to tell how an annotation on the Object should interact with those implicitly applied bounds. So, for now, we have it as an open question.

Presumably we should intersect them.

Bounded wildcards

A wildcard type in Java can specify either an upper bound (? extends Foo) or a lower bound (? super Foo). The bounds are normal type usages, so they can be annotated explicitly as @NotNull or @Nullable.

As far as I can see, this all makes sense. Ignoring legacy nullness for the time being:

Supplier<? extends @Nullable Foo> sn
Supplier<? extends @NotNull Foo> snn
Consumer<? super @Nullable Foo> cn
Consumer<? super @NotNull Foo> cnn

Now (note these are slightly out of order):

• snn is guaranteed not to produce null.
• sn makes no such guarantee.
• cn promises to accept null if I want to pass that.
• cnn makes no such promise.

The sn and cn behaviors do seem what I should expect if I used @DefaultNullable, and so on with snn/cnn and @DefaultNotNull. As for invariant types like List, if I put a wildcard on it at all, it expresses my intention to use it only as a producer (extends) or a consumer (super), and so all the preceding should be good enough for it as well. And if I don't want to use a wildcard, I should get either the permission to insert nulls (List<@Nullable Foo>) or the freedom from having to get them (List<@NotNull Foo>) and it all seems to work out.

By analogy with type parameter bounds, it would make sense to apply defaults to unannotated wildcard bounds.

I've convinced myself to agree.

"Write-only list of not-null Foo"

The intuition for List<? super Foo> is roughly "a write-only list of Foo". One may want to say something like "write-only list of not-null Foo", or "a list where you can add only not-null Foo's".

"I need you to give me something that can't do X" is a negative contract, and as such, I don't think we need to worry about it. I am capable of ensuring I don't add nulls to the list whether you pass me one that rejects nulls or not.

I would be very happy if we do NOT need to support constraining both bounds at once. There hasn't been a way to do that in base Java types (mostly) and it's been just fine.

[wmdietlGC]

By analogy with type parameter bounds, it would make sense to apply defaults to unannotated wildcard bounds.

...

Consumer<? super @Nullable Foo> cn
cn promises to accept null if I want to pass that.

If we are in a @DefaultNotNull setting, this means that the implicit upper bound of the wildcard in cn is @NotNull Object.
So this would result in an inconsistent type, where the upper bound is not a supertype of the lower bound.

Therefore, we would need to write

Consumer<@Nullable ? super @Nullable Foo> cn

or think about different rules.

[kevinb9n]

Consumer<? super @Nullable Foo> cn
cn promises to accept null if I want to pass that.

If we are in a @DefaultNotNull setting, this means that the implicit upper bound of the wildcard in cn is @NotNull Object.

Oh. That's not what I assumed it would mean. My perspective is that an upper bound is not present and thus there is nothing to default. The lower bound is present and it is what would default to @NotNull, but in the example it is given explicitly so the default never applies.

Applying <? super> to a type establishes that that type is being used as a consumer and will not be sanely useful in any other way (it will just produce Objects...). As a consumer, I don't believe it makes sense to want to restrict its upper bound. Programmers don't typically complain "I need this NOT to accept x"; they can simply not give it x. Problems only arise from trying to use the same type for both producing and consuming but this seems like "doctor, it hurts when I do this".

(aside: I have claimed that Java non-invariant types have either a lower or upper bound but not both, however, I do realize that a class Foo<T extends Number> can have a Foo<? super Integer> usage. All right, at least the bounds came from different places and each can be annotated. We can make sure that the sensible combinations work. I still think it will break people's heads to annotate each bound in the same single type usage.)

[abreslav]

As for invariant types like List, if I put a wildcard on it at all, it expresses my intention to use it only as a producer (extends) or a consumer (super), and so all the preceding should be good enough for it as well.

With the slight caveat in the case of ? super Foo. I can still call get() on a List<? super @Nullable Foo>, and the nullability of the result of this call can, in theory, be specified in the (implicit) upper bound. This does not apply to ? extends Foo, because set()/add() can not be called on it.

[abreslav]

After reading the comments above more carefully I propose that we treat the implicit upper bounds of ? super Foo as

• preferably always nullable (it can be @Nullable Object or @Nullable Bar if the corresponding parameter is declared as T extends Bar)
• if we absolutely have to, always legacy

And thus not allow annotations on unbounded ? at first, maybe adding this support in a later revision of the spec

[kevinb9n]

(True. This will happen, but my contention is that it's irregular enough that it's not worth contorting our design for. That's a very subtle judgment call, though.)

[abreslav]

Resolution from the meeting:

• unannotated explicit bounds always get defaults applied (for wildcards and type parameters alike)
• the implicit upper bound of an unbounded wildcard ? or type parameter that has no explicit bounds (<T>) always get defaults applied to it
• implicit bounds of bounded wildcards (? extends Foo, ? super Foo) never get defaults applied

[stephan-herrmann]

the implicit upper bound of an unbounded wildcard ? or type parameter that has no explicit bounds (<T>) always get defaults applied to it

If that's really type parameter, not type argument, I wonder how - in the context of nonnull default - a type with generic nullness should be declared? If all type parameters are implicitly annotated clients can no longer choose during instantiation of the generic type.

[wmdietlGC]

If the upper bound is @Nullable, it is parametric.
So if you have:

class C<T extends @Nullable Object> {
  T get() {...}
}

The result of get is parametric, so on C<@NotNull String> is @NotNull String.
It doesn't matter whether the @Nullable on the upper bound comes from a default or an explicit annotation.

[stephan-herrmann]

OK, this brings us to the question, what is expected to be more common / should be encouraged:

• type parametric, nullness fixed, or
• type parametric, nullness parametric.

For me, the latter seems more natural, and so the former should get the burden of additional verbosity, not the latter. I don't have the empirical data to back this, though.

[wmdietlGC]

I think the relevant discussion for your last point is here: #12 (comment)

[dzharkov]

I would like to add some possible clarifications to the current proposal about wildcards and nullability.

(Further, I assume that List and Consumer have type parameters annotated as parametric and Optional has a type parameter with not-nullable upper bound)

First of all, I'm totally in favor of the perception of wildcards as existential types.
In other words, for List<? extends CharSequence> and Consumer<? super CharSequence>
we have in mind a conceptual type X for which we know only that [VoidType] <: X <: CharSequence and a type Y for which we have CharSequence <: Y <: Object. And resulting types List<X> and Consumer<Y> that should be treated respectfully to the information coming from the bounds of X and Y.

Meaning that we don't have any information besides their bounds.

When considering nullability specifiers, I would suggest that they are only applicable to the bounds not to their occurrences of the existential type themselves. I.e., for any nullability annotations combination we should not treat any type as List<@NotNull X>, but we can treat X as having not-nullable upper bound (X <: @NotNull CharSequence) that would allow us, for example, to retrieve not-nullable char-sequences from list.get(0). The same applies to super wildcards: Consumer<? super @NotNull CharSequence> would mean that @NotNull CharSequence <: Y thus c.accept only allows not-nullable values.

Nullability from type parameter's bounds, in this case, might be just intersected with nullability of existential type's bounds.

So, the relevant piece from the proposal may be reworded as follows:

• For the wildcarded type argument, we need to define nullability for the bounds of the corresponding existential type X given an explicit/implicit wildcard's bound L and U is the intersection of any effective specifiers on inherited bounds.
• If the wildcard has an explicit “super” bound, then:
  • X will always have the same nullability (and types) for its upper bounds as the bounds from the corresponding type parameter
  • Nullability for lower bound of X is defined as intersection (picking the most specific) of U and L.
  • if L has a weaker nullability explicitly set by the annotation on type usage, tools may report a warning.
• Otherwise, the wildcard has an explicit or implicit "extends" bound:
  • Then lower bounds for X will always be a [VoidType] (a special type with null being an only legal instantiation) and its nullability is the same as nullability of U.
  • Nullability for the upper bound of X is defined as intersection (picking the most specific) of U and L.
  • if L has a weaker nullability explicitly set by the annotation on type usage, tools may report a warning.

When typechecking wildcarded types, tools should use lower or upper bound of the relevant existential type depending on position:

• Upper-bound when it's used in covariant position. E.g., return type of List<? extends @NotNull CharSequence>::get or when List<? extends CharSequence> used as an expected type and puts there List<@NotNull String>.
• Lower-bound when it's used in contravariant position. E.g, parameter type of Consumer<? super CharSequence>::accept or when Consumer<? super CharSequence> used as an expected type and puts there Consumer<@Nullable Object>

[kevin1e100]

Thanks, I like the idea of making X more explicit. I'm a little confused by this though:

Then lower bounds for X will always be a [VoidType] (a special type with null being an only legal instantiation) and its nullability is the same as nullability of U.

(which is in the context of a ? or ? extends wildcard I believe). It seems like the lower bound in that case is always @NotNull (since there is no explicit lower bound), am I missing something?

[dzharkov]

(which is in the context of a ? or ? extends wildcard I believe)

Yes, you're right. I've fixed the description.
Thanks!

It seems like the lower bound in that case is always @NotNull (since there is no explicit lower bound), am I missing something?

Looks like you're right.
The question is whether we can add null to List<?>.
I would report a warning if a relevant type parameter is annotated somehow besides @NullnessUnknown.

Maybe it's worth discussing it at a meeting.

[kevinb9n]

I am trying to parse this thread. Let me ask this simple question though:

// we are in a not-null-by-default context

void method(
    List<Foo> a,
    List<? extends Foo> b,
    List<? extends Object> c,
    List<? super Foo> d)

I think we are maybe agreeing that the default should be applied to every bound, so that it's as if you wrote:

void method(
    List<@NotNull Foo> a,
    List<? extends @NotNull Foo> b,
    List<? extends @NotNull Object> c,
    List<? super @NotNull Foo> d)

... even though there is a slight oddness here: this makes the element type's nullness fixed for the first three (non-nullable) but happens to make it unconstrained for the last one. I believe that is actually a fine and natural consequence, but it bears pointing out.

Aside from the overarching question of whether we should switch to Stephan's "orthogonal nullness and types" model -- apart from that, is what I've just said considered controversial at this point?

(I am deliberately leaving bare List<?> out of it, for now.)

[stephan-herrmann]

Aside from the overarching question of whether we should switch to Stephan's "orthogonal nullness and types" model -- apart from that, is what I've just said considered controversial at this point?

I only disagree regarding the case of ? extends Object becoming ? extends @NotNull Object (which otherwise would pre-determine how we handle List<?>). The rest is fine by me.

[kevinb9n]

(Last comment is about #83.)

I think Denis's comment #31 (comment) is an accurate depiction of how we are thinking about this now.