Make "PrimitiveParameterizedClass.default" a poly expression.

[jespersm]

Make .default a separate node type in the parser.

Issue

JDK-8211914: [lworld] Javac should support type inference for default value creation

Note: The Linux x86 builds in GitHub actions seem to fail with something completely unrelated to these changes.

---

Progress

• Change must not contain extraneous whitespace

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/valhalla pull/369/head:pull/369
$ git checkout pull/369

Update a local copy of the PR:
$ git checkout pull/369
$ git pull https://git.openjdk.java.net/valhalla pull/369/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 369

View PR using the GUI difftool:
$ git pr show -t 369

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/valhalla/pull/369.diff

[bridgekeeper]

👋 Welcome back jespersm! A progress list of the required criteria for merging this PR into lworld will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@jespersm This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

Make "PrimitiveParameterizedClass.default" a poly expression.

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 20 new commits pushed to the lworld branch:

• ad0491e: 8271017: [lworld] Implement withfield changes from JDK-8269408 on AArch64
• 8c11290: 8271154: [lworld] CDS LoaderConstraintsTest fails after injection of IdentityObject
• a4066d7: 8244231: [lworld] Add support for ref-default and val-default inline classes.
• 02584ab: 8269756: [lworld] Add tests for invalid withfield operands
• 01cab82: 8271141: [lworld] Remove unused jvaluetype q member in jvalue
• 64ad8d0: 8271113: [lworld] java/lang/constant/MethodHandleDescTest.java fails after JDK-8247376
• 6bc5ea9: 8247376: [lworld] Constant API support for primitive classes
• 4d42fe1: 8271025: [lworld] vmTestbase/jit/t/* tests fail after JDK-8237073
• 31d3bb0: 8269408: [lworld] Withfield and field resolution update
• b45994a: 8268946: [lworld] Re-implement JDK-8266015 after the merge
• ... and 10 more: https://git.openjdk.java.net/valhalla/compare/b3b7fb2bf96b40fd4636a699e150b8ab5c2d8610...lworld

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@mcimadamore) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

[mlbridge]

Webrevs

• 04: Full - Incremental (f4073c40)
• 03: Full - Incremental (131b9c7d)
• 02: Full - Incremental (0c088269)
• 01: Full (e9289cfa)
• 00: Full (0f969f6)

[mcimadamore]

Nice work - it's a very nice cleanup of the code base! I like how the code that was previously scattered in several places now can just move where it belongs.

Functionality-wise, I'm not sure this supports poly expressions yet - e.g. I see that you set the polyKind tag correctly, but I'm afraid that this along is not sufficient to support stuff like:

List<String> ls = List.default;

or:

void m(List<String> ls) { ... }

m(List.default)

I think you might be led to believe that the current impl is doing the right thing - but my feeling is that javac might be tricking you: if the expected type is e.g. List<String> but the qualifier of the default expression is List, the compiler will check (as per checkId) that List <: List - which is true by subtyping conversion. I don't think there is any inference going on, in other words. You might try debugging and looking at the type of the expression that comes out of checkId.

Anyway, even if this works and I did miss something, I don't see how this approach can scale to default expression passed as method arguments - given that in that case there is no pt(). To add support for that you need to add another case/node in ArgumentAttr (see what's been done for anonymous inner classes - this should be simpler as there's no constructor call).

In other words, you want Attr.visitApply to treat your default expression method argument in a special way - by creating what we call a "deferred type" - that is a type that is not fully inferred until after overload resolution (when you DO know the pt()).

One possibility would be to push this change as is - as the refactoring part is really good - and maybe issue raw type warning when generic types are involved, or when default expressions are used in method context. And then in another, separate PR we can refine what happens in truly poly cases.

[mcimadamore]

So, I debugged what happens in case where we have

List<String> ls = List.default

What I missed is that we create a variable symbol for default which has the right type (from the expected type) - so the expression is correctly typed as List<String>. But this seems to happen "just because" the compiler detects that the expected type has same base class as the default expression type. Then, the check performed by checkId is always vacuously satisfied, given that we're passing in a synthetic variable whose type is the same as the expected one.

Counter example:

interface Foo<X extends Number> extends List<X> { }
...
List<String> ls = Foo.default; // still ok, type is `Foo`

This example is more complex, for two reasons:

• in this case, Foo is a sub-interface, so the trick of saying "if base class is the same just use the expected type" doesn't work. In fact the compiler types this with Foo - which is a raw type
• in this particular instance, the code should not even be allowed - because the constraint on the Foo type parameter are incompatible with those of the expected type List (X must extend Number).

All this stuff needs to work correctly if we want to claim that default expressions are true poly expressions.

[mlbridge]

Mailing list message from Jesper Steen M��ller on valhalla-dev:

On 19 Mar 2021, at 13.28, Maurizio Cimadamore <mcimadamore at openjdk.java.net> wrote:

On Fri, 19 Mar 2021 10:57:22 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org <mailto:mcimadamore at openjdk.org>> wrote:

One possibility would be to push this change as is - as the refactoring part is really good - and maybe issue raw type warning when generic types are involved, or when default expressions are used in method context. And then in another, separate PR we can refine what happens in truly poly cases.

Yes, that might make sense. I?ll split the PR up into the two cases.

So, I debugged what happens in case where we have

`List<String> ls = List.default`

What I missed is that we create a variable symbol for `default` which has the right type (from the expected type) - so the expression is correctly typed as `List<String>`. But this seems to happen "just because" the compiler detects that the expected type has same base class as the default expression type. Then, the check performed by `checkId` is always vacuously satisfied, given that we're passing in a synthetic variable whose type is the same as the expected one.

Counter example:

interface Foo<X extends Number> extends List<X> { }
...
List<String> ls = Foo.default; // still ok, type is `Foo`

This example is more complex, for two reasons:

* in this case, Foo is a sub-interface, so the trick of saying "if base class is the same just use the expected type" doesn't work. In fact the compiler types this with `Foo` - which is a raw type
* in this particular instance, the code should not even be allowed - because the constraint on the `Foo` type parameter are incompatible with those of the expected type List<String> (X must extend Number).

All this stuff needs to work correctly if we want to claim that default expressions are true poly expressions.

I get it; this needs to be checked before blindly promoting to the expected type.
Thank you for the good counter example.

As for method overload, I?ve come up with this example, which seems to demonstrate the need for the "full poly treatment" using deferred types.

class Whatever {

static int foo\(Consumer\<Integer> c\) \{ return 1\; \}
static String foo\(BiConsumer\<Integer\, Integer> c\) \{ return \"x\"\; \}

static primitive class Quux\<T extends Number> implements Consumer\<Integer>\, BiConsumer\<T\, T> \{
	public void accept\(T t\, T u\) \{ \}
	public void accept\(Integer t\) \{ \}
\}

public static void ambiguous\(\) \{
	var result \= foo\(Quux\.default\)\; \/\/ foo here is ambiguous\, just like non\-primitive new Quux\<>\(\) is
\}

}

-Jesper

[jespersm]

@mcimadamore : I've opened https://bugs.openjdk.java.net/browse/JDK-8263900 for the refactoring alone. I'm not totally confident that all the checks in Attr are syntactically possible, and should likely be turned into assertions.

So this PR is temporarily on hold, awaiting merge of #370

[jespersm]

/test tier1

[openjdk]

@jespersm you need to get approval to run the tests in tier1 for commits up until e9289cf

[sadayapalam]

Hello Jesper, this latest patch is problematic. it does not properly treat Foo.default as a poly expression. As Maurizio's earlier review comments points out:

I don't see how this approach can scale to default expression passed as method arguments - given that in that case there is no pt(). To add support for that you need to add another case/node in ArgumentAttr (see what's been done for anonymous inner classes - this should be simpler as there's no constructor call).

In other words, you want Attr.visitApply to treat your default expression method argument in a special way - by creating what we call a "deferred type" - that is a type that is not fully inferred until after overload resolution (when you DO know the pt()).

I don't see the relevant code changes in your change set - i.e there is no changes in ArgumentAttr {} - I would have expected to see a visitDefaultValue method there.

As Maurizio further pointed out earlier, in Attr.visitApply, right after arguments are attributed, the type of Foo.default as a method argument should be a deferred type . If I debug the uploaded patch, the type of Foo.default is actually Foo<T>

The problem is masked by there being only negative tests in your patch.

If I compile the following:

public primitive class X<T> {

    
    static void m(X<String> ls) { }

    public static void main(String [] args) {
        m(new X<>()); // OK
        m(X<String>.default); // OK.
        m(X.default);   // OK, BUT get an error here!
    }

}

at the line with the .default usage I get:

 error: incompatible types: X<T> cannot be converted to X<String>
        m(X.default);
           ^
  where T is a type-variable:
    T extends Object declared in class X
Note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output
1 error

This shows that there is no real inference happening there.

(when a poly type argument is attributed without a target type in context, it should be typed to be DeferredType, when considered against overload candidates the positional parameter type becomes the pt() and the deferred type can now be typed to a concrete type)

[sadayapalam]

A way to see this in more detail is to debug through the Attr.visitApply call corresponding to the method invocation

m(new X<>(), X.default);

in this test case:

public primitive class X<T> {

    
    static void m(X<String> ls, X<String> ls2) { }

    public static void main(String [] args) {
        m(new X<>(), X.default);
    }

}

You will see that first argument is typed to be DeferredType and second one incorrectly to be X<T>

[jespersm]

Hey @sadayapalam and @mcimadamore, I think this patch is ready for review again.

I've not merged lworld (to keep a clearer history), but it still applies cleanly to the current lworld tip, so it should merge OK.

[sadayapalam]

The return statement is not reachable and so the block is not useful. This is because, Foo<>.default is a syntax error. Given a primitive class Foo we will only support Foo.default involving implicit diamond like inference and will not support Foo<>.default which is what this code is handling.

[sadayapalam]

(I think I meant to say return applyTree.arguments.isEmpty(); will never be true.)

[sadayapalam]

This method is problematic - This will always treat Foo.default as a poly expression in an invocation context even when the concerned primitive type is not a generic class. For instance the Foo.default expression in the following snippet becomes a poly expression when it is a standalone expression in reality:

primitive class X {

    static void foo(Object o) {
    }

    public static void main(String [] args) {
        foo(X.default);    
    }

}

[sadayapalam]

I think there is a copy + paste problem. We can never see a NEWCLASS here. The LHS of .default is just a type node and cannot be an rvalue expression ==> new X().default is illegal.

[sadayapalam]

I think perhaps what we really need to do here is to attribute the clazz node of JCDefaultValue first - perhaps by first making a copy of the node and then see if it is a generic class but lacks type arguments. Such as this:

@Override
    public void visitDefaultValue(JCDefaultValue that) {
        JCExpression copy = new TreeCopier<Void>(attr.make).copy(that.clazz);
        attr.attribType(copy, env);
        if (copy.type.getTypeArguments().isEmpty() && copy.type.tsym.type.isParameterized()) {
            processArg(that, speculativeTree -> new ResolvedDefaultType(that, env, speculativeTree));
        } else {
            //not a poly expression, just call Attr
            setResult(that, attr.attribTree(that, env, attr.unknownExprInfo));
        }
    }

[sadayapalam]

I am surprised that ResolvedDefaultType extends ResolvedMemberType rather than ArgumentType directly. While the Inference behavior could be indirectly modelled it as if the class has a generic factory method named 'default':

<T> static Foo<T> default() { ... }

this implementation detail could be pushed deep down and not manifest here and in elsewhere (setPolyKind())

[sadayapalam]

I don't see a test that triggers this diagnostic.

[sadayapalam]

This is surprising. I would expected polyKind to be set in a similar fashion to switch and conditional expressions.

[sadayapalam]

Hi Jesper, I have left some comments - but I will need to make a couple of more passes to fully understand the implementation as this is a fairly involved and complex task. I will actually have to tinker with the version you have posted to comprehend it - Could you give me some time - I will play around with it and propose a version that addresses the problems I have outlined and also includes further tweaks that could simplify the implementation more ? Then we can study it, finalize it and then ask Maurizio for a final review once both of us are satisfied.

[jespersm]

@sadayapalam : I can take another go at this one to adress the review comments above, which may make a second review pass easier.

[sadayapalam]

@sadayapalam : I can take another go at this one to adress the review comments above, which may make a second review pass easier.

Sure, this is recognized to be a tricky area, so it is normal to have iterate over it multiple times incrementally improving it. As long as this is had in mind, by all means it helps to have the comments made so far addressed. Thanks!