Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8315771: [JVMCI] Resolution of bootstrap methods with int[] static arguments #15588

Closed
wants to merge 13 commits into from

Conversation

Zeavee
Copy link
Contributor

@Zeavee Zeavee commented Sep 6, 2023

Currently, jdk.vm.ci.meta.ConstantPool.lookupBootstrapMethodInvocation does not support static arguments of type int[].

Supporting those static arguments allows to correctly lookup the BootstrapMethodInvocation of some InvokeDynamic and DynamicConstant.

To lookup the constant at the index in the static arguments index list, a new class is introduced, allowing to lazily resolve the constant or obtain the constant pool index of the arguments:

    static class CachedBSMArgs extends AbstractList<JavaConstant> {
        private final JavaConstant[] cache;
        private final HotSpotConstantPool cp;
        private final int bssIndex;

        CachedBSMArgs(HotSpotConstantPool cp, int bssIndex, int size) {
            this.cp = cp;
            this.bssIndex = bssIndex;
            this.cache = new JavaConstant[size];
        }

        @Override
        public JavaConstant get(int index) {
            JavaConstant res = cache[index];
            if (res == null) {
                int argCpi = compilerToVM().bootstrapArgumentIndexAt(cp, bssIndex, index);
                res = compilerToVM().lookupConstantInPool(cp, argCpi, false);
                if (res == null) {
                    res = JavaConstant.forInt(argCpi);
                }
                cache[index] = res;
            }
            return res;
        }

        @Override
        public int size() {
            return cache.length;
        }
    }

Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue

Issue

  • JDK-8315771: [JVMCI] Resolution of bootstrap methods with int[] static arguments (Enhancement - P4)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15588/head:pull/15588
$ git checkout pull/15588

Update a local copy of the PR:
$ git checkout pull/15588
$ git pull https://git.openjdk.org/jdk.git pull/15588/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15588

View PR using the GUI difftool:
$ git pr show -t 15588

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15588.diff

Webrev

Link to Webrev Comment

@bridgekeeper
Copy link

bridgekeeper bot commented Sep 6, 2023

👋 Welcome back Zeavee! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@Zeavee Zeavee changed the title [JVMCI] Resolution of bootstrap methods with int[] static arguments 8315771: [JVMCI] Resolution of bootstrap methods with int[] static arguments Sep 6, 2023
@openjdk
Copy link

openjdk bot commented Sep 6, 2023

@Zeavee The following labels will be automatically applied to this pull request:

  • graal
  • hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added graal graal-dev@openjdk.org hotspot-compiler hotspot-compiler-dev@openjdk.org rfr Pull request is ready for review labels Sep 6, 2023
@mlbridge
Copy link

mlbridge bot commented Sep 6, 2023

@@ -170,6 +170,17 @@ interface BootstrapMethodInvocation {
List<JavaConstant> getStaticArguments();
}

/**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very confusing documentation. It's not clear what index or entry you are referring to each time you say "index". Please provide a concrete example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have rewritten the documentation, I hope it is clearer. I also added a second argument to be able to access different different static arguments on the same bootstrap specifier.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It still needs an example showing how this API should be used. Please add a complete (pseudo-code) example to the javadoc of BootstrapMethodInvocation.

/**
* Gets the constant pool index of a static argument of a bootstrap specifier. Used when the list
* of static arguments in the {@link BootstrapMethodInvocation} is an {@code int[]}. The list has
* two elements. The first one is the number of arguments and the second one is the {@code cpi}.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you referring to BootstrapMethodInvocation.getStaticArguments() whose return type is List<JavaConstant>, not int[]?

The list has two elements.

That's not consistent with the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you referring to BootstrapMethodInvocation.getStaticArguments() whose return type is List, not int[]?

It is indeed the list returned by BootstrapMethodInvocation.getStaticArguments(). In this case, we have in fact a List which corresponds to the int[], but it is true it is confusing, I will rephrase it.

That's not consistent with the code.

In this case, the arguments come from here, which are always only two elements.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case

In what case? When introducing new API like this, you need to use precise language that spells out possible values, assumptions, typical usages etc. What you have written is a good start but needs more work.

* of static arguments in the {@link BootstrapMethodInvocation} is an {@code int[]}. The list has
* two elements. The first one is the number of arguments and the second one is the {@code cpi}.
*
* @param cpi the index of a bootstrap specifier in the constant pool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the index of a bootstrap specifier in the constant pool

What's a bootstrap specifier? That is not mentioned anywhere else so you either have to explain it or use existing terminology.

@@ -170,6 +170,17 @@ interface BootstrapMethodInvocation {
List<JavaConstant> getStaticArguments();
}

/**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It still needs an example showing how this API should be used. Please add a complete (pseudo-code) example to the javadoc of BootstrapMethodInvocation.

* {@link BootstrapMethodInvocation} is a {@code List<PrimitiveConstant>} of the form
* {{@code arg_count}, {@code pool_index}}, meaning the arguments are not already resolved and that
* the JDK has to lookup the arguments when they are needed. The {@code cpi} corresponds to
* {@code pool_index} and the {@code index} has to be smaller than {@code arg_count}.
*
* @param cpi the index of a bootstrap specifier in the constant pool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

     * @param cpi the index of a {@code CONSTANT_Dynamic_info} or @{code CONSTANT_InvokeDynamic_info} entry

* argCount = staticArguments.get(0).asInt();
* cpi = staticArguments.get(1).asInt();
* for (int i = 0; i < argCount; ++i) {
* arguments[i] = lookupConstant(cpi, i);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What lookupConstant method is this?
Where is the usage of the new bootstrapArgumentIndexAt method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a mistake sorry, it is now correct.
The idea is to use bootstrapArgumentIndexAt to obtain the constant pool index of the static argument and use it to call ConstantPool.lookupConstant(int cpi, boolean resolve) to get the constant.

* {@code pool_index} and the {@code index} has to be smaller than {@code arg_count}.
*
* @param cpi the index of a {@code CONSTANT_Dynamic_info} or @{code CONSTANT_InvokeDynamic_info} entry
* @param index the index of the static argument in the list of static arguments
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when index is out of bounds?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, it returns another constant pool index or a random number. I can add a check in the CachedBSMArgs.get method to prevent this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please. We should opt for exceptions over undefined behavior whenever possible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After double checking, there is in fact already an assert for checking if the index is out of bounds. Should I change it to an exception?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the assertion in code that can throw exceptions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assertion is here. I don't think we can throw exception there. I could check the size in jvmciCompilerToVM.bootstrapArgumentIndexAt, but it would require a bit of code duplication to obtain it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. You should just add a note in the javadoc that the result is undefined if index is out of bounds. This is all internal API so I assume we can guarantee the value is correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is only used in CachedBSMArgs.get, which will throw an ArrayIndexOutOfBoundsException before we arrive there.

@@ -165,6 +165,23 @@ interface BootstrapMethodInvocation {
/**
* Gets the static arguments with which the bootstrap method will be invoked.
*
* An argument of type {@link PrimitiveConstant} represents a {@code CONSTANT_Dynamic_info}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it not be an index to any valid LDC constant? This is implied by this code in your Graal PR:

if (constant instanceof PrimitiveConstant primitiveConstant) {
    int argCpi = primitiveConstant.asInt();
    Object argConstant = lookupConstant(argCpi, opcode == Opcodes.INVOKEDYNAMIC ? Opcodes.LDC : opcode, allowBootstrapMethodInvocation);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, sorry, I will fix this documentation.

Copy link
Contributor Author

@Zeavee Zeavee Sep 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the user point of view, a PrimitiveConstant can in fact only be a ConstantDynamic, because the other type of constant are automatically resolved in CachedBSMArgs.get. In the Graal PR, lookupConstant is a recursive call, allowing to call the associated bootstrap method, but the other type of entries will go in the else branch.

@@ -526,6 +529,37 @@ private int flags() {
return UNSAFE.getInt(getConstantPoolPointer() + config().constantPoolFlagsOffset);
}

static class CachedBSMArgs extends AbstractList<JavaConstant> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class needs documentation.

@dougxc
Copy link
Member

dougxc commented Sep 8, 2023

@PaulSandoz it would be great if you could have a look at this PR or suggest another domain expert who might be willing to look at it.

@dougxc
Copy link
Member

dougxc commented Sep 8, 2023

@Zeavee please keep the PR description up to date with the current changes.

@PaulSandoz
Copy link
Member

@PaulSandoz it would be great if you could have a look at this PR or suggest another domain expert who might be willing to look at it.

@dougxc i will look at it this week

* {@code pool_index} and the {@code index} has to be smaller than {@code arg_count}.
*
* @param cpi the index of a {@code CONSTANT_Dynamic_info} or @{code CONSTANT_InvokeDynamic_info} entry
* @param index the index of the static argument in the list of static arguments
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please. We should opt for exceptions over undefined behavior whenever possible.

* @return A {@link PrimitiveConstant} representing a {@code CONSTANT_Dynamic_info}
* entry or a {@link JavaConstant} representing the static argument requested
* @return A {@link PrimitiveConstant} representing an unresolved constant pool entry
* or a {@link JavaConstant} representing the static argument requested
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or a {@link JavaConstant}
a PrimitiveConstant is a JavaConstant isn't it?

Copy link
Contributor Author

@Zeavee Zeavee Sep 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a PrimitiveConstant is a JavaConstant isn't it?

Yes, it is, but since the primitives are returned in their boxed form, a PrimitiveConstant can only be a constant pool index, so I think it is better to specify the difference.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point is that a PrimitiveConstant is a subtype of JavaConstant so the "or" does not make sense. It's like saying "an int or a number".

} else if (object instanceof JavaType type) {
res = runtime().getReflection().forObject(type);
} else {
res = JavaConstant.forInt(argCpi);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my understanding and my testing, this can only be reached for CONSTANT_Dynamic constant pool entries, as other types of entries should resolve even with resolve = false. Am I correct or am I missing some edge cases?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's right based on the spec for ConstantPool.lookupConstant.

@dougxc
Copy link
Member

dougxc commented Sep 14, 2023

Please update test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.hotspot.test/src/jdk/vm/ci/hotspot/test/TestDynamicConstant.java to test the new functionality.

* entry. To resolve this entry, the corresponding bootstrap method has to be called first:
*
* <pre>
* lookupConstant(int index, boolean resolve) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example should focus on resolving the static arguments:

List<JavaConstant> args = cp.getStaticArguments();
List<JavaConstant> resolvedArgs = new ArrayList<>(args.size());
for (JavaConstant c : args) {
    JavaConstant r = c;
    if (c instanceof PrimitiveConstant pc) {
        r = ...
    }
    resolvedArgs.append(rc);
}

It should also use real Java code to remove all ambiguity.

@PaulSandoz
Copy link
Member

Some general comments:

Copy link
Member

@dougxc dougxc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me now.

@openjdk
Copy link

openjdk bot commented Sep 19, 2023

@Zeavee This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8315771: [JVMCI] Resolution of bootstrap methods with int[] static arguments

Reviewed-by: dnsimon, psandoz

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 232 new commits pushed to the master branch:

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@dougxc, @PaulSandoz) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Sep 19, 2023
@Zeavee
Copy link
Contributor Author

Zeavee commented Sep 19, 2023

/integrate

@openjdk openjdk bot added the sponsor Pull request is ready to be sponsored label Sep 19, 2023
@openjdk
Copy link

openjdk bot commented Sep 19, 2023

@Zeavee
Your change (at version ea4aa98) is now ready to be sponsored by a Committer.

@dougxc
Copy link
Member

dougxc commented Sep 19, 2023

This PR still needs another review given that it's not trivial. Maybe @TobiHartmann or @tkrodriguez can help out.

@PaulSandoz
Copy link
Member

This generally looks good to me. I was initially confused about the reuse (or overload?) of PrimitiveConstant for a static argument that is the constant pool index to a CONSTANT_Dynamic_info entry, from which it can be resolved or it's BootstrapMethodInvocation obtained.

It's very subtle, and I wondering if using a new type would be better and more clearly distinguish between a primitive constant and an index in the constant pool. However, i am not an expert in JVMCI so do not know what repercussions that might have. It's as if you want to expose a lazily resolved constant.

@dougxc
Copy link
Member

dougxc commented Sep 20, 2023

While it's tempting to introduce a new type to distinguish between a primitive constant and an index in the constant pool, it would involve changing API that is already used in Graal.
What's more, we're (ab)using the fact that java.lang.invoke.ConstantGroup.get(int index) always returns a boxed value. Boxed Java values in JVMCI are always represented by a JavaConstant for which c.getJavaKind() == JavaKind.Object. So we can use PrimitiveConstant here as a side channel to represent a constant pool index.

@Zeavee can you please update the javadoc of BootstrapMethodInvocation.getStaticArguments() one more time to make this crystal clear:

        /**
         * Gets the static arguments with which the bootstrap method will be invoked.
         *
         * The {@linkplain JavaConstant#getJavaKind kind} of each argument will be
         * {@link JavaKind#Object} or {@link JavaKind#Int}. The latter represents an
         * unresolved {@code CONSTANT_Dynamic_info} entry. To resolve this entry, the
         * corresponding bootstrap method has to be called first:
         *
         * <pre>
         * List<JavaConstant> args = bmi.getStaticArguments();
         * List<JavaConstant> resolvedArgs = new ArrayList<>(args.size());
         * for (JavaConstant c : args) {
         *     JavaConstant r = c;
         *     if (c.getJavaKind() == JavaKind.Int) {
         *         // If needed, access corresponding BootstrapMethodInvocation using
         *         // cp.lookupBootstrapMethodInvocation(pc.asInt(), -1)
         *         r = cp.lookupConstant(c.asInt(), true);
         *     } else {
         *         assert c.getJavaKind() == JavaKind.Object;
         *     }
         *     resolvedArgs.append(r);
         * }
         * </pre>

No need to re-run testing after this change as it's only a doc change.

@openjdk openjdk bot removed the sponsor Pull request is ready to be sponsored label Sep 20, 2023
Copy link
Member

@dougxc dougxc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still looks good to me.

Copy link
Member

@PaulSandoz PaulSandoz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I suppose you could generalize the test a little more to iterate over the static arguments, independent of the condyType. That exercises the code a little more esp. when running under the -XX:UseBootstrapCallInfo=3 argument. Up to you.

@Zeavee
Copy link
Contributor Author

Zeavee commented Sep 21, 2023

/integrate

@dougxc
Copy link
Member

dougxc commented Sep 21, 2023

/sponsor

@openjdk openjdk bot added the sponsor Pull request is ready to be sponsored label Sep 21, 2023
@openjdk
Copy link

openjdk bot commented Sep 21, 2023

@Zeavee
Your change (at version 7a8d9f2) is now ready to be sponsored by a Committer.

@openjdk
Copy link

openjdk bot commented Sep 21, 2023

Going to push as commit 015f6f5.
Since your change was applied there have been 232 commits pushed to the master branch:

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Sep 21, 2023
@openjdk openjdk bot closed this Sep 21, 2023
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review sponsor Pull request is ready to be sponsored labels Sep 21, 2023
@openjdk
Copy link

openjdk bot commented Sep 21, 2023

@dougxc @Zeavee Pushed as commit 015f6f5.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@@ -32,6 +32,9 @@
* @run testng/othervm
* -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -XX:-UseJVMCICompiler
* jdk.vm.ci.hotspot.test.TestDynamicConstant
* @run testng/othervm
* -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -XX:-UseJVMCICompiler -XX:UseBootstrapCallInfo=3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error: VM option 'UseBootstrapCallInfo' is diagnostic and must be enabled via -XX:+UnlockDiagnosticVMOptions.

New bug being filed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
graal graal-dev@openjdk.org hotspot-compiler hotspot-compiler-dev@openjdk.org integrated Pull request has been integrated
Development

Successfully merging this pull request may close these issues.

4 participants