8188055: (ref) Add Reference::refersTo predicate #498
Conversation
kimbarrett
changed the title
|
👋 Welcome back kbarrett! A progress list of the required criteria for merging this PR into |
kimbarrett
changed the title
|
@kimbarrett usage: |
|
@kimbarrett The |
Webrevs
|
|
|
||
| // For some collectors, calling get() will keep testObject4 alive. | ||
| // For example, SATB collectors or ZGC, but not collectors using | ||
| // incremental update barriers like CMS. |
There was a problem hiding this comment.
Remove obsolete CMS comment?
| private static PhantomReference<TestObject> testPhantom1 = null; | ||
|
|
||
| private static WeakReference<TestObject> testWeak2 = null; | ||
| private static WeakReference<TestObject> testWeak3 = null; |
There was a problem hiding this comment.
It looks like test*2 and test*3 test the same thing in both tests, so test*3 could be removed, and test*4 could be renamed test*3.
src/hotspot/share/prims/jvm.cpp
Outdated
| template<DecoratorSet on_ref> | ||
| static bool referenceRefersTo(jobject ref, jobject o) { | ||
| const int offset = java_lang_ref_Reference::referent_offset(); | ||
| oop ref_oop = JNIHandles::resolve_non_null(ref); | ||
| oop referent = HeapAccess<on_ref | AS_NO_KEEPALIVE>::oop_load_at(ref_oop, offset); | ||
| return referent == JNIHandles::resolve(o); | ||
| } |
There was a problem hiding this comment.
How about moving the inner logic to java_lang_ref_{Reference,PhantomReference}::refers_to() functions?
src/hotspot/share/prims/jvm.cpp
Outdated
| JVM_ENTRY(jboolean, JVM_ReferenceRefersTo(JNIEnv* env, jobject ref, jobject o)) | ||
| JVMWrapper("JVM_ReferenceRefersTo"); | ||
| return referenceRefersTo<ON_WEAK_OOP_REF>(ref, o); | ||
| JVM_END |
There was a problem hiding this comment.
... and let this be something like:
JVM_ENTRY(jboolean, JVM_ReferenceRefersTo(JNIEnv* env, jobject ref, jobject o))
JVMWrapper("JVM_ReferenceRefersTo");
oop ref_oop = JNIHandles::resolve_non_null(ref);
oop cmp_oop = JNIHandles::resolve(o);
return java_lang_ref_Reference::refers_to(ref_oop, cmp_oop);
JVM_END
There was a problem hiding this comment.
I like Per's suggestion that makes it explicitly connect to Reference::refers_to.
src/hotspot/share/prims/jvm.cpp
Outdated
| JVM_ENTRY(jboolean, JVM_PhantomReferenceRefersTo(JNIEnv* env, jobject ref, jobject o)) | ||
| JVMWrapper("JVM_PhantomReferenceRefersTo"); | ||
| return referenceRefersTo<ON_PHANTOM_OOP_REF>(ref, o); | ||
| JVM_END |
There was a problem hiding this comment.
... and let this be something like:
JVM_ENTRY(jboolean, JVM_PhantomReferenceRefersTo(JNIEnv* env, jobject ref, jobject o))
JVMWrapper("JVM_PhantomReferenceRefersTo");
oop ref_oop = JNIHandles::resolve_non_null(ref);
oop cmp_oop = JNIHandles::resolve(o);
return java_lang_ref_PhantomReference::refers_to(ref_oop, cmp_oop);
JVM_END
src/hotspot/share/prims/jvm.cpp
Outdated
| JVM_ENTRY(jboolean, JVM_ReferenceRefersTo(JNIEnv* env, jobject ref, jobject o)) | ||
| JVMWrapper("JVM_ReferenceRefersTo"); | ||
| return referenceRefersTo<ON_WEAK_OOP_REF>(ref, o); | ||
| JVM_END |
There was a problem hiding this comment.
I like Per's suggestion that makes it explicitly connect to Reference::refers_to.
| } | ||
|
|
||
| /* Type-erased implementation of refersTo(). */ | ||
| native boolean refersTo0(Object o); |
There was a problem hiding this comment.
Is there any reason why it can't use the parameter type like refersTo0(T o)? Same for PhantomReference::refersTo0(T o)
There was a problem hiding this comment.
But should refersTo takes a T? I mean - if I have a Reference<?> ref, the compiler will not let me query ref.refersTo(o); is this what we want?
There was a problem hiding this comment.
@dfuch I would expect the users of refersTo should know the type of the referent to be compared, i.e. it should know the parameter type of a Reference object. Do you have any specific use case in mind that you have a Reference<?> ref but wants to compare a referent of unknown type?
There was a problem hiding this comment.
@mlchung I have often used a Reference<?> in tests - but my main usage there would be to call ref.refersTo(null) which works in all cases.
My main concern here however is that using T in refersTo seems to go against the advertised usage of the method - I mean - if I have an object obj of type unknown, I can always do obj == ref.get() whatever the parameter type of ref is. But I won't be able to call ref.refersTo(obj) unless I use raw types and suppress warnings. So I just wanted to check that this was intentional.
There was a problem hiding this comment.
For the common cases, the application should know the type of the referent and using T in refersTo will benefit from the compiler type checking. For the unknown type case, cast to Reference<Object> is not ideal but reasonable? something like this:
Reference<Object> r = (Reference<Object>) ref;
r.refersTo(obj);
There was a problem hiding this comment.
Is there a consensus on this issue? It's not clear whether Daniel and Peter have agreed with Mandy's responses or have just not yet responded with further discussion.
There was a problem hiding this comment.
@mlchung I don't have many known use cases, but how about WeakHashMap.containsKey(Object key) for example? Currently WeakHashMap.Entry<K, V> extends WeakReference<Object> but it would be more type safe if it extended WeakReference<K>. In that case an entry.refersTo(key) would not compile...
What I'm trying to say is that even if Reference instances are not "leaked", you can get an untyped object reference from outside and you may want to identity-compare it with the Reference's referent.
There was a problem hiding this comment.
Some thoughts regarding the parameter type of refersTo. Summary: I think refersTo(T) is fine and that we don't want to change it to refersTo(Object).
I don't think we have a migration issue similar to generifying collections, where there was a possibility of changing contains(Object) to contains(E). If that had been done, it would have been a source compatibility issue, because changing the signature of the method potentially affects existing code that calls the method. That doesn't apply here because we're adding a new method.
The question now falls to whether it's preferable to have more convenience with refersTo(Object) or more type-safety with refersTo(T). With the generic collections issue, the migration issue probably drove the decision to keep contains(Object), but this has resulted in a continual set of complaints about the lack of an error when code passes an instance of the "wrong" type. I think that kind of error is likely to occur with refersTo. Since we don't have a source compatibility issue here, we can choose the safer API and avoid this kind of problem entirely.
The safer API does raise the possibility of having to add inconvenient unchecked casts and local variables in certain places, but I think Mandy's comment about the code already having a reference of the "right" type is correct. Her prototype webrev linked above shows that having to add unchecked casts is fairly infrequent.
There was a problem hiding this comment.
Some thoughts regarding the parameter type of refersTo. Summary: I think
refersTo(T)is fine and that we don't want to change it torefersTo(Object).
I agree that we don't have a migration problem here that collections had. So let it be refersTo(T) then.
| * java.lang.ref.PhantomReference | ||
| */ | ||
| JNIEXPORT jboolean JNICALL | ||
| JVM_PhantomReferenceRefersTo(JNIEnv *env, jobject ref, jobject o); |
There was a problem hiding this comment.
Instead of a separate PhantomReference::refersTo0 native method (and new PhantomReference.c), an alternative approach could be to detect if ref is an instance of PhantomReference or other Reference and get the referent accordingly.
| /* Type-erased implementation of refersTo(). */ | ||
| @Override | ||
| native final boolean refersTo0(Object o); | ||
|
|
There was a problem hiding this comment.
How does the existence of this method help future intrinsification?
If it was called somewhere it would be clearer.
Perhaps a comment now or later would help explain its function.
|
Mailing list message from Kim Barrett on hotspot-gc-dev:
I'll just delete lines 178-9.
The test seems to be missing some stuff. This looks like the test from the
I think refersTo functionality doesn't really belong in But it seems referent() does something rather wrong. It's doing a strong |
|
Mailing list message from Kim Barrett on hotspot-gc-dev:
Adding test/jdk/java/lang/ref/ReferenceRefersTo.java
I incorrectly assumed there would be a problem with determining the type of
See response to Roger's question about refersTo0. |
|
Mailing list message from Kim Barrett on hotspot-gc-dev:
public final refersTo(T o) calls refersTo0(o) We have package private: The reason for this split has to do with details in the VM support, in For the native method support we don't need this split. The original Phantom references need to be treated differently from stronger "weak" The Access API that provides the interface to the GC has support for For refersTo the intent is to have the interpreter and C1 use the native By having these intrinsifiable native methods be package private we can have I'll try to come up with some commentary. |
|
Mailing list message from Kim Barrett on hotspot-gc-dev:
I'm going to have to defer to the Java language experts on this question. Mandy? Alan? |
|
Hi Kim,
At what point would the @IntrinsicCandidate annotation be added to the
refersTo0 methods?
it would be useful documentation even if it is not needed for the mechanism.
Thanks for the explanation, Roger
…On 10/8/20 3:58 AM, mlbridge[bot] wrote:
/Mailing list message from Kim Barrett ***@***.***>
on hotspot-gc-dev ***@***.***>:/
On Oct 5, 2020, at 9:15 PM, Roger Riggs <rriggs at
openjdk.java.net> wrote:
src/java.base/share/classes/java/lang/ref/PhantomReference.java
line 66:
64: @OverRide
<https://urldefense.com/v3/__https://github.com/override__;!!GqivPVa7Brio!MXZs7XLDGF7idnDuZ1hTIGY4LTzS5Lx_xCAlYuwDG6C1Vno9_RgMleOn8nJgqULa$>
65: native final boolean refersTo0(Object o);
66:
How does the existence of this method help future intrinsification?
If it was called somewhere it would be clearer.
Perhaps a comment now or later would help explain its function.
public final refersTo(T o) calls refersTo0(o)
We have package private:
native boolean Reference::refersTo0(Object)
final native boolean PhantomReference::refersTo0(Object)
The reason for this split has to do with details in the VM support, in
particular C2 intrinsification.
For the native method support we don't need this split. The original
version from back in April just had a single native method in
Reference. (It
did have native refersTo0, but that was my not realizing native methods
could have a parameterized type that would get type-erased automatically;
see response to Mandy.) That native method performed a runtime check
on the
ReferenceType of the reference's klass to determine what to do. (See
below.)
Phantom references need to be treated differently from stronger "weak"
reference types, because phantom references are weaker than
finalization, so
might not be cleared when a stronger reference to the same object is
cleared. For collectors with STW reference processing this doesn't make
much difference (the referent is either cleared or not), but making this
distinction correctly is necessary for concurrent reference processing.
The Access API that provides the interface to the GC has support for
"unknown" referent accesses that perform a runtime lookup. This is
supported in C++ code, but not in the various Java code processors
(interpreter and compilers). We didn't previously need to support that
case
for Java because the only place where it mattered was accessing the
referent
of a PhantomReference, and that is blocked by PhantomReference::get that
always returns null.
For refersTo the intent is to have the interpreter and C1 use the native
method, but have C2 have special knowledge for it. We could add
support for
the "unknown" reference type to C2, but that's a substantial amount of
work,
and only useful for this one place. Or we can take the same approach
as for
get(), i.e. have a second method on PhantomReference so that calls
that can
select the appropriate method at compile time (usually the case) can have
distinct intrinsics for the two cases.
By having these intrinsifiable native methods be package private we
can have
the public refersTo API function be final, while also preventing any
further
overrides by classes not in the same package.
I'll try to come up with some commentary.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<https://urldefense.com/v3/__https://github.com/openjdk/jdk/pull/498*issuecomment-705400053__;Iw!!GqivPVa7Brio!MXZs7XLDGF7idnDuZ1hTIGY4LTzS5Lx_xCAlYuwDG6C1Vno9_RgMleOn8m3Tmikx$>,
or unsubscribe
<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AAIRSVBIN2XWNEM3ILBOLI3SJVWI3ANCNFSM4SDN74VA__;!!GqivPVa7Brio!MXZs7XLDGF7idnDuZ1hTIGY4LTzS5Lx_xCAlYuwDG6C1Vno9_RgMleOn8skFFiqO$>.
|
|
Mailing list message from Kim Barrett on hotspot-gc-dev:
I now have a patch to implement the C2 intrinsic, mostly provided by eosterlund. Because of that, I decided to hold off and keep it separate, but I intend to make I don?t know what happens if one annotates as an intrinsic candidate with no |
| } else if (ref.refersTo(unexpectedValue)) { | ||
| fail(kind + " refers to unexpected value"); |
There was a problem hiding this comment.
This part, and the use of obj3, below seems odd and unnecessary. If the Reference doesn't have the expected value there's no special reason to think it has obj3 as value (as opposed to something else). I don't think this check will help debugging if this test fails.
There was a problem hiding this comment.
A possible failure mode is to implement the predicate incorrectly, perhaps flipping the sense of the predicate. This may help notice such. It's distinct from the null test, since null is a special value that might be handled differently.
| private static final class TestObject { | ||
| public final int value; | ||
|
|
||
| public TestObject(int value) { | ||
| this.value = value; | ||
| } | ||
| } |
There was a problem hiding this comment.
Introducing this class seems unnecessary, since its value is never used. So all instances of TestObject could just be Object.
There was a problem hiding this comment.
I'd intended to use the value field when printing failure messages, but that became a little messy and probably not worth the trouble in the face of possible null values. So yeah, just using Object is fine; I'll make that change.
| expectNotValue(testWeak2, testObject3, "testWeak2"); | ||
| expectValue(testWeak3, testObject3, "testWeak3"); |
There was a problem hiding this comment.
It looks to me like the expectNotCleared() and expectNotValue() methods can be removed. All expect-tests can be done with just expectCleared() and expectValue().
There was a problem hiding this comment.
I don't think so. For example,
expectNotCleared(testWeak1, "testWeak1")
is not testing the same thing as
expectValue(testWeak1, testObject1, "testWeak1")
If the implementation is correct they will indeed always be consistent. But they could be different with an incorrect implementation.
There was a problem hiding this comment.
Doesn't that just depend on how you implement expectValue()/expectCleared()? You could for example implement expectValue()/expectCleared() to be supersets of expectNotCleared()/expectNotValue(), which I think would make this easier to read. Something like this:
private static void expectValue(...) {
if (ref.refersTo(null)) {
fail("expected " + which + " to not be cleared");
}
if (!ref.refersTo(value)) {
fail(which + " refers to unexpected value");
}
}
private static void expectCleared(...) {
if (ref.refersTo(new TestObject(5))) {
fail(which + " refers to unexpected value");
}
if (!ref.refersTo(null)) {
fail("expected " + which + " to be cleared");
}
}
|
|
||
| /* | ||
| * @test | ||
| * @summary Basic functional test of Reference.refersTo. |
There was a problem hiding this comment.
Thanks for adding this basic test. Can you add @bug 8188055
| } | ||
|
|
||
| private static final <T extends Reference> | ||
| void test(T ref, |
There was a problem hiding this comment.
Nit: I prefer this is merged with line 48 and the line is not too awfully long.
| fail(kind + "refers to null"); | ||
| } else if (!ref.refersTo(expectedValue)) { | ||
| fail(kind + " doesn't refer to expected value"); | ||
| } else if (ref.refersTo(unexpectedValue)) { |
There was a problem hiding this comment.
Nit: It seems easier to read if these are 3 separate checks, i.e. dropping else
|
@kimbarrett the issue for this pull request, JDK-8188055, already has an approved CSR request: JDK-8241029 |
|
@kimbarrett This change now passes all automated pre-integration checks. ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details. After integration, the commit message for the final commit will be: You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been 68 new commits pushed to the
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details. ➡️ To integrate this PR with the above commit message to the |
|
@kimbarrett |
|
@kimbarrett the issue for this pull request, JDK-8188055, already has an approved CSR request: JDK-8241029 |
|
@kimbarrett The |
|
Mailing list message from David Holmes on hotspot-gc-dev: On 20/10/2020 5:01 pm, Kim Barrett wrote:
Mandy's comment implied that references with a null referent never get David
|
|
Mailing list message from Kim Barrett on hotspot-gc-dev:
You said ?immediately cleared?; that?s the trigger. Mandy said ?never cleared or enqueued?, which is different. |
|
Mailing list message from David Holmes on hotspot-gc-dev: On 20/10/2020 5:51 pm, Kim Barrett wrote:
Mandy said never enqueued and that is what I would expect even if "cleared" is just a synonym for "referent == null". David |
Sorry I should have been clearer. What I try to say is that Kim is right that
There are existing use cases depending on [1] https://mail.openjdk.java.net/pipermail/core-libs-dev/2018-July/054325.html |
|
David, Mandy, and I discussed the wording in refersTo javadoc and reached a consensus that is reflected in 3a15b6a. |
|
Build changes look good. |
|
/integrate |
openjdk
bot
added
integrated
|
@kimbarrett Since your change was applied there have been 2 commits pushed to the
Your commit was automatically rebased without conflicts. Pushed as commit 6023f6b. 💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored. |
Reviewed-by: mchung, pliden, rriggs, dholmes, ihse, smarks, alanb
|
Hello! As an IDE developer, I'm thinking about IDE inspection that may suggest the new method. My idea is to suggest replacing every |
|
Thanks to a whole host of folks for reviews and comments. |
Those have different behaviors when ref's class overrides get. Sometimes that might be intentional (PhantomReference, where get blocks access to the referent, and SoftReference, where get may update heuristics for recent accesses delaying GC clearing). But if some further subclass overrides get for some reason, such a change might not be appropriate. |
|
Checking if a reference has been cleared i.e. |
Finally returning to this review that was started in April 2020. I've
recast it as a github PR. I think the security concern raised by Gil
has been adequately answered.
https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-April/029203.html
https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-July/030401.html
https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-August/030677.html
https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-September/030793.html
Please review a new function: java.lang.ref.Reference.refersTo.
This function is needed to test the referent of a Reference object without
artificially extending the lifetime of the referent object, as may happen
when calling Reference.get. Some garbage collectors require extending the
lifetime of a weak referent when accessed, in order to maintain collector
invariants. Lifetime extension may occur with any collector when the
Reference is a SoftReference, as calling get indicates recent access. This
new function also allows testing the referent of a PhantomReference, which
can't be accessed by calling get.
The new function uses native methods whose implementations are in the VM so
they can use the Access API. It is the intent that these methods will be
intrinsified by optimizing compilers like C2 or graal, but that hasn't been
implemented yet. Bear that in mind before rushing off to change existing
uses of Reference.get.
There are two native methods involved, one in Reference and an override in
PhantomReference, both package private in java.lang.ref. The reason for this
split is to simplify the intrinsification. This is a change from the version
from April 2020; that version had a single native method in Reference,
implemented using the ON_UNKNOWN_OOP_REF Access reference strength category.
However, adding support for that category in the compilers adds significant
implementation effort and complexity. Splitting avoids that complexity.
Testing:
mach5 tier1
Locally (linux-x64) verified the new test passes with various garbage collectors.
/csr needed
/label hotspot-gc, core-libs
Progress
Issue
Reviewers
Download
$ git fetch https://git.openjdk.java.net/jdk pull/498/head:pull/498$ git checkout pull/498