Skip to content
This repository has been archived by the owner on Sep 2, 2022. It is now read-only.

JDK-8270866: NPE in DocTreePath.getTreePath() #264

Closed
wants to merge 4 commits into from

Conversation

hns
Copy link
Member

@hns hns commented Jul 21, 2021

This change fixes a NPE in JavadocLog when the DocTreePath for an inherited doc comment can't be retrieved.

The most important change is to fix the overriddenElement feature in CommentHelper. The purpose of this feature is to enable lookup of the DocTreePath or Element if the comment was inherited from an overridden member, but it was not implemented and used correctly. With this change, the feature is implemented in CommentHelper.getDocTreePath(DocTree) and that method is used whenever the DocTreePath is needed instead of duplicating the functionality elsewhere.

The CommentHelper.setOverrideElement(Element) method was previously used in four places:

  • In InheritDocTaglet.java when generating a piece of inherited documentation
  • In ReturnTaglet when generating inherited return value documentation
  • In ParamTaglet when generating inherited documentation for a parameter
  • In MemberSummaryBuilder when generating inherited summary documentation for an executable member

The first three usages have been removed with this change because they were not necessary. We can simply use the overridden member owning the comment instead of the overriding one to generate the output. In InheritDocTaglet we already did that, in ReturnTaglet and ParamTaglet I changed the first argument to the doc-generating method from holder to inheritedDoc.holder. (Note that in ParamTaglet the name of the parameter which can change in the overriding member is passed as separate argument.)

The remaining use of CommentHelper.setOverrideElement(Element) in MemberSummaryBuilder was a bit more difficult, since the invoked method MemberSummaryWriter.addMemberSummary generates not just the doc comment, but the whole summary line including the member signature. I tried adding the CommentHelper or Element owning the comment as an additional parameter, but there is pretty long line of methods that must carry the extra parameter around (as can be seen in the stack trace in the JBS issue). In the end I decided to keep the current mechanism to register the overridden holder element with the comment helper.

One thing I am unsure about is whether it is possible for the CommentHelper we register the overridden element on in MemberSummaryBuilder.buildSummary to be garbage collected under extreme memory pressure before it is retrieved and used again down the call graph. The local reference we use to register the comment holder is no longer reachable at that time and it is referenced by a SoftReference in CommentHelperCache. This is probably more of a theoretical issue, and it has existed before, but I thought I should mention it.

Although it should no longer be required, I added back the null checks in the methods to retrieve the DiagnosticSource and DiagnosticPosition instances in JavadocLog. These null checks have been there in the method's precursors in the Messager class and were dropped in JDK-8267126. As far as I can tell, all the reporting functionality in JavadocLog accepts null values for source and position, so this seemed like the right thing to do.

Finally, as a byproduct of my attempt of adding a new parameter I dropped some unused parameters in various writer classes. Since this is just cleanup I can undo these changes to keep it as small as possible.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk17 pull/264/head:pull/264
$ git checkout pull/264

Update a local copy of the PR:
$ git checkout pull/264
$ git pull https://git.openjdk.java.net/jdk17 pull/264/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 264

View PR using the GUI difftool:
$ git pr show -t 264

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk17/pull/264.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Jul 21, 2021

👋 Welcome back hannesw! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Jul 21, 2021

@hns The following label will be automatically applied to this pull request:

  • javadoc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the javadoc javadoc-dev@openjdk.java.net label Jul 21, 2021
@hns hns marked this pull request as ready for review July 21, 2021 17:13
@openjdk openjdk bot added the rfr Pull request is ready for review label Jul 21, 2021
@mlbridge
Copy link

mlbridge bot commented Jul 21, 2021

Webrevs

if (path == null || path.getTreePath() == null) {
return null;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please justify this change and the other similar changes in this file.
It was a deliberate change to drop these checks; I don't see why they're being undone. These checks have the effect of covering up bugs. Just because the old code was sloppy doesn't mean this code should be as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know the removal was a deliberate change. The decision to propose to reintroduce those checks was based on the following considerations:

  • reporting/formatting code tolerates missing source/position
  • NPE crashes on the other hand are quite serious
  • this code is called from a lot of places
  • we removed these checks quite recently, and we're very close to release JDK 17

That said, I'm not fully convinced we need those checks either. If you feel we can do without it I'm fine with that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a bit of a surprise when I removed the null checks in JavadocLog as I saw the same NPE when building JDK API docs (the tests were obviously fine). Turns out the problem was the code I removed from CommentHelper that was labeled as "Case B":

https://github.com/openjdk/jdk17/pull/264/files#diff-8e5643f82a3b1712561fbeb4ab7a60b0a0cc3e88d7abf9b86b4d15a6bb303d87L183-L187

Although the purpose of this code was supposedly to handle the case of a comment inherited via @inheritDoc (which I confirmed, and which is why I concluded the code was no longer needed because InheritDocTaglet already uses the "correct" member), there is another case where this code was needed, which is when an overriding member has a doc comment to only add a block tag but still inherits the main description from the overridden member (without {@inheritDoc} tag).

I added commit afc0605 that adds code to CommentHelper.getPath that is equivalent to the old "Case B" code. It also adds a test for this case and removes the null checks in JavadocLog.

Copy link
Contributor

@jonathan-gibbons jonathan-gibbons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks a better (but more aggressive) solution than I might have done, which would have been to pass around a record containing the CommentHelper with the list of DocTrees. The "cost" is passing around a different holder element which is a bigger deal.

The comment in the description about the use of SoftReference should be followed up on, later.

I'd prefer the checks to stay out of JavadocLog because long term I think they cover up bugs when nulls are incorrectly being passed around. If anything, I'd consider replacing the checks with either assert or Objects.requireNonNull. But I'm OK to leave the checks in 17 if you think there is significant benefit to do so, and because we don't have the time to track down why nulls might be being passed down.

I recommend building JDK docs before/after this change, to verify no changes in the generated output. Bonus points for comparing the output of jtreg tests before/after this change, to verify that only expected differences are found/

@@ -38,7 +38,6 @@
import jdk.javadoc.internal.doclets.formats.html.markup.HtmlStyle;
import jdk.javadoc.internal.doclets.formats.html.markup.TagName;
import jdk.javadoc.internal.doclets.formats.html.markup.HtmlTree;
import jdk.javadoc.internal.doclets.formats.html.markup.RawHtml;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay!

@openjdk
Copy link

openjdk bot commented Jul 22, 2021

@hns This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8270866: NPE in DocTreePath.getTreePath()

Reviewed-by: jjg

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 24 new commits pushed to the master branch:

  • 4812e53: 8271094: runtime/duplAttributes/DuplAttributesTest.java doesn't check exit code
  • 6a9ab6a: 8271093: remove deadcode from runtime/Thread/TestThreadDumpSMRInfo.java test
  • e7f9009: 8270085: Suspend during block transition may deadlock if lock held
  • 39b486d: 8271126: ProblemList runtime/InvocationTests/invokevirtualTests.java
  • 4119a52: 8270461: ZGC: Invalid oop passed to ZBarrierSetRuntime::load_barrier_on_oop_array
  • 89f7998: 8266347: assert(Dependencies::is_concrete_root_method(fm, ctxk) == Dependencies::is_concrete_method(m, ctxk)) failed: mismatch
  • e1051ae: 8264066: Enhance compiler validation
  • add995b: 8265201: JarFile.getInputStream not validating invalid signed jars
  • ca6b222: 8258432: Improve File Transfers
  • a3acce5: 8264079: Improve abstractions
  • ... and 14 more: https://git.openjdk.java.net/jdk17/compare/61359c46a74b03b94738744d309334a5af162f08...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jul 22, 2021
public void addSummaryLinkComment(Element member, Content contentTree) {
List<? extends DocTree> tags = utils.getFirstSentenceTrees(member);
addSummaryLinkComment(mw, member, tags, contentTree);
addSummaryLinkComment(member, tags, contentTree);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this (a bit) surprising, since it is (potentially) useful to have access to the enclosing writer, but that being said, if we ever really need access to the enclosing AbstractMemberWriter it is probably better passed in to the constructor.

@hns
Copy link
Member Author

hns commented Jul 23, 2021

Thanks for the review, Jon!

I did a recursive diff on the generated docs before and after this change, there are no differences. I'm leaving the JavadocLog checks out as you suggest.

@hns
Copy link
Member Author

hns commented Jul 27, 2021

/integrate

@openjdk
Copy link

openjdk bot commented Jul 27, 2021

Going to push as commit fbe28e4.
Since your change was applied there have been 39 commits pushed to the master branch:

  • f662127: 8270491: SEGV at read_string_field(oopDesc*, char const*, JavaThread*)+0x54
  • cea7bc2: 8271223: two runtime/ClassFile tests don't check exit code
  • b76a838: 8269150: UnicodeReader not translating \u005c\u005d to \]
  • 7ddabbf: 8271175: runtime/jni/FindClassUtf8/FindClassUtf8.java doesn't have to be run in othervm
  • 3c27f91: 8271222: two runtime/Monitor tests don't check exit code
  • 049b2ad: 8015886: java/awt/Focus/DeiconifiedFrameLoosesFocus/DeiconifiedFrameLoosesFocus.java sometimes failed on ubuntu
  • 8adf008: 8269984: [macos] JTabbedPane title looks like disabled
  • e90ed6c: 8271173: serviceability/jvmti/GetObjectSizeClass.java doesn't check exit code
  • b4c6229: 8271189: runtime/handshake/HandshakeTimeoutTest.java can be run in driver mode
  • e3800e6: 8271162: runtime/StackTrace/LargeClassTest.java can be run in driver mode
  • ... and 29 more: https://git.openjdk.java.net/jdk17/compare/61359c46a74b03b94738744d309334a5af162f08...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Jul 27, 2021
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jul 27, 2021
@openjdk
Copy link

openjdk bot commented Jul 27, 2021

@hns Pushed as commit fbe28e4.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
integrated Pull request has been integrated javadoc javadoc-dev@openjdk.java.net
2 participants