Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8320458: Improve structural navigation in API documentation #17062

Closed
wants to merge 12 commits into from

Conversation

hns
Copy link
Member

@hns hns commented Dec 11, 2023

This is a rather big change to update the structural navigation in API documentation generated by JavaDoc. It adds a table of contents for the current page to module, package, and class documentation, and replaces the old sub-navigation bar with a breadcrumb-style links in those pages. The table of contents is displayed as sidebar on wide/desktop displays and integrated into the collapsible menu for narrow/mobile displays. The generated docs can be browsed here:

https://cr.openjdk.org/~hannesw/8320458/api.00/

This change includes improvements in stylesheet.css and script.js that are not strictly related to the navigation changes, but happened as a consequence of the necessary restructuring. All these are described together with the more on-topic changes in the list below.

  • The table of contents is composed as the respective writers generate the pages. For this purpose, HtmlDocletWriter has a new tocBuilder field of new type ListBuilder. If the field is not null it is used to build the table of contents as the page is built using either one of the newHtmlDocletWriter.addToTableOfContents methods or the ListBuilder directly.
  • Once the TOC is built, HtmlDocletWriter.getSideBar is used to generate the markup for the sidebar and it is added to the page via the BodyContents.setSideContent method.
  • Both existing navigation bars (top and sub-navigation) get an additional <div> container with CSS class nav-content that uses a flex layout for its content. This also handles vertical positioning, so the old workaround for vertical of the language version label in Docs.gmk is not necessary anymore.
  • Apart from modules, packages, and classes, other pages that were converted to obtain a table of contents are the "Constant Field Values" page and the Help page.
  • Originally, I used the <aside> element for the sidebar, but I learned that this was the wrong element as it is meant for content that is not strictly related to the main content of the page. The prevailing notion seems to be that a table of contents is a navigation element and therefore should use the <nav> element, so I used that for the TOC sidebar. The same applies for the breadcrumbs sub-navigation, so I left the header <nav> element wrapped around both top and sub-navigation.
  • For the new lists in TOC and breadcrumbs I used ordered list elements <ol> instead of unordered <ul> we use everywhere else, as that is what should be used when list order is important.
  • The SEARCH link that also served as label for the search input field has moved to the top navigation bar. The search input field uses "Search" as placeholder and also got a "Search in documentation" aria-label attribute.
  • The code to locate target anchors in relation to the header navigation has been much improved. For this purpose, new CSS custom properties --top-nav-height, --sub-nav-height and --nav-height have been introduced which are also updated dynamically to avoid any flickering seen in the previous version.
  • Also introduced were a few new CSS custom properties and rules for font sizes that line height that were previously missing. One example is package pages which previously had different line spacing depending on whether a <p> tag was present or not.
  • The Navigation has been simplified a bit and now uses a single addOverviewLink to generate a link to the overview page, the first module page, or the first package page, depending on configuration. a new addPageElementLink method creates a link to the current page element.
  • Several tests were added to TestNavigation and TestModuleNavigation to check the navigation links in previously untested configurations.
  • Some functionality in search.js.template such as implementation of the mobile menu and creating the heading anchor links was not search related and therefore belonged into script.js. This required script.js localized properties to be injected into script.js which is therefore renamed to script.js.template Unfortunately, git did not recognize this as renaming but rather as removal of script.js and addition of script.js.template, but the file is pretty much unchanged up to line 232 with the new code starting in line 233.
  • The reset x button in the search input field and the new TOC filter field behave a bit differently than previously in that they are only visible if there is some text in the input field.
  • An empty string "" in a fragment was so far interpreted as no fragment. However, the empty fragment is now used to navigate to the top of the page (this is standard behaviour and implemented in all browsers and different from clicking on a link without # which results in reloading the page rather than scrolling to the top). This required the changes in DocLink and HtmlLinkInfo as well as the LinkChecker test.
  • Empty and single element lists are required or hard to avoid in breadcrumb and TOC navigation (empty list is present in subheader without breadcrumb links to keep the search input in place, and singleton lists often appear in sublists for single members or headings). Therefore, ample exceptions had to be made in the TestSingletonLists test.
  • The TestSubTitle test was removed as module and package subtitles have been replaced by the breadcrumb links.

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

  • JDK-8320458: Improve structural navigation in API documentation (Enhancement - P3)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/17062/head:pull/17062
$ git checkout pull/17062

Update a local copy of the PR:
$ git checkout pull/17062
$ git pull https://git.openjdk.org/jdk.git pull/17062/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 17062

View PR using the GUI difftool:
$ git pr show -t 17062

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/17062.diff

Webrev

Link to Webrev Comment

@bridgekeeper
Copy link

bridgekeeper bot commented Dec 11, 2023

👋 Welcome back hannesw! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Dec 11, 2023

@hns The following labels will be automatically applied to this pull request:

  • build
  • javadoc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added javadoc javadoc-dev@openjdk.org build build-dev@openjdk.org labels Dec 11, 2023
Comment on lines 46 to 47
import jdk.javadoc.internal.doclets.formats.html.Navigation.PageMode;
import jdk.javadoc.internal.doclets.formats.html.markup.Text;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quibble: order of imports?

.setNavLinkClass(classLinkContent);
List<Content> subnavLinks = new ArrayList<>();
if (configuration.showModules) {
ModuleElement mdle = configuration.docEnv.getElementUtils().getModuleOf(typeElement);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's more common to use utils.elementUtils than the expression configuration.docEnv.getElementUtils()

@@ -122,14 +122,4 @@ static class SerializedForm {
static class TypeUse {
static final TagName SUMMARY_HEADING = TagName.H2;
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay!

Comment on lines 36 to 41
/**
* A utility class for building nested HTML lists. This class is implemented as a
* stack of nested list elements where list items are added to the inner-most
* list. Lists can be added to and removed from the stack using the {@code pushNested}
* and {@code popNested} methods.
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@hns hns marked this pull request as ready for review December 14, 2023 17:14
@openjdk openjdk bot added the rfr Pull request is ready for review label Dec 14, 2023
@mlbridge
Copy link

mlbridge bot commented Dec 14, 2023

Webrevs

Copy link
Member

@erikj79 erikj79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build changes look good given that the stylesheet workaround is no longer needed.

/reviewers 2

@openjdk
Copy link

openjdk bot commented Dec 14, 2023

@hns This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8320458: Improve structural navigation in API documentation

Reviewed-by: erikj, jjg

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 621 new commits pushed to the master branch:

  • 4c1a0fc: 8323995: Suppress notes generated on incremental microbenchmark builds
  • a2b117a: 8324132: G1: Remove unimplemented G1MonitoringSupport::recalculate_eden_size
  • 8e53459: 8323993: Serial: Refactor gc_prologue and gc_epilogue
  • aeb304b: 8324074: increase timeout for jvmci test TestResolvedJavaMethod.java
  • a22ae90: 8321938: java/foreign/critical/TestCriticalUpcall.java does not need a core file
  • 806ffb1: 8324082: more monitoring test timeout adjustments
  • 52f787f: 8323595: is_aligned(p, alignof(OopT))) assertion fails in Jetty without compressed OOPs
  • cbfddf4: 8317287: [macos14] InterJVMGetDropSuccessTest.java: Child VM: abnormal termination
  • 57fad67: 8323667: Library debug files contain non-reproducible full gcc include paths
  • ff8cc26: 8323694: RISC-V: Unnecessary ResourceMark in NativeCall::set_destination_mt_safe
  • ... and 611 more: https://git.openjdk.org/jdk/compare/26c3390421f4888eb59017cadb2bf21a15e25b5e...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Dec 14, 2023
@openjdk
Copy link

openjdk bot commented Dec 14, 2023

@erikj79
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

@openjdk openjdk bot removed the ready Pull request is ready to be integrated label Dec 14, 2023
…mats/html/HtmlDocletWriter.java

Co-authored-by: Andrey Turbanov <turbanoff@gmail.com>
@@ -350,7 +350,7 @@ protected void addLinksForIndexes(List<Character> allFirstCharacters, Content co
}

content.add(new HtmlTree(TagName.BR));
List<Content> pageLinks = Stream.of(IndexItem.Category.values())
List<? extends Content> pageLinks = Stream.of(IndexItem.Category.values())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List<? extends Content> is not wrong and is better than List<Content> but would var be simpler and even better?

Comment on lines 104 to 105
writer.addToTableOfContents(HtmlIds.METHOD_DETAIL, contents.methodDetailLabel);
writer.tocBuilder.pushNested(HtmlTree.OL(HtmlStyle.tocList));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coding-style question:
Rather than have multiple members (methods and fields) on HtmlDocletWriter would it be better to treat the TOC as a first class item for each page and put all the functionality there? Or maybe that's a separate cleanup for later.

@@ -169,35 +134,28 @@ private void addMainNavLinks(Content target) {
switch (documentedPage) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe not now, but I have long felt this big switch statement could be better served as a virtual abstract method, with suitable implementations in each kind of page.

Comment on lines 218 to 224
if (element != null && !(element instanceof ModuleElement)) {
addPageElementLink(target);
if (options.classUse()) {
if (element instanceof PackageElement || element instanceof TypeElement) {
addItemToList(target, links.createLink(DocPaths.PACKAGE_USE, contents.useLabel));
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a matter of policy, it is generally incorrect to use instanceof for subtypes of Element. The correct coding style is to get and check the ElementKind.

That being said, the code here will work on the JDK implementation of the Language Model API, but the coding style may not work on other implementations.

if (element != null) {
addPageElementLink(target);
if (options.classUse()) {
if (element instanceof PackageElement pkg) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah ... this is where it is nice to use instanceof patterns, even though the recommended form is to check the ElementKind and do a separate cast. As I said above, this will work on JDK, but the coding style may not work in other contexts.

.setNavLinkModule(linkContent);
List<Content> subnavLinks = new ArrayList<>();
if (configuration.showModules) {
ModuleElement mdle = configuration.docEnv.getElementUtils().getModuleOf(packageElement);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor suggestion: use utils.elementUtils instead of configuration.docEnv.getElementUtils()

"""
<a href="../../../overview-tree.html">Tree</a>""",
<a href="../package-tree.html">Tree</a>""",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the significance of this filename change?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For context-specific links in the top navigation bar we always link to the target closest to the current page; for example, The "USE" link in class documentation links to the Class Use page of the current class, and the "TREE" link in files belonging to a package links to the Package Tree page. This was previously not handled correctly for package-level doc files. It is now fixed, so the test has to be updated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation.

Comment on lines +125 to +128
<ol class="toc-list">
<li><a href="#" tabindex="0">Description</a></li>
<li><a href="#packages-summary" tabindex="0">Packages</a></li>
</ol>""");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had missed this subtlety earlier. The vertical bars have gone from the generated code. Nice!

@@ -576,7 +577,7 @@ void checkHtml5NoDescription(boolean found) {
}

void checkModuleLink() {
checkOutput("index.html", true,
checkOutput("index.html", false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting change ;-). And yes, I agree.

Comment on lines 57 to 58
public void testOverview(Path ignore) {
javadoc("-d", "out-overview",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, I would recommend renaming Path ignore to Path base and then generating a filename for use in the javadoc call, such as base.resolve("api").toString(). This helps guarantee the files for each test case are isolated.

@@ -201,57 +172,82 @@ public static class Y extends X {
package pkg1; public interface InterfaceWithNoMembers {
}""");

javadoc("-d", "out-3",
javadoc("-d", "out-navlinks",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See preceding comment. I would recommend using the equivalent of base.resolve("api:).toString() here.

Comment on lines +209 to +214
<li><a href="#nested-class-summary" tabindex="0">Nested Class Summary</a></li>
<li><a href="#field-summary" tabindex="0">Field Summary</a></li>
<li><a href="#constructor-summary" tabindex="0">Constructor Summary</a></li>
<li><a href="#method-summary" tabindex="0">Method Summary</a></li>
<li><a href="#constructor-detail" tabindex="0">Constructor Details</a>
<ol class="toc-list">
Copy link
Contributor

@jonathan-gibbons jonathan-gibbons Jan 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General comment, not specific to this instance:
I like that we cleaned up HTML id names in recent releases.

if ("tag-list".equals(attrs.get("class"))) {
inSeeList = true;
String classAttr = attrs.get("class");
if ("tag-list".equals(classAttr) || "toc-list".equals(classAttr) || "sub-nav-list".equals(classAttr)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not wrong, but it might be a good place to use Set.contains in some manner.

Comment on lines +208 to +218
// special handling for strings in .js.template files
for (String fileName : List.of("resources/search.js.template", "resources/script.js.template")) {
FileObject fo = fm.getFileForInput(javadocLoc,
"jdk.javadoc.internal.doclets.formats.html",
fileName);
CharSequence search_js = fo.getCharContent(true);
Pattern p = Pattern.compile("##REPLACE:(?<key>[A-Za-z0-9._]+)##");
Matcher m = p.matcher(search_js);
while (m.find()) {
results.add(m.group("key"));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See earlier suggestion about possibly merging template-y values into a single file.

@openjdk
Copy link

openjdk bot commented Jan 11, 2024

⚠️ @hns This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

@hns
Copy link
Member Author

hns commented Jan 12, 2024

The last commit (70d06e5) moves the TOC-related methods from HtmlDocletWriter to a new TableOfContents class.

This includes some not totally obvious changes:

  • <h2> headings in the main description are now added directly to the TOC/ListBuilder instead of collecting them separately and then adding them as a complete sublist. This has a few other consequences:
    • Method ListBuilder.addNested(HtmlTree) is no longer needed and has been removed, all sublists are added via a sequence of push-add-pop.
    • The stack in ListBuilder now stacks both the current list and the current list item as an HtmlTree[] array of length 2. This is because a nested list is only added to the parent list item after it is popped to make sure it is not empty.
  • The constant values page did previously not add tabindex="0" attributes. This has been fixed which required the respective test to be updated.

@@ -97,7 +96,7 @@ public ClassWriter(HtmlConfiguration configuration, TypeElement typeElement,
this.classTree = classTree;

pHelper = new PropertyUtils.PropertyHelper(configuration, typeElement);
tocBuilder = new ListBuilder(HtmlTree.OL(HtmlStyle.tocList));
tableOfContents = new TableOfContents(this);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to do this here (and also in ModuleWriter and PackageWriter etc) rather than in the (shared) super-constructor for HtmlDocletWriter ?

Copy link
Member Author

@hns hns Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for doing it in the concrete writer classes is that only some HtmlDocletWriter subclasses have a table of contents, and I don't think there's an easy way to figure out in the super-constructor which instances need a table of contents and which don't. We could have the subclasses pass it as an argument to the super-constructor I guess.

protected final Map<String, String> headings = new LinkedHashMap<>();

protected ListBuilder tocBuilder;
protected TableOfContents tableOfContents;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you init it in the constructor for HtmlDocletWriter, the field can be final

*/
public class ListBuilder extends Content {

private final HtmlTree root;
private final Deque<HtmlTree> stack = new ArrayDeque<>();
private final Deque<HtmlTree[]> stack = new ArrayDeque<>();
Copy link
Contributor

@jonathan-gibbons jonathan-gibbons Jan 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of an array of fixed-size 2, consider using a locally defined record, which means you get nice helpful names for the two components, instead of just [0] and [1]

For example, you could insert the following immediately before this line:

private record Pair(HtmlTree list, HtmlTree item) { }

and then update the use sites to match.

* @param hasFilterInput whether to add a filter text input
* @return a content object
*/
protected Content getSideBar(boolean hasFilterInput) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Just asking)

In other builders for "complex" content, like Table and Navigation , we have methods like toContent or getContent. Should we follow that naming convention? While the name getSidebar is certainly more descriptive, is the returned object inherently a sidebar -- or is the fact that it is a sidebar just an artifact of how we currently choose to present the content?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is actually up to the CSS how the table of contents is rendered. I'm renaming the method to toContent().

Copy link
Contributor

@jonathan-gibbons jonathan-gibbons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, very nice: the change to introduce and use a new TableOfContents class is definitely a worthwhile improvement.

  • Medium-size suggestion/question:
    consider init-ing HtmlDocletWriter.tableOfContents directly in the constructor for HtmlDocletWriter and not in the constructors for the subtypes.

  • Small suggestion:
    consider using a locally defined record instead of a two-element array in the new TableOfContents class.

@hns
Copy link
Member Author

hns commented Jan 18, 2024

* Medium-size suggestion/question:
  consider init-ing `HtmlDocletWriter.tableOfContents` directly in the constructor for `HtmlDocletWriter` and not in the constructors for the subtypes.

I agree it would be nice, but not sure how to do it without proliferation of HtmlDocletWriter constructors and/or constructor arguments (we have 15 direct HtmlDocletWriter subclasses of which 5 have a table of contents, and they are using both available super-constructors).

 - Use record instead of array in ListBuilder
 - Rename TableOfContents.getSideBar to toContent
Copy link
Contributor

@jonathan-gibbons jonathan-gibbons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jan 18, 2024
@hns
Copy link
Member Author

hns commented Jan 18, 2024

/integrate

@openjdk
Copy link

openjdk bot commented Jan 18, 2024

Going to push as commit 81df265.
Since your change was applied there have been 625 commits pushed to the master branch:

  • a6c0b10: 8323684: TypeMirror.{getAnnotationsByType, getAnnotation} return uninformative results
  • 5c874c1: 8324161: validate-source fails after JDK-8275338
  • b6233c3: 8321925: sun/security/mscapi/KeytoolChangeAlias.java fails with "Alias <246810> does not exist"
  • bfd2afe: 8275338: Add JFR events for notable serialization situations
  • 4c1a0fc: 8323995: Suppress notes generated on incremental microbenchmark builds
  • a2b117a: 8324132: G1: Remove unimplemented G1MonitoringSupport::recalculate_eden_size
  • 8e53459: 8323993: Serial: Refactor gc_prologue and gc_epilogue
  • aeb304b: 8324074: increase timeout for jvmci test TestResolvedJavaMethod.java
  • a22ae90: 8321938: java/foreign/critical/TestCriticalUpcall.java does not need a core file
  • 806ffb1: 8324082: more monitoring test timeout adjustments
  • ... and 615 more: https://git.openjdk.org/jdk/compare/26c3390421f4888eb59017cadb2bf21a15e25b5e...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Jan 18, 2024
@openjdk openjdk bot closed this Jan 18, 2024
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jan 18, 2024
@openjdk
Copy link

openjdk bot commented Jan 18, 2024

@hns Pushed as commit 81df265.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build build-dev@openjdk.org integrated Pull request has been integrated javadoc javadoc-dev@openjdk.org
Development

Successfully merging this pull request may close these issues.

4 participants