-
Notifications
You must be signed in to change notification settings - Fork 6.1k
8352645: Add tool support to check order of includes #24247
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
👋 Welcome back dnsimon! A progress list of the required criteria for merging this PR into |
|
@dougxc This change now passes all automated pre-integration checks. ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details. After integration, the commit message for the final commit will be: You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been 175 new commits pushed to the
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details. ➡️ To integrate this PR with the above commit message to the |
ee585f1 to
c82e6a0
Compare
c82e6a0 to
6277947
Compare
Webrevs
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for updating to use the lower-case comparison. I wonder if a small tweak can fix the extra blank lines I complained about in the other PR.
The tool removes the extra blank line we have in our .inline.hpp. From the Style Guide:
All .inline.hpp files should include their corresponding .hpp file as the first include line with a blank line separating it from the rest of the include lines. Declarations needed by other files should be put in the .hpp file, and not in the .inline.hpp file. This rule exists to resolve problems with circular dependencies between .inline.hpp files.
I think this needs to be fixed, otherwise people will start to remove these.
| if (!userIncludes.isEmpty() && !sysIncludes.isEmpty() && blankLines.isEmpty()) { | ||
| blankLines = List.of(""); | ||
| } | ||
| result.addAll(blankLines); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this line is removed you don't get the extra blank lines I mentioned in the previous PR. It also removes the extra blank line that you get inserted into oopMap.inline.hpp before the INCLUDE_JVMCI block.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or, rather if the code is changed to:
if (!userIncludes.isEmpty() && !sysIncludes.isEmpty()) {
result.add("");
}
| #define SHARE_VM_COMPILER_OOPMAP_INLINE_HPP | ||
|
|
||
| #include "compiler/oopMap.hpp" | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This blank line should not be removed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for doing the last two fixes. I think this looks good now, but I need a bit more time to do some deeper verification. Thanks!
| import java.util.stream.Collectors; | ||
|
|
||
| public class SortIncludes { | ||
| private static final String INCLUDE_LINE = "^ *#include *(<[^>]+>|\"[^\"]+\") *$\\n"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are files that have spaces between the # and include. I'm kind of inclined to suggest we fix those
at some point (not in this PR). But the regex here needs to allow for that possibility, and perhaps (eventually)
complain about such.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since there are no such cases in the files processed in this PR, I'd suggest not adding support for them. They can be fixed in follow up PRs as the relevant directories are added to TestIncludesAreSorted.HOTSPOT_SOURCES_TO_CHECK.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The regex needs to detect that case eventually anyway, so I think it should be done now. Either we allow that
case, in which case the regex must match to work properly where they are present. Or we forbid that case,
in which case the regex must match to detect future mistakes even after we've cleaned up existing usage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To me it seems like a small adjustment fixes this
| private static final String INCLUDE_LINE = "^ *#include *(<[^>]+>|\"[^\"]+\") *$\\n"; | |
| private static final String INCLUDE_LINE = "^ *# *include *(<[^>]+>|\"[^\"]+\") *$\\n"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
| return String.join("\n", result) + "\n"; | ||
| } | ||
|
|
||
| /// Processes the C++ source file in `path` to sort its include statements. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we want to apply this to hotspot jtreg test code, then C source files also come into the picture.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the tool will need to be updated to handle C source files. At that point, the comment should be generalized.
| } | ||
|
|
||
| /// Processes the C++ source files in `paths` to check if their include statements are sorted. | ||
| /// Include statements with any non-space characters after the closing `"` or `>` will not |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps this should be mentioned in the style guide?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
|
|
||
| #include "runtime/interfaceSupport.inline.hpp" | ||
|
|
||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extra blank line inserted?
| #define SHARE_CI_CIUTILITIES_INLINE_HPP | ||
|
|
||
| #include "ci/ciUtilities.hpp" | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extra blank line not removed?
| #include "compiler/abstractCompiler.hpp" | ||
| #include "compiler/compilationFailureInfo.hpp" | ||
| #include "compiler/compileTask.hpp" | ||
| #ifdef COMPILER2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Conditional includes are supposed to follow unconditional in a section.
Out of scope for this PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep. From the PR description:
The tool does nothing about re-ordering blocks of conditional includes vs unconditional includes. I briefly looked into that but it gets very complicated, very quickly. That kind of re-ordering will have to continue to be done manually for now.
| #include "utilities/globalDefinitions.hpp" | ||
| #include "utilities/macros.hpp" | ||
|
|
||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Extra blank line inserted?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems to be left-overs from an earlier run. If I run the tool on this file it doesn't add this blank line.
|
Probably we want to eventually apply this to gtests, but there might be additional rules there. The include of Applying it to jtreg test support files could also introduce some challenges. Or at least discover a lot of These kinds of things can be followups once we have the basic mechanism in place. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran the latest script over the HotSpot source and see that it messes up corner-cases with our platform includes.
diff --git a/src/hotspot/cpu/aarch64/continuationEntry_aarch64.inline.hpp b/src/hotspot/cpu/aarch64/continuationEntry_aarch64.inline.hpp
index df4d3957239..e8816767a96 100644
--- a/src/hotspot/cpu/aarch64/continuationEntry_aarch64.inline.hpp
+++ b/src/hotspot/cpu/aarch64/continuationEntry_aarch64.inline.hpp
@@ -25,10 +25,9 @@
#ifndef CPU_AARCH64_CONTINUATIONENTRY_AARCH64_INLINE_HPP
#define CPU_AARCH64_CONTINUATIONENTRY_AARCH64_INLINE_HPP
-#include "runtime/continuationEntry.hpp"
-
#include "code/codeCache.hpp"
#include "oops/method.inline.hpp"
+#include "runtime/continuationEntry.hpp"
#include "runtime/frame.inline.hpp"
#include "runtime/registerMap.hpp"
The includes are:
.hpp --------------> _aarch64.hpp
^ ^
| |
| +------------------+
| |
.inline.hpp -------> _aarch64.inline.hpp
So, continuationEntry.hpp acts like the .hpp file for continuationEntry_aarc64.inline.hpp.
Unfortunately, we don't have a fully consistent way to write our platform includes, so I don't know how to codify this in a tool without breaking things.
| #include "utilities/globalDefinitions.hpp" | ||
| #include "utilities/macros.hpp" | ||
|
|
||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems to be left-overs from an earlier run. If I run the tool on this file it doesn't add this blank line.
|
I verified that adding a comment to the end of the |
I would suggest someone open issue(s) for follow up enhancements to the tool. I think having something in place now and incrementally improving it and adjusting it for all the special cases makes most sense. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm happy with the capabilities of the tool now and think that it is good enough to include and promote to HotSpot devs.
One questions is where to put the tool? I don't think the test directory is the best place. Maybe somewhere in src/utils/. There is a tools dir here src/utils/src/build/tools/ but I don't know if it is appropriate to put it there. Maybe @magicus knows a good place for this?
A couple of nits:
- jcheck fails because of whitespaces
- The /// style comments is a style I haven't encountered before.
I fixed the whitespaces. |
|
I just noticed that TestIncludesAreSorted is not run by GHA. How about we move |
Maybe someone else knows the preferred style for this? I don't think we need to block the integration because of this. If someone comes late with the proper comment style, we'll update it in a separate PR. |
I would actually recommend just the |
This is for the new markdown comments. Personally, I very much prefer them and have been looking forward to these for a long time. But I don't know if we have any policy for or against those in the JDK. Using them in a script like this seems fine to me, at any rate. |
I went ahead and pushed this change. |
Fine by me but I'm not sure how to then use |
|
Hm... I know the source code is bundled with the test image, but I'm not 100% sure if it just includes |
There was some discussion of this when recently adding the sources/TestNoNULL.java test. The code used here |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me.
I personally would have preferred to have the tool somewhere other than in the test directory, but I've gotten feedback from other HotSpot devs that they think its better to have the tool there.
I leave the review of TEST.group to someone else.
| -serviceability/jvmti/negative | ||
|
|
||
| tier1_common = \ | ||
| sources \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand this change. How does this end up doing anything different than before?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes sources be tested in GHA:
jdk/.github/workflows/test.yml
Line 88 in a1ab1d8
| test-suite: 'test/hotspot/jtreg/:tier1_common' |
An alternative would be to add a separate GHA jobs just for sources:
- test-name: 'hs/tier1 sources'
test-suite: 'test/hotspot/jtreg/:tier1_sources'
debug-suffix: -debug
Given how small sources is (currently only 1 test), it felt like it should just be folded into common.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, the workflows definition is what I was having trouble finding. I
understand now. In light of that, the proposed change to the groups looks
fine.
| -serviceability/jvmti/negative | ||
|
|
||
| tier1_common = \ | ||
| sources \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, the workflows definition is what I was having trouble finding. I
understand now. In light of that, the proposed change to the groups looks
fine.
|
Thanks for all the discussion and reviews. /integrate |
|
Going to push as commit 814730e.
Your commit was automatically rebased without conflicts. |
This PR adds
test/hotspot/jtreg/sources/SortIncludes.java, a tool to check that blocks of include statements in C++ files are sorted and that there's at least one blank line between user and sys includes (as per the style guide).By virtue of using
SortedSet, the tool also removes duplicate includes (e.g."compiler/compilerDirectives.hpp"on line 37 and line 41). Sorting uses lowercased strings so that_sorts before letters, preserving the prevailing convention in the code base. I've also updated the style guide to clarify this sort-order.The tool does nothing about re-ordering blocks of conditional includes vs unconditional includes. I briefly looked into that but it gets very complicated, very quickly. That kind of re-ordering will have to continue to be done manually for now.
I have used the tool to fix the ordering of a subset of HotSpot sources and added a test to keep them sorted. That test can be expanded over time to keep includes sorted in other HotSpot directories.
When
TestIncludesAreSorted.javafails, it tries to provide actionable advice. For example:This PR includes a commit with ordering suppression comments for some files I discovered needed it while playing around in #24180 .
This PR replaces #24180.
Progress
Issue
Reviewers
Reviewing
Using
gitCheckout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/24247/head:pull/24247$ git checkout pull/24247Update a local copy of the PR:
$ git checkout pull/24247$ git pull https://git.openjdk.org/jdk.git pull/24247/headUsing Skara CLI tools
Checkout this PR locally:
$ git pr checkout 24247View PR using the GUI difftool:
$ git pr show -t 24247Using diff file
Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/24247.diff
Using Webrev
Link to Webrev Comment