8256156: JFR: Allow 'jfr' tool to show metadata without a recording by y1yang0 · Pull Request #1904 · openjdk/jdk

y1yang0 · 2020-12-29T07:44:36Z

Hi all,

May I please have a review for this minor patch?

It simplifies the use of subcommand metadata of jfr tool. Recording
file is no longer mandatory, users can directly use jfr metadata to
output JDK builtin meta information. As JDK-8256156 mentioned, it also
supports events and categories filters:

$ jfr metadata 
$ jfr metadata recording.jfr # compatible
$ jfr metadata --events jdk.ThreadStart,jdk.ThreadEnd
$ jfr metadata --events jdk.ThreadStart,jdk.ThreadEnd recording.jfr
$ jfr metadata --categories GC,Detailed
$ jfr metadata --categories GC,Detailed recording.jfr

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8256156: JFR: Allow 'jfr' tool to show metadata without a recording

Reviewers

Erik Gahlin (@egahlin - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jdk pull/1904/head:pull/1904
$ git checkout pull/1904

bridgekeeper · 2020-12-29T07:45:22Z

Hi @kelthuzadx, welcome to this OpenJDK project and thanks for contributing!

We do not recognize you as Contributor and need to ensure you have signed the Oracle Contributor Agreement (OCA). If you have not signed the OCA, please follow the instructions. Please fill in your GitHub username in the "Username" field of the application. Once you have signed the OCA, please let us know by writing /signed in a comment in this pull request.

If you already are an OpenJDK Author, Committer or Reviewer, please click here to open a new issue so that we can record that fact. Please use "Add GitHub user kelthuzadx" as summary for the issue.

If you are contributing this work on behalf of your employer and your employer has signed the OCA, please let us know by writing /covered in a comment in this pull request.

y1yang0 · 2020-12-29T07:49:18Z

/covered

bridgekeeper · 2020-12-29T07:49:30Z

Thank you! Please allow for a few business days to verify that your employer has signed the OCA. Also, please note that pull requests that are pending an OCA check will not usually be evaluated, so your patience is appreciated!

y1yang0 · 2021-01-05T02:06:03Z

/signed

bridgekeeper · 2021-01-05T02:09:58Z

You are already a known contributor!

y1yang0 · 2021-01-05T02:52:14Z

/label add hotspot-jfr

openjdk · 2021-01-05T02:52:23Z

@kelthuzadx
The hotspot-jfr label was successfully added.

y1yang0 · 2021-01-05T02:54:49Z

/label add ready

mlbridge · 2021-01-05T02:55:04Z

Webrevs

15: Full - Incremental (c401df4)
14: Full - Incremental (a2cedbf9841a60222e727006e24309fd84da2a9c)
13: Full - Incremental (ff13850)
12: Full - Incremental (690f42242126e7f1493bb72eef54b3cf0856c4b6)
11: Full - Incremental (a26202f)
10: Full - Incremental (15a3ab4)
09: Full - Incremental (5ffcaca04bbda95f540b586dde8424f2b8685449)
08: Full - Incremental (6c97d4ba85398ca223b9a9847c033529c3a5a7a4)
07: Full - Incremental (f4faa40)
06: Full - Incremental (586a182)
05: Full - Incremental (6840cdf84173ada6751f78249718d67001cf9d36)
04: Full - Incremental (9c8d0d9)
03: Full - Incremental (d43684e)
02: Full - Incremental (81a61405d21143202db26c90ff9b24e4c636261e)
01: Full - Incremental (b3c2ea58b8bcff2aaa74f2c6ca6f67d593f2be68)
00: Full (ca42a73)

openjdk · 2021-01-05T02:55:07Z

@kelthuzadx The label ready is not a valid label. These labels are valid:

serviceability
hotspot
sound
hotspot-compiler
kulla
i18n
shenandoah
jdk
javadoc
2d
security
swing
hotspot-runtime
jmx
build
nio
beans
core-libs
compiler
net
hotspot-gc
hotspot-jfr
awt

mlbridge · 2021-01-08T02:15:57Z

Mailing list message from Yang Yi on hotspot-jfr-dev:

Happy New Year.

Can anyone help reviewing this patch? ;)

-------------
PR: https://git.openjdk.java.net/jdk/pull/1904

------------------------------------------------------------------
From:Yang Yi <github.com+5010047+kelthuzadx at openjdk.java.net>
Send Time:2021 Jan. 5 (Tue.) 10:55
To:hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>
Subject:RFR: 8256156: JFR: Allow 'jfr' tool to show metadata without a recording

Hi all,

May I please have a review for this minor patch?

It simplifies the use of subcommand `metadata` of jfr tool. Recording
file is no longer mandatory, users can directly use `jfr metadata` to
output JDK builtin meta information. As JDK-8256156 mentioned, it also
supports events and categories filters:
$ jfr metadata
$ jfr metadata recording.jfr # compatible
$ jfr metadata --events jdk.ThreadStart,jdk.ThreadEnd
$ jfr metadata --events jdk.ThreadStart,jdk.ThreadEnd recording.jfr
$ jfr metadata --categories GC,Detailed
$ jfr metadata --categories GC,Detailed recording.jfr

-------------

Commit messages:
- Allow jfr tool to show metadata without a recording

Changes: https://git.openjdk.java.net/jdk/pull/1904/files
Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1904&range=00
Issue: https://bugs.openjdk.java.net/browse/JDK-8256156
Stats: 151 lines in 5 files changed: 131 ins; 8 del; 12 mod
Patch: https://git.openjdk.java.net/jdk/pull/1904.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/1904/head:pull/1904

PR: https://git.openjdk.java.net/jdk/pull/1904

egahlin · 2021-01-08T04:15:05Z

src/jdk.jfr/share/classes/jdk/jfr/internal/tool/Metadata.java

-            } catch (IOException ioe) {
-                couldNotReadError(file, ioe);
+            }
+        } catch (Exception e) {


This error handling seems inadequate. If there is IOException, it still makes sense to get proper message, and not a stack trace

egahlin · 2021-01-08T04:16:50Z

src/jdk.jfr/share/classes/jdk/jfr/internal/tool/Metadata.java

+                    continue;
+                }
+                if (foundEventFilter) {
+                    if (acceptedEvents.contains(type.getName())) {


I would expect filtering for the metadata command to work like the 'print' command (accepting acronyms, wildcards etc.)

egahlin · 2021-01-08T04:17:16Z

src/jdk.jfr/share/classes/jdk/jfr/internal/tool/Metadata.java

+        if (categoryAnno != null) {
+            String[] categories = categoryAnno.value();
+            for (String category : categories) {
+                if (categoryNames.contains(category)) {


I would expect filtering for the metadata command to work like the 'print' command (wildcards etc.)

egahlin · 2021-01-08T04:18:58Z

src/jdk.jfr/share/classes/jdk/jfr/internal/tool/Metadata.java

+                    options.removeLast();
+                    return tmp;
+                } catch (Exception e) {
+                    // ignored since recording file for jfr metadata is optional


Inadequate error handling. If there is something wrong with the file (permission etc) I like to know it, and not get the event types for the current JVM.

It would not swallow the exception now.

egahlin · 2021-01-08T04:20:43Z

test/jdk/jdk/jfr/tool/TestMetadata.java

+                eventNames.add(line.substring(7, line.indexOf("\"", 7)));
+            }
+        }
+        List<Type> eventTypes = TypeLibrary.getInstance().getTypes();


I think it would be better to use public API (FlightRecorder.getEventTypes()) instead of calling internals.

egahlin · 2021-01-08T04:23:13Z

src/jdk.jfr/share/classes/jdk/jfr/internal/tool/Metadata.java

@@ -99,7 +106,11 @@ public String getName() {

    @Override


I think ine would like to get similar help as for the ''print' command when it comes to filtering.

egahlin · 2021-01-08T04:26:42Z

/csr

openjdk · 2021-01-08T04:27:16Z

@egahlin has indicated that a compatibility and specification (CSR) request is needed for this pull request.
@kelthuzadx please create a CSR request and add link to it in JDK-8256156. This pull request cannot be integrated until the CSR request is approved.

egahlin · 2021-02-16T19:17:49Z

It still doesn't work the same way as when you specify a file.

Here is a patch I think it will fix the functionality. Not tested or formatted.

diff --git a/src/jdk.jfr/share/classes/jdk/jfr/internal/tool/Metadata.java    
b/src/jdk.jfr/share/classes/jdk/jfr/internal/tool/Metadata.java
index 7a4b49703cf..1b6c50df441 100644
--- a/src/jdk.jfr/share/classes/jdk/jfr/internal/tool/Metadata.java
+++ b/src/jdk.jfr/share/classes/jdk/jfr/internal/tool/Metadata.java
@@ -39,6 +39,7 @@ import java.util.List;
import java.util.function.Predicate;

import jdk.jfr.EventType;
+import jdk.jfr.FlightRecorder;
import jdk.jfr.consumer.RecordingFile;
import jdk.jfr.internal.PlatformEventType;
import jdk.jfr.internal.PrivateAccess;
@@ -202,38 +203,40 @@ final class Metadata extends Command {
         if (filter != null) {
             filter = addCache(filter, type -> type.getId());
         }
-
-            List<Type> types = getAllTypes(file);
+           List<Type> types = findTypes(file);
             Collections.sort(types, new TypeComparator());
             for (Type type : types) {
-                // if it's an event type, apply filter on it
-                if (Type.SUPER_TYPE_EVENT.equals(type.getSuperType())) {
-                    EventType et = PrivateAccess.getInstance().newEventType((PlatformEventType) type);
-                    if (filter != null && !filter.test(et)) {
-                        continue;
+                if (filter != null) { 
+                    // If --events or --categories, only operate on events
+                    if (Type.SUPER_TYPE_EVENT.equals(type.getSuperType())) {
+                        EventType et = PrivateAccess.getInstance().newEventType((PlatformEventType) type);
+                        if (filter.test(et)) {
+                            prettyWriter.printType(type);
+                        }
                 }
+                } else {
+                    prettyWriter.printType(type);
             }
-
-                prettyWriter.printType(type);
         }
         prettyWriter.flush(true);
         pw.flush();
     }
 }

-    private List<Type> getAllTypes(Path file) throws UserDataException {
+    private List<Type> findTypes(Path file) throws UserDataException {
          // Determine whether reading from recording file or reading from the JDK where
          // the jfr tool is located will be used
          if (file == null) {
+            // Force initialization
+            FlightRecorder.getFlightRecorder().getEventTypes();
             return TypeLibrary.getInstance().getTypes();
          }
-        List<Type> types = null;
         try (RecordingFile rf = new RecordingFile(file)) {
-            types = PRIVATE_ACCESS.readTypes(rf);
+            return PRIVATE_ACCESS.readTypes(rf);
         } catch (IOException ioe) {
             couldNotReadError(file, ioe);
         }
-        return types;
+        return null; // Can't reach
     }

 private Path getOptionalJFRInputFile(Deque<String> options) throws UserDataException {

y1yang0 · 2021-02-18T02:25:03Z

Hi Erik, I have updated this patch according to your diff content.

I'm fine with changes about force initialization. But it looks like the changes in execute() method does the same thing? If no filter is given, they both print all types. If filter applied, they print only event type.

egahlin

I think it is easier to grasp what the code does without the continue statement.

egahlin · 2021-03-02T04:09:28Z

test/jdk/jdk/jfr/tool/TestMetadata.java

+        for (EventType eventType : eventTypes) {
+            expectedNames.add(eventType.getName());
+        }
+        Asserts.assertGTE(eventNames.size(), expectedNames.size());


Seems something like this would be sufficient:

public void testNumberOfEventTypes() { int count = 0; for (String line : output.asLines()) { if (line.contains("extends jdk.jfr.Event")) { count++; } Assert.assertEquals(count, FlightRecorder.getFlightRecorder().getEventTypes().size()); }

≈

egahlin · 2021-03-02T04:11:01Z

test/jdk/jdk/jfr/tool/TestMetadata.java

+        testWildcardAndAcronym();
+    }
+
+    static void testBasic() throws Throwable {


Perhaps change the name of the method to testUnfiltered() to make it more clear what the test does and create a separate method for the --wrongOption, i.e. testIllegalOption()

egahlin · 2021-03-02T04:12:56Z

test/jdk/jdk/jfr/tool/TestMetadata.java

+        Asserts.assertGTE(eventNames.size(), expectedNames.size());
+    }
+
+    static void testDeterministic() throws Throwable {


Change name of the method to testEventFilter().

egahlin · 2021-03-02T04:13:38Z

test/jdk/jdk/jfr/tool/TestMetadata.java

+        List<String> lines = output.asLines();
+
+        for (String line : lines) {
+            if (line.startsWith("@Name(\"")) {


Use contains "extends jdk.jfr.Event" as it is more stable

egahlin · 2021-03-02T04:15:23Z

test/jdk/jdk/jfr/tool/TestMetadata.java

+
+        for (String line : lines) {
+            if (line.startsWith("@Name(\"")) {
+                eventNames.add(line.substring(7, line.indexOf("\"", 7)));


Should be easy to check if the event is correct, not just the number.

if (line.contains("XXZQZYY.") { event1 = true; } if (line.contains("ZQZPP") { event2 = true; } Regarding events names, see following comment.

egahlin · 2021-03-02T04:25:45Z

test/jdk/jdk/jfr/tool/TestMetadata.java

+    }
+
+    static void testWildcardAndAcronym() throws Throwable {
+        OutputAnalyzer output = ExecuteHelper.jfr("metadata", "--events", "Thread*");


Instead of depending on name of existing JVM events (which may change, be removed or interfere with wildcard expansion done by the OS) it would be better to create two custom Java events with an unlikely name pattern, i.e. com.example.XXZQZYY and com.stuff.ZQZPP, and then filter on "ZQZ".

To make sure they are registered, you can do:

FlightRecorder.register(XXZQZYY.class); FlightRecorder.register(ZQZPP.class);

in static initializer so it is executed before calling ExecuteHelper.jfr("metadata", "--events", "ZQZ*");

Make sense, I re-implement the test, using customized events.

egahlin · 2021-03-02T04:31:36Z

test/jdk/jdk/jfr/tool/TestMetadata.java

+        Asserts.assertGTE(eventNames.size(), 2);
+    }
+
+    static void testWildcardAndAcronym() throws Throwable {


The name of the seems misleading since it doesn't test acronyms, testWildcard() shuld be sufficient.

egahlin

Looks good.

I still think the name of the test method should be called testWildcard() instead of testWildcardAndAcronym() since it is not testing acronyms, i.e using "GC" instead of "GarbageCollection", which is fine, but the name of the method should be appropriate.

openjdk · 2021-03-11T14:02:35Z

@kelthuzadx This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8256156: JFR: Allow 'jfr' tool to show metadata without a recording

Reviewed-by: egahlin

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 996 new commits pushed to the master branch:

43524cc: 8243455: Many SA tests can fail due to trying to get the stack trace of an active method
e834f99: 8263412: ClassFileInstaller can't be used by classes outside of default package
bf9b5fa: 8263501: compiler/oracle/TestInvalidCompileCommand.java fails with release VMs
0c8350e: 8263460: DynamicArchiveRelocationTest.java fails in product VM
b2f7c58: 8263055: hsdb Command Line Debugger does not properly direct output for some commands
ecfa712: 8263326: Remove ReceiverTypeData check from serviceability/sa/TestPrintMdo.java
b932a62: 8263470: Consolidate copies of getClassBytes in various tests
0ea48d9: 8194129: Regression automated Test '/open/test/jdk/java/awt/Window/ShapedAndTranslucentWindows/TranslucentChoice.java' fails
4b5c664: 8178348: left_n_bits(0) invokes undefined behavior
0b10c6b: 8263017: Read barriers are missing in nmethod printing code
... and 986 more: https://git.openjdk.java.net/jdk/compare/779ee1104ca7f506bd1113004a84745824a507d4...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@egahlin) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

y1yang0 · 2021-03-12T02:03:02Z

Thank Erik for your patient and reviews. Your naming suggestion makes sense, I forgot to rename that method in the previous commit. Now it has been corrected.

/integrate

openjdk · 2021-03-12T02:04:06Z

@kelthuzadx
Your change (at version a2cedbf9841a60222e727006e24309fd84da2a9c) is now ready to be sponsored by a Committer.

egahlin · 2021-03-12T20:45:14Z

/sponsor

openjdk · 2021-03-12T20:45:28Z

@egahlin The PR has been updated since the change author (@kelthuzadx) issued the integrate command - the author must perform this command again.

y1yang0 · 2021-03-13T00:31:08Z

/integrate

openjdk · 2021-03-13T00:32:04Z

@kelthuzadx
Your change (at version c401df4) is now ready to be sponsored by a Committer.

egahlin · 2021-03-13T02:02:25Z

/sponsor

openjdk · 2021-03-13T02:03:22Z

@egahlin @kelthuzadx Since your change was applied there have been 997 commits pushed to the master branch:

0b68ced: 8263548: runtime/cds/appcds/SharedRegionAlignmentTest.java fails to compile after JDK-8263412
43524cc: 8243455: Many SA tests can fail due to trying to get the stack trace of an active method
e834f99: 8263412: ClassFileInstaller can't be used by classes outside of default package
bf9b5fa: 8263501: compiler/oracle/TestInvalidCompileCommand.java fails with release VMs
0c8350e: 8263460: DynamicArchiveRelocationTest.java fails in product VM
b2f7c58: 8263055: hsdb Command Line Debugger does not properly direct output for some commands
ecfa712: 8263326: Remove ReceiverTypeData check from serviceability/sa/TestPrintMdo.java
b932a62: 8263470: Consolidate copies of getClassBytes in various tests
0ea48d9: 8194129: Regression automated Test '/open/test/jdk/java/awt/Window/ShapedAndTranslucentWindows/TranslucentChoice.java' fails
4b5c664: 8178348: left_n_bits(0) invokes undefined behavior
... and 987 more: https://git.openjdk.java.net/jdk/compare/779ee1104ca7f506bd1113004a84745824a507d4...master

Your commit was automatically rebased without conflicts.

Pushed as commit 86e4c75.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

bridgekeeper bot added the oca Needs verification of OCA signatory status label Dec 29, 2020

bridgekeeper bot added the oca-verify Needs verification of OCA signatory status label Dec 29, 2020

y1yang0 closed this Jan 5, 2021

y1yang0 reopened this Jan 5, 2021

bridgekeeper bot removed oca Needs verification of OCA signatory status oca-verify Needs verification of OCA signatory status labels Jan 5, 2021

openjdk bot added the rfr Pull request is ready for review label Jan 5, 2021

y1yang0 closed this Jan 5, 2021

y1yang0 reopened this Jan 5, 2021

openjdk bot added the hotspot-jfr hotspot-jfr-dev@openjdk.org label Jan 5, 2021

egahlin reviewed Jan 8, 2021

View reviewed changes

openjdk bot added the csr Pull request needs approved CSR before integration label Jan 8, 2021

y1yang0 force-pushed the JDK_8256156_impl branch from ca42a73 to ad96ede Compare January 8, 2021 09:24

openjdk bot removed the rfr Pull request is ready for review label Jan 8, 2021

y1yang0 force-pushed the JDK_8256156_impl branch from ad96ede to b3c2ea5 Compare January 8, 2021 09:27

openjdk bot added the rfr Pull request is ready for review label Jan 8, 2021

y1yang0 marked this pull request as draft January 8, 2021 09:31

fix

a26202f

openjdk bot removed the csr Pull request needs approved CSR before integration label Feb 24, 2021

egahlin reviewed Mar 2, 2021

View reviewed changes

reimpl test

ff13850

y1yang0 force-pushed the JDK_8256156_impl branch from 690f422 to ff13850 Compare March 6, 2021 11:40

egahlin approved these changes Mar 11, 2021

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Mar 11, 2021

openjdk bot added the sponsor Pull request is ready to be sponsored label Mar 12, 2021

rename

c401df4

y1yang0 force-pushed the JDK_8256156_impl branch from a2cedbf to c401df4 Compare March 12, 2021 13:06

openjdk bot removed the sponsor Pull request is ready to be sponsored label Mar 12, 2021

egahlin approved these changes Mar 12, 2021

View reviewed changes

openjdk bot added the sponsor Pull request is ready to be sponsored label Mar 13, 2021

openjdk bot closed this Mar 13, 2021

openjdk bot added integrated Pull request has been integrated and removed sponsor Pull request is ready to be sponsored ready Pull request is ready to be integrated rfr Pull request is ready for review labels Mar 13, 2021

y1yang0 deleted the JDK_8256156_impl branch March 14, 2021 13:03

Conversation

y1yang0 commented Dec 29, 2020 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Download

Uh oh!

bridgekeeper bot commented Dec 29, 2020

Uh oh!

y1yang0 commented Dec 29, 2020

Uh oh!

bridgekeeper bot commented Dec 29, 2020

Uh oh!

y1yang0 commented Jan 5, 2021

Uh oh!

bridgekeeper bot commented Jan 5, 2021

Uh oh!

y1yang0 commented Jan 5, 2021

Uh oh!

openjdk bot commented Jan 5, 2021

Uh oh!

y1yang0 commented Jan 5, 2021

Uh oh!

mlbridge bot commented Jan 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

openjdk bot commented Jan 5, 2021

Uh oh!

mlbridge bot commented Jan 8, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

egahlin commented Jan 8, 2021

Uh oh!

openjdk bot commented Jan 8, 2021

Uh oh!

egahlin commented Feb 16, 2021

Uh oh!

y1yang0 commented Feb 18, 2021

Uh oh!

egahlin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

y1yang0 commented Dec 29, 2020 •

edited by openjdk bot

Loading

mlbridge bot commented Jan 5, 2021 •

edited

Loading

openjdk bot commented Mar 11, 2021 •

edited

Loading