8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file) #18824

caojoshua · 2024-04-17T18:44:57Z

The bug occurs when Shenandoah optimizations resets post_loop_opts, and we may create a MaxL after macro expansion. MaxL does not have a matcher rule, and we run into an assertion failure.

This PR guards the MaxL creation with a new began_macro_expansion() flag. I think there are many other instances in code that should use the new flag instead of post_loop_opts(), which can be explored in JDK-8330531.

The bug was originally found in h2 Index::getCostRangeIndex() through Dacapo. Its easy to reproduce by creating a loop that includes a ShenandoahLoadReferenceBarrier (load any object) and a MaxL.

Caveat: I created test cases for both MaxL and MinL for completeness. The MinL test case does not actually fail before this PR. Somehow the CMove condition is converted to non-canonical >, which is not accepted by the Idealization. The MinL is never created and there is no crash.

Passing hotspot tier1 locally on Linux machine.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

JDK-8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file) (Bug - P3)

Reviewers

Aleksey Shipilev (@shipilev - Reviewer) ⚠️ Review applies to 35cff12c
Tobias Hartmann (@TobiHartmann - Reviewer) ⚠️ Review applies to 35cff12c

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/18824/head:pull/18824
$ git checkout pull/18824

Update a local copy of the PR:
$ git checkout pull/18824
$ git pull https://git.openjdk.org/jdk.git pull/18824/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 18824

View PR using the GUI difftool:
$ git pr show -t 18824

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/18824.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2024-04-17T18:45:23Z

👋 Welcome back caojoshua! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2024-04-17T18:46:04Z

@caojoshua This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file)

Reviewed-by: shade, thartmann

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@shipilev, @TobiHartmann) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

openjdk · 2024-04-17T18:46:32Z

@caojoshua The following label will be automatically applied to this pull request:

hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2024-04-17T18:49:26Z

Webrevs

shipilev

Good catch!

shipilev · 2024-04-19T08:37:05Z

src/hotspot/share/opto/compile.hpp

@@ -318,6 +318,7 @@ class Compile : public Phase {
  uintx                 _max_node_limit;        // Max unique node count during a single compilation.

  bool                  _post_loop_opts_phase;  // Loop opts are finished.
+  bool                  _began_macro_expansion; // Macro expansion is started.


I think this sounds better as inverse, bool _allow_macro_nodes; // Allow creating macro nodes. Then we can also assert _allow_macro_nodes in add_macro_node?

Can't add the assert for free. Apparently macro nodes are added later in the compiler pipeline, Matcher in this case.

Stack: [0x00007fb2c6d50000,0x00007fb2c6e51000], sp=0x00007fb2c6e4caa0, free space=1010k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x63bd9d] Compile::add_macro_node(Node*)+0x49 (compile.hpp:742) V [libjvm.so+0x124ccc5] Node::clone() const+0x15f (node.cpp:501) V [libjvm.so+0x119a93a] Matcher::xform(Node*, int)+0x3a2 (matcher.cpp:1150) V [libjvm.so+0x1195413] Matcher::match()+0xe8b (matcher.cpp:359) V [libjvm.so+0x9a9723] Compile::Code_Gen()+0x95 (compile.cpp:2947) V [libjvm.so+0x99fd8d] Compile::Compile(ciEnv*, ciMethod*, int, Options, DirectiveSet*)+0x1827 (compile.cpp:895)

Eh? We do Compile::add_macro_node from Code_Gen while cloning the nodes during matching, which means we add to _macro_nodes unnecessarily? Same for expensive nodes. That's unfortunate...

shipilev · 2024-04-19T08:38:45Z

test/hotspot/jtreg/compiler/c2/irTests/TestIfMinMax.java

 */
 public class TestIfMinMax {
    private static final Random RANDOM = Utils.getRandomInstance();

    public static void main(String[] args) {
        TestFramework.run();
+        TestFramework.runWithFlags("-XX:+UseShenandoahGC");


I don't think we add GC-specific testing here. For one, the test would fail for the builds that do not include Shenandoah.

The common practice it to rely on test pipelines running the test suites with different GCs. Does make test TEST=compiler/c2/irTests/TestIfMinMax.java TEST_VM_OPTS=-XX:+UseShenandoahGC work?

Yes, it works. Confirms it can reproduce the crash. Will remove that line.

flag from test.

caojoshua · 2024-04-19T16:41:17Z

@shipilev updated based on your comments and also updated the comment in CMoveNode::Ideal_minmax

shipilev · 2024-04-19T16:49:53Z

src/hotspot/share/opto/compile.hpp

+  bool       allow_macro_nodes() { return _allow_macro_nodes;  }
+  void  dont_allow_macro_nodes() { _allow_macro_nodes = false;  }


dont_allow_macro_nodes is a confusing name here, especially given it is a setter in contrast to allow_macro_nodes. Let's call it reset_allow_macro_nodes().

shipilev · 2024-04-19T16:50:49Z

src/hotspot/share/opto/macro.cpp

@@ -2445,6 +2445,7 @@ void PhaseMacroExpand::eliminate_macro_nodes() {
 //------------------------------expand_macro_nodes----------------------
 //  Returns true if a failure occurred.
 bool PhaseMacroExpand::expand_macro_nodes() {
+  C->dont_allow_macro_nodes();


Leave a comment here:

// Do not allow new macro nodes once we started to expand

dont_allow_macro_nodes to reset_allow_macro_nodes.

shipilev

I am good with this version, but of course someone from compiler team needs to take a look.

/reviewers 2

openjdk · 2024-04-19T17:13:28Z

@shipilev
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

TobiHartmann

Looks good to me. I submitted testing and will report back once it passed.

Please adjust the description of JDK-8330531 according to the new naming.

Somehow the CMove condition is converted to non-canonical >, which is not accepted by the Idealization. The MinL is never created and there is no crash.

Is the problem that the condition is not canonicalized or that the CMoveNode is not process by IGVN after canonicalization of the cmp?

caojoshua · 2024-04-22T17:02:00Z

Is the problem that the condition is not canonicalized or that the CMoveNode is not process by IGVN after canonicalization of the cmp?

The CMoveNode is processed, but its input Bool and Cmp are never processed. Maybe we need to transform the CMove's inputs in

jdk/src/hotspot/share/opto/loopopts.cpp

Line 842 in 0b9350e

    
           Node *cmov = CMoveNode::make(cmov_ctrl, iff->in(1), phi->in(1+flip), phi->in(2-flip), _igvn.type(phi));

.

TobiHartmann · 2024-04-23T08:07:17Z

All tests passed.

The CMoveNode is processed, but its input Bool and Cmp are never processed. Maybe we need to transform the CMove's inputs in

Maybe verify if a _igvn._worklist.push(...) helps.

caojoshua · 2024-04-25T04:36:19Z

Correction: The Bool does get processed, but does not get transformed. BoolNode::Ideal actually does not canonicalize bools. They are canonicalized in IfNode::idealize_test. Created JDK-8331090 to canonicalize CMov tests.

caojoshua · 2024-04-25T04:41:16Z

/integrate

openjdk · 2024-04-25T04:42:03Z

@caojoshua
Your change (at version 851b21c) is now ready to be sponsored by a Committer.

TobiHartmann · 2024-04-25T07:10:56Z

Thanks for the additional details and filing JDK-8331090!

TobiHartmann · 2024-04-25T07:11:00Z

/sponsor

openjdk · 2024-04-25T07:11:13Z

Going to push as commit d32f109.
Since your change was applied there have been 3 commits pushed to the master branch:

ccc0d0f: 8325373: Improve StackCounter error reporting for bad switch cases
e818ab6: 8330815: Use pattern matching for instanceof in KeepAliveCache
d43654e: 8331030: langtools/tools/javac/tree tests fail with SOE with fastdebug and -Xcomp

Your commit was automatically rebased without conflicts.

openjdk · 2024-04-25T07:11:19Z

@TobiHartmann @caojoshua Pushed as commit d32f109.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file)

ebfaf34

openjdk bot added the rfr Pull request is ready for review label Apr 17, 2024

openjdk bot added the hotspot-compiler hotspot-compiler-dev@openjdk.org label Apr 17, 2024

shipilev reviewed Apr 19, 2024

View reviewed changes

caojoshua added 3 commits April 19, 2024 17:34

Merge branch 'master' into shen

c253ad7

Rename began_macro_expansion to allow_macro_nodes. Remove shenandoah

4cacffe

flag from test.

Update comment for MinL/MaxL based on renaming of allow_macro_nodes

88c3c1d

shipilev reviewed Apr 19, 2024

View reviewed changes

Comment on not allowing macro nodes after we start expanding. Rename

35cff12

dont_allow_macro_nodes to reset_allow_macro_nodes.

shipilev approved these changes Apr 19, 2024

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Apr 19, 2024

openjdk bot removed the ready Pull request is ready to be integrated label Apr 19, 2024

TobiHartmann approved these changes Apr 22, 2024

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Apr 22, 2024

caojoshua added 2 commits April 25, 2024 06:31

Merge branch 'master' into shen

4a4c32c

Merge branch 'master' into shen

851b21c

openjdk bot added the sponsor Pull request is ready to be sponsored label Apr 25, 2024

openjdk bot added the integrated Pull request has been integrated label Apr 25, 2024

openjdk bot closed this Apr 25, 2024

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review sponsor Pull request is ready to be sponsored labels Apr 25, 2024

jaskarth mentioned this pull request Jun 26, 2024

8331090: Run Ideal_minmax before de-canonicalizing CMoves #19914

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file) #18824

8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file) #18824

caojoshua commented Apr 17, 2024 •

edited by openjdk bot

Loading

bridgekeeper bot commented Apr 17, 2024

openjdk bot commented Apr 17, 2024 •

edited

Loading

openjdk bot commented Apr 17, 2024

mlbridge bot commented Apr 17, 2024 •

edited

Loading

shipilev left a comment

shipilev Apr 19, 2024

caojoshua Apr 19, 2024

shipilev Apr 19, 2024 •

edited

Loading

shipilev Apr 19, 2024

caojoshua Apr 19, 2024

caojoshua commented Apr 19, 2024

shipilev Apr 19, 2024

shipilev Apr 19, 2024

shipilev left a comment

openjdk bot commented Apr 19, 2024

TobiHartmann left a comment

caojoshua commented Apr 22, 2024

TobiHartmann commented Apr 23, 2024

caojoshua commented Apr 25, 2024 •

edited

Loading

caojoshua commented Apr 25, 2024

openjdk bot commented Apr 25, 2024

TobiHartmann commented Apr 25, 2024

TobiHartmann commented Apr 25, 2024

openjdk bot commented Apr 25, 2024

openjdk bot commented Apr 25, 2024

		bool allow_macro_nodes() { return _allow_macro_nodes; }
		void dont_allow_macro_nodes() { _allow_macro_nodes = false; }

8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file) #18824

8329797: Shenandoah: Default case invoked for: "MaxL" (bad AD file) #18824

Conversation

caojoshua commented Apr 17, 2024 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

Webrev

bridgekeeper bot commented Apr 17, 2024

openjdk bot commented Apr 17, 2024 • edited Loading

openjdk bot commented Apr 17, 2024

mlbridge bot commented Apr 17, 2024 • edited Loading

Webrevs

shipilev left a comment

Choose a reason for hiding this comment

shipilev Apr 19, 2024

Choose a reason for hiding this comment

caojoshua Apr 19, 2024

Choose a reason for hiding this comment

shipilev Apr 19, 2024 • edited Loading

Choose a reason for hiding this comment

shipilev Apr 19, 2024

Choose a reason for hiding this comment

caojoshua Apr 19, 2024

Choose a reason for hiding this comment

caojoshua commented Apr 19, 2024

shipilev Apr 19, 2024

Choose a reason for hiding this comment

shipilev Apr 19, 2024

Choose a reason for hiding this comment

shipilev left a comment

Choose a reason for hiding this comment

openjdk bot commented Apr 19, 2024

TobiHartmann left a comment

Choose a reason for hiding this comment

caojoshua commented Apr 22, 2024

TobiHartmann commented Apr 23, 2024

caojoshua commented Apr 25, 2024 • edited Loading

caojoshua commented Apr 25, 2024

openjdk bot commented Apr 25, 2024

TobiHartmann commented Apr 25, 2024

TobiHartmann commented Apr 25, 2024

openjdk bot commented Apr 25, 2024

openjdk bot commented Apr 25, 2024

caojoshua commented Apr 17, 2024 •

edited by openjdk bot

Loading

openjdk bot commented Apr 17, 2024 •

edited

Loading

mlbridge bot commented Apr 17, 2024 •

edited

Loading

shipilev Apr 19, 2024 •

edited

Loading

caojoshua commented Apr 25, 2024 •

edited

Loading