8316679: C2 SuperWord: wrong result, load should not be moved before store if not comparable

[eme64]

Context
In SuperWord::output, we replace the scalars with vectors, pack by pack.

In JDK-8052081 (changeset) we added an optimization to move the memory state of a LoadVector as early as possible. We go up the memory chain, and skip any StoreVector that provably can never alias with the LoadVector (provably no overlap).

Details

For this, we must understand the following function VPointer::cmp:

jdk/src/hotspot/share/opto/vectorization.hpp

Lines 110 to 120 in 96781ba

	int cmp(VPointer& q) {
	if (valid() && q.valid() &&
	(_adr == q._adr || (_base == _adr && q._base == q._adr)) &&
	_scale == q._scale && invar_equals(q)) {
	bool overlap = q._offset < _offset + memory_size() &&
	_offset < q._offset + q.memory_size();
	return overlap ? Equal : (_offset < q._offset ? Less : Greater);
	} else {
	return NotComparable;
	}
	}

We have 4 possible states: NotComparable means that we cannot determine the relative offsets between the two memory addresses. We have no guarantee about aliasing at all. Less means that one address lies before the other, Greater means that it comes after. Equal means that the two memory regions have an overlap or are identical. To determine overlap, we must know the position of the pointer, as well as the memory_size of each memory reference. VPointer::not_equal is true iff we get states Less or Greater (we know there cannot be overlap). VPointer::comparable is true iff the state is NotComparable, hence we have no guarantee about aliasing.

Problem

Currently, there are 2 bugs in this "move-load-before-store" logic:

1. JDK-8316679 (PR): We do not only skip the not_equal but also the !comparable case. However, when the two pointers are not comparable, that could be due to different invariants for example (see TestMovingLoadBeforeStore.java). The two pointers are not comparable, because we do not know the relative position, due to the invariants. But this exactly means that we cannot know at compile-time that there will not be aliasing. Thus, we cannot allow moving a load before such a store. Otherwise we may get wrong results. (*)
2. JDK-8316594 (PR): Currently, we only compare the first element of the load-pack with the StoreVector. This is not correct, we need to compare all elements of the load-pack with the StoreVector. Otherwise it may be that the first element lays below/before the memory region of the StoreVector (Less), but the last element of the load-pack for example could have overlap with the StoreVector (Equal).

We could fix both issues at once. But I am fixing them separately, first JDK-8316679, which can be backported to JDK-11, and then JDK-8316594 which only seems to have caused regressions in JDK-22.

Testing

I added correpsonding regression tests.
Tier1-6 + stress-testing.
Performance testing.

---

(*) We also use !not_equal in SuperWord::dependence_graph to determine if two memrefs need to have a dependency edge. The condition there and the condition here (for move-load-before-store) should be equivalent.

jdk/src/hotspot/share/opto/superword.cpp

Lines 1112 to 1116 in 96781ba

	if (!VPointer::not_equal(cmp)) {
	// Possibly same address
	_dg.make_edge(s1, s2);
	sink_dependent = false;
	}

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8316679: C2 SuperWord: wrong result, load should not be moved before store if not comparable (Bug - P3)

Reviewers

• Vladimir Kozlov (@vnkozlov - Reviewer) ⚠️ Review applies to 8630fb80
• Tobias Hartmann (@TobiHartmann - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15864/head:pull/15864
$ git checkout pull/15864

Update a local copy of the PR:
$ git checkout pull/15864
$ git pull https://git.openjdk.org/jdk.git pull/15864/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15864

View PR using the GUI difftool:
$ git pr show -t 15864

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15864.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back epeter! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@eme64 The following label will be automatically applied to this pull request:

• hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 02: Full - Incremental (5b909dbd)
• 01: Full - Incremental (32be7ca0)
• 00: Full (8630fb80)

[vnkozlov]

Good.

[openjdk]

@eme64 This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8316679: C2 SuperWord: wrong result, load should not be moved before store if not comparable

Reviewed-by: kvn, thartmann

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[vnkozlov]

Thank you for detailed explanation.
What about case if one array is locally allocated and one comes from outside - compiler knows they are different. But with this change loads will not be moved. I understand that that if both comes from outside (as arguments or static reference) we can't guarantee that they are different. But if at least one of them is local and did not escape, we know they are different and can move pointers.
May be it is separate RFE.

[eme64]

@vnkozlov The issue with knowing if different array references can alias (if they are the same) is something that is currently not implemented for SuperWord. But it could theoretically be solved outside of SuperWord, by putting them on separate memory slices. I think this is definately a separate RFE and would not just benefit SuperWord. Alternatively, we could do special logic inside SuperWord, determining if references can be proven not to be the same. Or even specualte / trap. The last one would be powerful, and probably the most beneficial in general. In most cases different array references are actually different arrays, but we cannot know that at compile time, because they are for example two arguments to a method.
What do you think?

[vnkozlov]

I like the idea to add loop predicate to check arrays difference. It can be folded by compiler based on EA analysis: we were talking for long time to add instance ID for not global escaping arrays regardless their size. And yes, we can do that outside SuperWord.

[TobiHartmann]

Thanks for the nice summary. The fix looks good to me.

[TobiHartmann]

The test should have @key randomness

[TobiHartmann]

You might want to remove the extra whitespaces in the index expressions.

[eme64]

Thanks @TobiHartmann @vnkozlov for the reviews!
/integrate

[openjdk]

Going to push as commit 48f1a92.
Since your change was applied there have been 15 commits pushed to the master branch:

• 0b0f8b5: 8219652: [aix] Tests failing with JNI attach problems.
• 8c0d026: 8315042: NPE in PKCS7.parseOldSignedData
• f7deaf4: 8316778: test hprof lib: invalid array element type from JavaValueArray.elementSize
• 8ff10a0: 8317446: ProblemList gc/arguments/TestNewSizeFlags.java on macosx-aarch64 in Xcomp
• 1809b8c: 8317265: ListFormat::format specification could be made clearer regarding handling IllegalArgumentException.
• cfabcbf: 8317121: vector_masked_load instruction is moved too early after JDK-8286941
• b859da9: 8316696: Remove the testing base classes: IntlTest and CollatorTest
• b438cff: 8314085: Fixing scope from benchmark to thread for JMH tests having shared state
• ae796a4: 8316923: Add DEF_STATIC_JNI_OnLoad for librmi
• 89987db: 8303773: Replace "main.wrapper" with "test.thread.factory" property in test code
• ... and 5 more: https://git.openjdk.org/jdk/compare/287b24322135b54641f013970c4545ce069c4350...master

Your commit was automatically rebased without conflicts.

[openjdk]

@eme64 Pushed as commit 48f1a92.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.