8324641: [IR Framework] Add Setup method to provide custom arguments and set fields

[eme64]

Bigger goal

I am tired of always writing IR tests where I have to write elaborate code in the @Run method to create inputs, and then run the test method to create gold output values in (hopefully) interpreter mode, and then later verify the results of the compiled method.

Hence, I now introduce the "Setup" method, which can create custom argument values.
In a later RFE, I will implement automatic result verification, which implicitly intercepts the inputs and outputs of the test method, and compares the behaviour of the interpreter and the compiled code. That way, the pattern will be:

@Setup
... specify your arguments and fields ...

-> intercept arguments, cache them

@Test
... write a test with arbitrary inputs and outputs...

-> interpreter mode: intercept outputs and cache them (i.e. gold values)
-> compiled mode: intercept outputs and compare them to the gold values.

(optional)
@Check
.. do custom verification...

In this RFE: the Setup Method

The first step is the setup method.

Example:

jdk/test/hotspot/jtreg/testlibrary_tests/ir_framework/examples/SetupExample.java

Lines 49 to 87 in 08670a5

	// Test with static setup, test and check method.
	@Setup
	static Object[] setupTwoIntArrays() {
	int[] a = new int[10_000];
	int[] b = new int[10_000];
	for (int i = 0; i < a.length; i++) {
	a[i] = i - 2;
	b[i] = i + 2;
	}
	return new Object[]{a, b}; // passed as arguments to test method
	}
	@Test
	@Arguments(setup = "setupTwoIntArrays")
	static Object[] testWithSetupRandomIntArray(int[] a, int[] b) {
	for (int i = 0; i < a.length; i++) {
	int aa = a[i];
	int bb = b[i];
	a[i] = aa + bb;
	b[i] = aa - bb;
	}
	return new Object[]{a, b}; // passed as argument to check method
	}
	@Check(test = "testWithSetupRandomIntArray")
	void checkTestWithSetupRandomIntArray(Object[] args) {
	int[] a = (int[])args[0]; // parse return values of test method
	int[] b = (int[])args[1];
	if (a.length != 10_000 || b.length != 10_000) {
	throw new RuntimeException("bad length");
	}
	for (int i = 0; i < a.length; i++) {
	if ((a[i] != 2 * i) || (b[i] != -4)) {
	throw new RuntimeException("bad value: " + i + ": " + a[i] + " " + b[i]);
	}
	}
	}

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8324641: [IR Framework] Add Setup method to provide custom arguments and set fields (Sub-task - P4)

Reviewers

• Christian Hagedorn (@chhagedorn - Reviewer)
• Tobias Hartmann (@TobiHartmann - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/17557/head:pull/17557
$ git checkout pull/17557

Update a local copy of the PR:
$ git checkout pull/17557
$ git pull https://git.openjdk.org/jdk.git pull/17557/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 17557

View PR using the GUI difftool:
$ git pr show -t 17557

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/17557.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back epeter! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@eme64 The following label will be automatically applied to this pull request:

• hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 19: Full - Incremental (1f7872ad)
• 18: Full - Incremental (617e40a1)
• 17: Full - Incremental (47b8d253)
• 16: Full - Incremental (f03f3b84)
• 15: Full - Incremental (33029f68)
• 14: Full - Incremental (4a8f96b9)
• 13: Full - Incremental (d10fba5b)
• 12: Full - Incremental (252815ed)
• 11: Full - Incremental (6efd271d)
• 10: Full - Incremental (b37b7480)
• 09: Full - Incremental (af5e7dc1)
• 08: Full - Incremental (a1dda565)
• 07: Full - Incremental (12c9a452)
• 06: Full - Incremental (164665fa)
• 05: Full - Incremental (8ff7dfac)
• 04: Full - Incremental (fddcdfef)
• 03: Full - Incremental (66c3ef71)
• 02: Full - Incremental (852e6e16)
• 01: Full - Incremental (1237c028)
• 00: Full (24eb09d2)

[chhagedorn]

That's a nice feature! I have a few comments.

As already discussed offline, I think we should clean up the entire code in the test VM and group verification code together to avoid spreading it all over the place. But this should not block this PR and should be done separately.

[chhagedorn]

Thanks for the updates. I will have a closer look again at the README tomorrow but the code changes look good.

What's missing are some good and bad tests that use the new features. SetupExample.java already provides some good working examples and maybe that's sufficient. Maybe you can double check if it's worth to add some more good (i.e. non-failing) tests.

The bad tests can be added to TestBadFormat which should include things like a @Setup test without a corresponding @Test method, mixing setup and value etc. Can you add some of these bad tests?

[eme64]

@chhagedorn I have added quite a few tests now, and fixed a few bugs with them 😊
Would you mind reviewing again?

[chhagedorn]

Thanks for adding all these tests! I have some suggestions but only minor things and some more bad test ideas (if I did not miss that you already tested that).

[chhagedorn]

That was hard to spot as we've discussed offline :-)

[eme64]

I'm glad I fixed it!
Thanks for the help when debugging!

[chhagedorn]

Thanks for all the updates - looks good!

[openjdk]

@eme64 This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8324641: [IR Framework] Add Setup method to provide custom arguments and set fields

Reviewed-by: chagedorn, thartmann

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[TobiHartmann]

Very nice. The changes look good to me.

[eme64]

Thanks @chhagedorn for all the discussions, brainstorming etc, and the detailed review 😊
Thanks @TobiHartmann for the review!
/integrate

[openjdk]

Going to push as commit 8d9ad97.
Since your change was applied there have been 4 commits pushed to the master branch:

• b797652: 8322927: Unused code in LIR_Assembler::verify_oop_map
• 9936aee: 8324824: AArch64: Detect Ampere-1B core and update default options for Ampere CPUs
• d91fb17: 8325505: Fix Javadoc ResourceBundle::getString
• 10beb31: 8325456: Rename nsk_mutex.h

Your commit was automatically rebased without conflicts.

[openjdk]

@eme64 Pushed as commit 8d9ad97.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.