Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8287525: Extend IR annotation with new options to test specific target feature. #8999

Closed
wants to merge 6 commits into from

Conversation

swati-sha
Copy link
Contributor

@swati-sha swati-sha commented Jun 2, 2022

Hi All,

Currently test invocations are guarded by @requires vm.cpu.feature tags which are specified as the part of test tag specifications. This results into generating multiple test cases if some test points in a test file needs to be guarded by a specific features while others should still be executed in absence of missing target feature.

This is specially important for IR checks based validation since C2 IR nodes creation may heavily rely on existence of specific target feature. Also, test harness executes test points only if all the constraints specified in tag specifications are met, thus imposing an OR semantics b/w @requires tag based CPU features becomes tricky.

Patch extends existing @ir annotation with following two new options:-

  • applyIfCPUFeatureAnd:
    Accepts a list of feature pairs where each pair is composed of target feature string followed by a true/false value where a true value necessities existence of target feature and vice-versa. IR verifications checks are enforced only if all the specified feature constraints are met.
  • applyIfCPUFeatureOr: Accepts similar arguments as above option but IR verifications checks are enforced only when at least one of the specified feature constraints are met.

Example usage:
@ir(counts = {IRNode.ADD_VI, "> 0"}, applyIfCPUFeatureOr = {"avx512bw", "true", "avx512f", "true"})
@ir(counts = {IRNode.ADD_VI, "> 0"}, applyIfCPUFeatureAnd = {"avx512bw", "true", "avx512f", "true"})

Please review and share your feedback.

Thanks,
Swati


Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue

Issue

  • JDK-8287525: Extend IR annotation with new options to test specific target feature.

Reviewers

Contributors

  • Jatin Bhateja <jbhateja@openjdk.org>

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/8999/head:pull/8999
$ git checkout pull/8999

Update a local copy of the PR:
$ git checkout pull/8999
$ git pull https://git.openjdk.org/jdk pull/8999/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 8999

View PR using the GUI difftool:
$ git pr show -t 8999

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/8999.diff

@swati-sha
Copy link
Contributor Author

/label add hotspot-compiler-dev

@bridgekeeper
Copy link

bridgekeeper bot commented Jun 2, 2022

👋 Welcome back swati-sha! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added rfr Pull request is ready for review hotspot-compiler hotspot-compiler-dev@openjdk.org labels Jun 2, 2022
@openjdk
Copy link

openjdk bot commented Jun 2, 2022

@swati-sha
The hotspot-compiler label was successfully added.

@mlbridge
Copy link

mlbridge bot commented Jun 2, 2022

Webrevs

@swati-sha
Copy link
Contributor Author

/contributor add @jatin-bhateja

@openjdk
Copy link

openjdk bot commented Jun 2, 2022

@swati-sha
Contributor Jatin Bhateja <jbhateja@openjdk.org> successfully added.

Comment on lines +109 to +110
* Accepts a list of feature pairs where each pair is composed of target feature string followed by a true/false
* value where a true value necessities existence of target feature and vice-versa.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need to repeat the same 2 lines.

* value where a true value necessities existence of target feature and vice-versa.
* IR verifications checks are enforced only if all the specified feature constraints are met.
*/
String[] applyIfTargetFeatureAnd() default {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why you used Target instead of original CPU (you check output of getCPUFeatures() only)?
Do you plan to extend this to check flags too in a future?

Copy link
Member

@chhagedorn chhagedorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good feature to have. Thanks for adding support for it! This will definitely simplify the vector IR tests.

String value = andRules[i];
TestFormat.check((value.contains("true") || value.contains("false")), "Incorrect value in " + ruleType + failAt());
if (!checkTargetFeature(feature, value)) {
// Rule will not be applied but keep processing the other flags to verify that they are same.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo:

Suggested change
// Rule will not be applied but keep processing the other flags to verify that they are same.
// Rule will not be applied but keep processing the other target features to verify that they are sane.

}

@Test
@IR(counts = {"AddVI", "> 0"}, applyIfTargetFeatureAnd = {"avx512bw", "false"})
Copy link
Member

@chhagedorn chhagedorn Jun 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this file represents a usage example, please consider using IRNode.AddVI by adding a new IR regex to IRNode. But it looks like that this test is rather a correctness test for AddVI and could be moved to the other vector IR tests. The tests in the ir_framework.examples package are more about providing information about the usage rather than actually testing something meaningful.

But I think it's good to have an example how to use applyIfTargetFeature* as well. But maybe such an example would better fit into the existing IRExample.java file where we have existing IR examples and descriptions for applyIf*.

if (irAnno.applyIfTargetFeatureAnd().length != 0) {
boolean check = hasRequiredFeaturesAnd(irAnno.applyIfTargetFeatureAnd(), "applyIfTargetFeatureAnd");
if (!check) {
System.out.println("Disabling IR validation for " + m + ", all feature constraints not met.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this message will be printed inside the test VM together with a lot of other messages (-XX:+PrintCompilation etc.). This output will not be shown by default. So, it will be hard to find this message again and it might not provide an additional value. For printing log messages inside the test VM, you can use the TestFrameworkSocket which pipes messages to the JTreg driver VM to print them there. Since the JTreg driver VM only does a minimal printing, it can easily be found again towards the end of the output under "Messages from Test VM". Specify a tag and then you can use it like that:

TestFrameworkSocket.write("Disabling IR matching for " + m + ": Not all feature constraints met.", 
                          "[IREncodingPrinter]", true);

JTreg Output:

STDOUT:
Run Flag VM:
[...]

Messages from Test VM
---------------------
[IREncodingPrinter] Disabling IR matching for test2: Could not match all feature constraints.

[...]

I guess we could use the same kind of messages for the other applyIf* methods above. If you want to add them as well, feel free to do so. Otherwise, this could also be done separately in an RFE at some point.

@@ -137,6 +154,16 @@ private void checkIRAnnotations(IR irAnno) {
TestFormat.checkNoThrow(irAnno.applyIf().length <= 2,
"Use applyIfAnd or applyIfOr or only 1 condition for applyIf" + failAt());
}
if (irAnno.applyIfTargetFeatureAnd().length != 0) {
applyRules++;
TestFormat.checkNoThrow((irAnno.applyIfTargetFeatureAnd().length & 1) == 0,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to use:

Suggested change
TestFormat.checkNoThrow((irAnno.applyIfTargetFeatureAnd().length & 1) == 0,
TestFormat.checkNoThrow((irAnno.applyIfTargetFeatureAnd().length % 2) == 0,

instead which seems cleaner. Same for the other check below.

Comment on lines 254 to 257
if ((value.contains("true") && s.contains(feature)) ||
(value.contains("false") && !s.contains(feature))) {
return true;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be simplified to:

Suggested change
if ((value.contains("true") && s.contains(feature)) ||
(value.contains("false") && !s.contains(feature))) {
return true;
}
return (value.contains("true") && s.contains(feature)) || (value.contains("false") && !s.contains(feature));

TestFormat.check((value.contains("true") || value.contains("false")), "Incorrect value in " + ruleType + failAt());
if (!checkTargetFeature(feature, value)) {
// Rule will not be applied but keep processing the other flags to verify that they are same.
return false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should cache the return value to keep processing the remaining target feature user strings to check for format errors (similar to what you are doing in hasRequiredFeaturesOr()). We should probably separate the format checking and the actual evaluation of the values at some point. But that would exceed the scope of this RFE.

@@ -176,6 +203,62 @@ private boolean hasAllRequiredFlags(String[] andRules, String ruleType) {
return returnValue;
}

private boolean hasRequiredFeaturesAnd(String[] andRules, String ruleType) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ruleType is always applyIfTargetFeatureAnd and can be replaced as such.

Suggestion for method name: hasAllRequiredTargetFeatures() to follow the existing naming scheme.

}

private boolean checkTargetFeature(String feature, String value) {
String s = WHITE_BOX.getCPUFeatures();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to name it cpuFeatures instead:

Suggested change
String s = WHITE_BOX.getCPUFeatures();
String cpuFeatures = WHITE_BOX.getCPUFeatures();

return true;
}

private boolean hasRequiredFeaturesOr(String[] orRules, String ruleType) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ruleType is always applyIfTargetFeatureOr and can be replaced as such.

Suggestion for method name: hasAnyRequiredTargetFeature() (I think the related existing hasNoRequiredFlags() method should be renamed/refactored to follow that convention as well but that's also for another day).

* value where a true value necessities existence of target feature and vice-versa.
* IR verifications checks are enforced if any of the specified feature constraint is met.
*/
String[] applyIfTargetFeatureOr() default {};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we should follow the existing scheme to also have at least applyIfTargetFeature for a single constraint or not. Back there when I've introduced these constraints I was not happy with writing applyIfAnd/Or for many tests where I actually did not care about AND and OR. I guess you can leave it like that and we can come back to this and maybe clean these things up with JDK-8280120 which wants to introduce another attribute to filter based on the architecture.

@openjdk
Copy link

openjdk bot commented Jun 8, 2022

⚠️ @swati-sha This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

TestFormat.checkNoThrow((irAnno.applyIfTargetFeatureOr().length & 1) == 0,
"Argument count for applyIfTargetFeatureOr should be multiple of two" + failAt());
TestFormat.checkNoThrow((irAnno.applyIfCPUFeatureOr().length % 2) == 0,
"Argument count for applyIfCPUFeatureOr should be multiple of two" + failAt());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The format check on L208 cannot be used like that anymore (I can somehow not comment on lines that are hidden):

        TestFormat.checkNoThrow(applyRules <= 1,
                                "Can only specify one apply constraint " + failAt());

We should be allowed to specify one applyIf* flag constraint together with one applyIfCPUFeature* constraint while not being allowed to specify multiple applyIf* flag constraints and/or multiple applyIfCPUFeature* constraints.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you suggest commenting out check on L208 to enable application of multiple applyIf* constraints OR should that be handled in a separate RFE.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep it as specifying multiple constraint attributes of the same kind (flag or CPU feature) should not be allowed. But we need separate the count (applyRules) for the flag based constraints and the CPU feature based constraints. You could rename applyRules -> flagConstraints and change the ones belonging to the new CPU features to cpuFeatureConstraints. Then you can have these checks here, for example:

TestFormat.checkNoThrow(flagConstraints <= 1, "Can only specify one flag constraint" + failAt());
TestFormat.checkNoThrow(cpuFeatureConstraints <= 1, "Can only specify one CPU feature constraint" + failAt());

}

private boolean hasRequiredFeaturesOr(String[] orRules, String ruleType) {
private boolean hasAnyRequiredCPUFeature(String[] orRules, String ruleType) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paremeter is not used anymore:

Suggested change
private boolean hasAnyRequiredCPUFeature(String[] orRules, String ruleType) {
private boolean hasAnyRequiredCPUFeature(String[] orRules) {

@@ -203,37 +239,49 @@ private boolean hasAllRequiredFlags(String[] andRules, String ruleType) {
return returnValue;
}

private boolean hasRequiredFeaturesAnd(String[] andRules, String ruleType) {
private boolean hasAllRequiredCPUFeature(String[] andRules, String ruleType) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paremeter is not used anymore:

Suggested change
private boolean hasAllRequiredCPUFeature(String[] andRules, String ruleType) {
private boolean hasAllRequiredCPUFeature(String[] andRules) {

return true;
}
if (isKNLFlagEnabled == null ||
(isKNLFlagEnabled.booleanValue() && (!knlFeatureSet.contains(feature.toUpperCase()) || falseValue))) {
Copy link
Member

@chhagedorn chhagedorn Jun 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unboxing is not required:

Suggested change
(isKNLFlagEnabled.booleanValue() && (!knlFeatureSet.contains(feature.toUpperCase()) || falseValue))) {
(isKNLFlagEnabled && (!knlFeatureSet.contains(feature.toUpperCase()) || falseValue))) {

*/

public class TargetFeatureCheckExample {
public class TestCPUFeatureCheck {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a full correctness test on emitting AddVI, I suggest to move it to the other existing vector IR tests. The tests in ir_framework.examples and ir_framework.tests are only executed in tier5 and 6 and are more about testing the framework implementation than the actual correctness of C2 transformations.

Comment on lines 190 to 191
TestFormat.checkNoThrow((irAnno.applyIfCPUFeature().length % 2) == 0,
"Argument count for applyIfCPUFeature should be multiple of two" + failAt());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also check that applyIfCPUFeature is only applied for one constraint and applyIfCPUFeatureAnd for two or more constraints. This follows the format checks for the flag constraints. This flag constraint design, however, is not very satisfying and has some redundancy. I'm planning to rework this with JDK-8280120.

Copy link
Member

@chhagedorn chhagedorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks good to me. Thanks for doing all the updates!

I'll submit some testing.

@openjdk
Copy link

openjdk bot commented Jun 13, 2022

@swati-sha This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8287525: Extend IR annotation with new options to test specific target feature.

Co-authored-by: Jatin Bhateja <jbhateja@openjdk.org>
Reviewed-by: chagedorn, kvn

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 77 new commits pushed to the master branch:

  • 86c9241: 8287028: AArch64: [vectorapi] Backend implementation of VectorMask.fromLong with SVE2
  • fbe9266: 8288378: [BACKOUT] DST not applying properly with zone id offset set with TZ env variable
  • 1904353: Merge
  • 7aafc69: 8288105: [PPC64] Problems with -XX:+VerifyStack
  • f4b05a1: 8288173: JDK-8202449 fix causes conformance test failure : api/java_util/Random/RandomGenerator/NextFloat.html
  • d9c1364: 8288101: False build warning-as-error with GCC 9 after JDK-8214976
  • a9c2ab6: 8288080: (fc) FileChannel::map for MemorySegments should state it always throws UOE
  • e90b579: 8288332: Tier1 validate-source fails after 8279614
  • b42c1ad: 8279614: The left line of the TitledBorder is not painted on 150 scale factor
  • 9b6d0a7: 8285838: DST not applying properly with zone id offset set with TZ env variable
  • ... and 67 more: https://git.openjdk.org/jdk/compare/a9d46f3413ef64c87520509fd70ac42629fbce91...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@chhagedorn, @vnkozlov) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jun 13, 2022
Copy link
Contributor

@vnkozlov vnkozlov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last version looks good.

@swati-sha
Copy link
Contributor Author

/integrate

@openjdk openjdk bot added the sponsor Pull request is ready to be sponsored label Jun 14, 2022
@openjdk
Copy link

openjdk bot commented Jun 14, 2022

@swati-sha
Your change (at version f7dcf31) is now ready to be sponsored by a Committer.

@jatin-bhateja
Copy link
Member

/sponsor

@openjdk
Copy link

openjdk bot commented Jun 14, 2022

Going to push as commit 03dca56.
Since your change was applied there have been 77 commits pushed to the master branch:

  • 86c9241: 8287028: AArch64: [vectorapi] Backend implementation of VectorMask.fromLong with SVE2
  • fbe9266: 8288378: [BACKOUT] DST not applying properly with zone id offset set with TZ env variable
  • 1904353: Merge
  • 7aafc69: 8288105: [PPC64] Problems with -XX:+VerifyStack
  • f4b05a1: 8288173: JDK-8202449 fix causes conformance test failure : api/java_util/Random/RandomGenerator/NextFloat.html
  • d9c1364: 8288101: False build warning-as-error with GCC 9 after JDK-8214976
  • a9c2ab6: 8288080: (fc) FileChannel::map for MemorySegments should state it always throws UOE
  • e90b579: 8288332: Tier1 validate-source fails after 8279614
  • b42c1ad: 8279614: The left line of the TitledBorder is not painted on 150 scale factor
  • 9b6d0a7: 8285838: DST not applying properly with zone id offset set with TZ env variable
  • ... and 67 more: https://git.openjdk.org/jdk/compare/a9d46f3413ef64c87520509fd70ac42629fbce91...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Jun 14, 2022
@openjdk openjdk bot closed this Jun 14, 2022
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review sponsor Pull request is ready to be sponsored labels Jun 14, 2022
@openjdk
Copy link

openjdk bot commented Jun 14, 2022

@jatin-bhateja @swati-sha Pushed as commit 03dca56.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-compiler hotspot-compiler-dev@openjdk.org integrated Pull request has been integrated
Development

Successfully merging this pull request may close these issues.

4 participants