Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8254129: IR Test Framework to support regex-based matching on the IR in JTreg compiler tests #3508

Closed
wants to merge 29 commits into from
Closed
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
a563fb3
8254129: IR Test Framework to support regex-based matching on the IR …
chhagedorn Apr 13, 2021
aa005f3
Add Javadoc files
chhagedorn Apr 14, 2021
5ada49e
Fix whitespaces
chhagedorn Apr 15, 2021
4ec7f0e
Fix whitespaces + minor fix
chhagedorn Apr 15, 2021
cc6e285
Remove accidentally added print statement
chhagedorn Apr 15, 2021
e284341
Fix comment
chhagedorn Apr 15, 2021
7ed789d
Adjust whitelist
chhagedorn Apr 15, 2021
b3f5811
Apply review comments in code from Igor I. and Vladimir K.
chhagedorn Apr 19, 2021
6e5a954
Fix typos and grammar, code format (JohnTortugo), fix exception throw…
chhagedorn Apr 20, 2021
5451ad2
Replace "\n" by System.lineSeparator()
chhagedorn Apr 20, 2021
433006c
Improve error format output by changing some new lines
chhagedorn Apr 20, 2021
72eece6
Remove Javadocs and README.html, update README.md to reference java f…
chhagedorn Apr 20, 2021
0720a33
Review comments by Tobias: Formatting fields, spacing, README.md typo…
chhagedorn Apr 23, 2021
0a3cc3f
Apply review comments: Added new Compiler annotation class for @DontC…
chhagedorn Apr 27, 2021
90a0064
Fix XCOMP cases from old framework and turn it into new debug flag -D…
chhagedorn Apr 27, 2021
5890c4a
Apply review comments: Extract Test classes into own files, extract F…
chhagedorn May 3, 2021
8f56dd6
Rename TestFrameworkPrepareFlags -> FlagVM and rename TestFrameworkEx…
chhagedorn May 3, 2021
b08235c
Minor improvements, comment fixes, and test fixes
chhagedorn May 3, 2021
d6c72ec
Remove TestFramework: both runWithScenarios, both runWithHelperClasse…
chhagedorn May 3, 2021
fd25de7
Move framework to test/hotspot/jtreg/compiler/lib and tests to test/h…
chhagedorn May 4, 2021
a9f0429
Fix package names and fixing internal tests, examples and README file…
chhagedorn May 4, 2021
4424e01
Splitting classes into subpackages and updating README accordingly, f…
chhagedorn May 4, 2021
85a5921
Merge branch 'master' into JDK-8254129
chhagedorn May 31, 2021
c35c658
Fix Compiler and CompLevel ANY and fix tests after merge
chhagedorn May 31, 2021
7a316de
Add more whitelisted flags
chhagedorn Jun 2, 2021
df7576f
Fix failing internal tests on Windows and add missing flag descriptio…
chhagedorn Jun 3, 2021
2ba1338
Move tests and examples #1
chhagedorn Jun 4, 2021
e6b2f60
Move tests and examples #2
chhagedorn Jun 4, 2021
7b77370
Update test and example package names, README files and fix some test…
chhagedorn Jun 4, 2021
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
@@ -0,0 +1,121 @@
/*
* Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*/

package compiler.lib.ir_framework;

import compiler.lib.ir_framework.shared.TestRunException;
import compiler.lib.ir_framework.test.TestVM;
import jdk.test.lib.Utils;

import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.Random;
import java.util.stream.Collectors;

/**
* Base info class which provides some useful utility methods and information about a test.
* <p>
* <b>Base tests</b> and <b>checked tests</b> use {@link TestInfo} while <b>custom run tests</b> use {@link RunInfo}.
*
* @see Test
* @see Check
* @see Run
*/
abstract public class AbstractInfo {
private static final Random RANDOM = Utils.getRandomInstance();

protected final Class<?> testClass;
private boolean onWarmUp = true;

AbstractInfo(Class<?> testClass) {
this.testClass = testClass;
}

/**
* Get the initialized {@link Random} object.
*
* @return the random object.
*/
public Random getRandom() {
return RANDOM;
}

/**
* Returns a boolean indicating if the framework is currently warming up the associated test.
*
* @return the warm-up status of the associated test.
*
* @see Warmup
*/
public boolean isWarmUp() {
return onWarmUp;
}

/**
* Get the method object of the method {@code name} of class {@code c} with arguments {@code args}.
*
* @param c the class containing the method.
* @param name the name of the method.
* @param args the arguments of the method, leave empty if no arguments.
*
* @return the method object of the requested method.
*/
public Method getMethod(Class<?> c, String name, Class<?>... args) {
try {
return c.getMethod(name, args);
} catch (NoSuchMethodException e) {
String parameters = args == null || args.length == 0 ? "" :
" with arguments [" + Arrays.stream(args).map(Class::getName).collect(Collectors.joining(",")) + "]";
throw new TestRunException("Could not find method " + name + " in " + c + parameters, e);
}
}

/**
* Get the method object of the method {@code name} of the test class with arguments {@code args}.
*
* @param name the name of the method in the test class.
* @param args the arguments of the method, leave empty if no arguments.
*
* @return the method object of the requested method in the test class.
*/
public Method getTestClassMethod(String name, Class<?>... args) {
return getMethod(testClass, name, args);
}

/**
* Returns a boolean indicating if the test VM runs with flags that allow C2 compilations.
*
* @return {@code true} if C2 compilations are allowed;
* {@code false} otherwise (run with {@code -XX:TieredStopAtLevel={1,2,3}, -XX:-UseCompiler}).
*/
public boolean isC2CompilationEnabled() {
return TestVM.USE_COMPILER && !TestVM.TEST_C1;
}

/**
* Called by {@link TestFramework} when the warm-up is finished. Should not be called by user code.
*/
public void setWarmUpFinished() {
onWarmUp = false;
}
}
@@ -0,0 +1,81 @@
/*
* Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*/

package compiler.lib.ir_framework;

/**
* Well-defined argument values that can be used in the {@link Arguments} annotation at a {@link Test} method for a
* <b>base test</b> or a <b>checked test</b>.
*
* @see Arguments
* @see Test
* @see Check
*/
public enum Argument {
/**
* Provides the default value for any kind of primitive type and object type if the class provides a default constructor.
*/
DEFAULT,
/**
* Provides the number 42 for any primitive number type.
*/
NUMBER_42,
/**
* Provides the number -42 for any primitive number type.
*/
NUMBER_MINUS_42,
/**
* Provides the minimum value of the specified primitive number type.
*/
MIN,
/**
* Provides the maximum value of the specified primitive number type.
*/
MAX,
/**
* Provides the boolean value false.
*/
FALSE,
/**
* Provides the boolean value true.
*/
TRUE,
/**
* Provides a different boolean value on each test invocation, starting with false.
*/
BOOLEAN_TOGGLE_FIRST_FALSE,
/**
* Provides a different boolean value on each test invocation, starting with true.
*/
BOOLEAN_TOGGLE_FIRST_TRUE,
/**
* Provides a random primitive value on the first test invocation and reuses the same value for all invocation of the test.
* Float and double values are restricted to the range [-10000,10000].
*/
RANDOM_ONCE,
/**
* Provides a different random primitive value on each test invocation.
* Float and double values are restricted to the range [-10000,10000].
*/
RANDOM_EACH,
}
@@ -0,0 +1,43 @@
/*
* Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*/

package compiler.lib.ir_framework;

import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;

/**
* This annotation is used to specify well-defined {@link Argument} values for test methods (specifying {@link Test}) when
* used as part of a <b>base test</b> or <b>checked test</b>.
*
* @see Argument
* @see Test
* @see Check
*/
@Retention(RetentionPolicy.RUNTIME)
public @interface Arguments {
/**
* Get the argument value.
*/
Argument[] value();
}
@@ -0,0 +1,114 @@
/*
* Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*/

package compiler.lib.ir_framework;

import compiler.lib.ir_framework.shared.TestFormatException;

import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;

/**
* Annotation for a check method of a <b>checked test</b>.
*
* <p>
* Let {@code t} be a test method specifying the {@link Test @Test} annotation and {@code c} be a check method specifying
* the {@code @Check(test = "t")} annotation. These two methods represent a so-called <i>checked test</i>. The only
* difference to a <i>base test</i> (see {@link Test}) is that the framework will invoke the check method {@code c}
* directly after the invocation of the test method {@code t} which allows to do some additional verification,
* including the return value of {@code t}. The framework does the following, similar as for <i>base tests</i>:
* <ol>
* <li><p>The framework warms {@code t} up by invoking it for a predefined number of iterations (default: 2000)
* or any number specified by an additional {@link Warmup @Warmup} annotation at {@code t} or by using
* {@link TestFramework#setDefaultWarmup(int)} (could also be 0 which skips the warm-up completely which is
* similar to simulating {@code -Xcomp}). After each invocation of {@code t}, the framework also invokes
* {@code c} if the {@code @Check} annotation specifies {@link CheckAt#EACH_INVOCATION} at {@link #when()}.
* More information about the warm-up in general can be found at {@link Warmup}</li>
* <li><p>After the warm-up, the framework compiles {@code t} at the specified compilation level set by
* {@link Test#compLevel()} (default {@link CompLevel#ANY} will pick the highest available level which is
* usually {@link CompLevel#C2}).</li>
* <li><p>The framework invokes {@code t} one more time to run the compilation. Afterwards, the framework will
* always invoke {@code c} again to be able perform additional checks after the compilation of {@code t}.</li>
* <li><p>The framework checks any specified {@link IR @IR} constraints at the test method {@code t}.
* More information about IR matching can be found at {@link IR}.</li>
* </ol>
*
* <p>
* The test method {@code t} has the same properties and follows the same constraints as stated in {@link Test}.
* <p>
* The following additional constraints must be met for the test method {@code t} and check method {@code c}:
* <ul>
* <li><p>{@code c} must specify the method name {@code t} as property in {@code @Check(test = "t")}
* (see {@link #test()}. Specifying a non-{@code @Test} annotated method or a {@code @Test} method that
* has already been used by another {@code @Check} or {@link Run @Run} method results in a {@link TestFormatException}.
* <li><p>{@code c} can specify the following method parameter combinations:
* <ul>
* <li><p>void</li>
* <li><p>One parameter: {@link TestInfo} which provides some information about {@code t} and utility methods.</li>
* <li><p>One parameter: the <i>exact</i> same type as the return value of {@code t}. When {@code c} is
* invoked by the framework, this parameter contains the return value of {@code t}.</li>
* <li><p>1st parameter: {@link TestInfo}; 2nd parameter: the <i>exact</i> same type as the return value of
* {@code t} (see above)</li>
* <li><p> Any other combination will result in a {@link TestFormatException}.
* </ul>
* <li><p>{@code c} is not compiled nor inlined.
* <li><p>{@code c} must be part of the test class. Using {@code @Check} in nested or other classes is not allowed.</li>
* <li><p>{@code c} cannot specify any helper-method-specific compile command annotations
* ({@link ForceCompile @ForceCompile}, {@link DontCompile @DontCompile}, {@link ForceInline @ForceInline},
* {@link DontInline @DontInline}).</li>
* </ul>
*
* <p>
* If no verification is required, use a <i>base test</i> (see {@link Test}). If {@code t} must be invoked with more
* complex or varying arguments and/or the {@code t} must be invoked differently in subsequent invocations, use a
* <i>custom run test</i> (see {@link Run}).
*
* <p>
* Examples on how to write checked tests can be found in {@link jdk.test.lib.hotspot.ir_framework.examples.CheckedTestExample}
* and also as part of the internal testing in the package {@link jdk.test.lib.hotspot.ir_framework.tests}.
*
* @see Test
* @see TestInfo
* @see CheckAt
*/
@Retention(RetentionPolicy.RUNTIME)
public @interface Check {
/**
* The unique associated {@link Test} method for this {@code @Check} annotated check method. The framework will directly
* invoke the {@code @Check} method after each invocation or only after the compilation of the associated {@code @Test}
* method (depending on the value set with {@link #when()}).
* <p>
* If a non-{@code @Test} annotated method or a {@code @Test} method that has already been used by another
* {@code @Check} or {@link Run} method is specified, then a {@link TestFormatException} is thrown.
*
* @see Test
*/
String test();
/**
* When should the {@code @Check} method be invoked? By default, the check is done after each invocation which is
* encouraged if performance is not critical.
*
* @see CheckAt
*/
CheckAt when() default CheckAt.EACH_INVOCATION;
}
Loading