Add examples to running tests user guide. #770

jerboaa · 2018-12-21T18:04:51Z

No description provided.

smlambert · 2018-12-21T20:39:22Z

Yes, I guess we should house some examples in this doc, rather than point folks to the openj9 functional tests readme from this doc as we do now. Could also duplicate these examples: https://github.com/eclipse/openj9/wiki/Reproducing-Test-Failures-Locally#openjdk-tests for a shorter set of commands to run.

smlambert

Thanks for pulling this together @jerboaa - I have added a few notes/suggestions.

smlambert · 2018-12-21T21:14:02Z

doc/userGuide.md

+$ export BUILD_LIST=openjdk_regression
+$ TOP_DIR=$(pwd)
+$ export BUILD_ROOT=$TOP_DIR/openjdk_regression
+$ export JRE_IMAGE=$OPENJDK_BUILD/../j2re-image


You do not need or use JRE_IMAGE in your example as it stands.

Are you sure? My experiments showed that I needed to set this because as per openjdk_regression.mk a default of j2jre-image will be used. OpenJDK builds only have j2re-image.

Weird, as we should have same naming convention as OpenJDK builds.

So are you saying that j2jre-image in openjdk_regression.mk should get fixed to j2re-image? My upstream JDK 8u build has this, fwiw:

$ ls -1 build/linux-x86_64-normal-server-fastdebug/images/ | grep image j2re-image j2sdk-image

Probably AdoptOpenJDK builds updated j2jre to j2re as I believed that 'j2jre-image' was used before. Need to double check.

smlambert · 2018-12-21T21:15:46Z

doc/userGuide.md

+_sanity.openjdk done
+
+[...]
+```


You may additionally want to add that additional test output gets put into a test_output_xxxtimestamp folder, so if there was a failure, you can find the .jtr files to get more details.

Thanks, added.

smlambert · 2018-12-21T21:16:34Z

doc/userGuide.md

+$ ./maketest.sh $TEST_DIR
+$ cd $TEST_DIR
+$ wget https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/lastSuccessfulBuild/artifact/jtreg-4.2.0-tip.tar.gz
+$ tar -xf jtreg-4.2.0-tip.tar.gz


You do not need to explicitly get jtreg and untar it, the "make compile" target (called in line 26 in maketest.sh) will take care of this... as it will invoke the build.xml ant script inside any dir you are trying to compile, and in the case of openjdk tests, there is a getJtreg target in its build script.

This doesn't seem to work for me. build.xml in openjdk_regression folder is never being executed. Example:

$ cat run_adopt_openjdk_jtreg_tests.sh #!/bin/bash OPENJDK_SOURCES=$1 OPENJDK_BUILD=$2 git clone https://github.com/AdoptOpenJDK/openjdk-tests cd openjdk-tests TEST_DIR="openjdk_regression" export JAVA_BIN=$OPENJDK_BUILD/jre/bin export SPEC=linux_x86-64 export JDK_IMPL=hotspot export BUILD_LIST=openjdk_regression TOP_DIR=$(pwd) export BUILD_ROOT=$TOP_DIR/openjdk_regression export JRE_IMAGE=$OPENJDK_BUILD/../j2re-image ./get.sh -t $TEST_DIR -p x64_linux -j SE80 -i hotspot -R latest -T jdk ./maketest.sh $TEST_DIR #cd $TEST_DIR #wget https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/lastSuccessfulBuild/artifact/jtreg-4.2.0-tip.tar.gz #tar -xf jtreg-4.2.0-tip.tar.gz #cd $TOP_DIR OPENJDK_DIR=$OPENJDK_SOURCES ./maketest.sh $TEST_DIR _sanity.openjdk $ rm -rf test-me; mkdir test-me; pushd test-me; bash ../run_adopt_openjdk_jtreg_tests.sh /disk/openjdk/upstream-sources/openjdk8u-dev /disk/openjdk/upstream-sources/openjdk8u-dev/build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image; popd --> file downloaded to /home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression/TestConfig/../TestConfig/lib/jaxb-api.jar downloaded dependent third party jars successfully make[1]: Entering directory '/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression' make[1]: Nothing to be done for 'compile'. make[1]: Leaving directory '/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression' ant -f scripts/build_test.xml -DTEST_ROOT=/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression/TestConfig/.. -DBUILD_ROOT=/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression -DJAVA_BIN=/disk/openjdk/upstream-sources/openjdk8u-dev/build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/jre/bin -DJDK_VERSION=8 -DJCL_VERSION=latest -DBUILD_LIST=openjdk_regression -DRESOURCES_DIR=/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression/TestConfig/resources -DPLATFORM=linux_x86-64 -DJDK_HOME=/disk/openjdk/upstream-sources/openjdk8u-dev/build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/jre/bin/../.. -DJVM_VERSION=openjdk8 Buildfile: /home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression/TestConfig/scripts/build_test.xml [echo] JAVA_BIN is /disk/openjdk/upstream-sources/openjdk8u-dev/build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/jre/bin [echo] build.list is openjdk_regression/build.xml -create_test_directory: stage_test_material: build: [subant] No sub-builds to iterate on BUILD SUCCESSFUL Total time: 0 seconds make: Entering directory '/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression' Running make 4.2.1 set TEST_ROOT to /home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression/TestConfig/scripts/testKitGen/../../.. set JDK_VERSION to 8 set JDK_IMPL to hotspot set JVM_VERSION to openjdk8 set JCL_VERSION to latest set JAVA_BIN to /disk/openjdk/upstream-sources/openjdk8u-dev/build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/jre/bin set SPEC to linux_x86-64 Running sanity.openjdk ... "/disk/openjdk/upstream-sources/openjdk8u-dev/build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/jre/bin/java" -version openjdk version "1.8.0-internal-fastdebug" OpenJDK Runtime Environment (build 1.8.0-internal-fastdebug-sgeholf_2018_12_19_18_33-b00) OpenJDK 64-Bit Server VM (build 25.71-b00-fastdebug, mixed mode) =============================================== Running test jdk_io_0 ... =============================================== jdk_io_0 Start Time: Wed Jan 2 12:15:04 2019 Epoch Time (ms): 1546427704256 Nothing to be done for setup. variation: NoOptions JVM_OPTIONS: { itercnt=1; \ mkdir -p "/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_15464277039037/jdk_io_0"; \ cd "/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_15464277039037/jdk_io_0"; \ "/disk/openjdk/upstream-sources/openjdk8u-dev/build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/jre/bin/java" -Xmx512m -jar "/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression//jtreg/lib/jtreg.jar" \ -agentvm -a -ea -esa -v:fail,error,time,nopass -retain:fail,error -ignore:quiet -timeoutFactor:8 -xml:verify -concurrency:8 -vmoptions:"-Xmx512m " \ -w "/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression//work" \ -r "/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression//report" \ -jdk:"/disk/openjdk/upstream-sources/openjdk8u-dev/build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/jre/bin/../.." \ -exclude:"/disk/openjdk/upstream-sources/openjdk8u-dev/jdk/test/ProblemList.txt" \ -exclude:"/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression//ProblemList_openjdk8.txt" \ "/disk/openjdk/upstream-sources/openjdk8u-dev/jdk/test:jdk_io"; \ if [ $? -eq 0 ] ; then echo ""; echo "jdk_io_0""_PASSED"; echo ""; cd /home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression/TestConfig/scripts/testKitGen/../../..; else echo ""; echo "jdk_io_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "/home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression/TestConfig/scripts/testKitGen/../../../TestConfig/test_output_15464277039037/TestTargetResult"; Error: Unable to access jarfile /home/sgehwolf/Documents/openjdk/adopt-openjdk-prototype/test-me/openjdk-tests/openjdk_regression//jtreg/lib/jtreg.jar jdk_io_0_FAILED Nothing to be done for teardown. jdk_io_0 Finish Time: Wed Jan 2 12:15:04 2019 Epoch Time (ms): 1546427704279

see Sophia's note re: TEST_DIR

Alright. There seems to be an implicit assumption of what TEST_DIR points to unfortunately it seems. Anyway, I've updated the PR which now fetches jtreg via the getJtreg target. Thanks!

sophia-guo · 2019-01-02T14:24:27Z

doc/userGuide.md

+$ pushd $tmpdir
+$ git clone https://github.com/AdoptOpenJDK/openjdk-tests
+$ cd openjdk-tests
+$ TEST_DIR="openjdk_regression"


TEST_DIR=$pwd ( openjdk-tests folder)

sophia-guo · 2019-01-02T14:28:51Z

doc/userGuide.md

+$ TOP_DIR=$(pwd)
+$ export BUILD_ROOT=$TOP_DIR/openjdk_regression
+$ export JRE_IMAGE=$OPENJDK_BUILD/../j2re-image
+$ ./get.sh -t $TEST_DIR -p x64_linux -j SE80 -i hotspot -R latest -T jdk


As the developer has a fastdebug build, no need to download a SDK from AdoptOpenJDK. That is ./get.sh -t $TEST_DIR is good ( with TEST_DIR is openjdk-tests folder), which should also solve the following issue 'build.xml in openjdk_regression folder is never being executed'.

@jerboaa Could you please update this line to ./get.sh -t $TEST_DIR as other parameters are for download SDK, which is not used for your scenario.

OK, thanks. This wasn't clear to me when you mentioned it earlier. Done now, thanks!

jerboaa · 2019-01-03T17:22:40Z

@smlambert @sophia-guo Any more thoughts on this? Can this be merged?

sophia-guo

Please update the command get.sh to ./get.sh -t $TEST_DIR. Other than that LGTM.

smlambert · 2019-01-03T19:39:13Z

Yes, only waiting on update to ./get.sh line, so you are not needlessly downloading an SDK, when your example describes a scenario where you already have one local.

The get script downloads testkitgen and some test material, and optionally will go and pick up a build from AdoptOpenJDK also, but your scenario does not require it, so best to not fetch it needlessly.

jerboaa · 2019-01-04T09:27:45Z

@smlambert @sophia-guo Updated now. How does it look?

smlambert reviewed Dec 21, 2018

View reviewed changes

karianna added this to TODO in aqa-tests via automation Dec 21, 2018

karianna added the enhancement label Dec 21, 2018

karianna moved this from TODO to QA / Review in aqa-tests Dec 21, 2018

jerboaa force-pushed the add-examples-to-docs branch from abcb7b0 to 8f0bf34 Compare January 2, 2019 13:11

sophia-guo reviewed Jan 2, 2019

View reviewed changes

jerboaa force-pushed the add-examples-to-docs branch from 8f0bf34 to 320d9c2 Compare January 2, 2019 19:41

sophia-guo approved these changes Jan 3, 2019

View reviewed changes

Add examples to running tests user guide.

f538a79

jerboaa force-pushed the add-examples-to-docs branch from 320d9c2 to f538a79 Compare January 4, 2019 09:26

smlambert approved these changes Jan 4, 2019

View reviewed changes

smlambert merged commit d002e31 into adoptium:master Jan 4, 2019

aqa-tests automation moved this from QA / Review to Done Jan 4, 2019

jerboaa deleted the add-examples-to-docs branch January 11, 2019 17:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add examples to running tests user guide. #770

Add examples to running tests user guide. #770

jerboaa commented Dec 21, 2018

smlambert commented Dec 21, 2018

smlambert left a comment

smlambert Dec 21, 2018

jerboaa Jan 2, 2019

sophia-guo Jan 2, 2019

jerboaa Jan 2, 2019

sophia-guo Jan 2, 2019

smlambert Dec 21, 2018

jerboaa Jan 2, 2019

smlambert Dec 21, 2018

jerboaa Jan 2, 2019

smlambert Jan 2, 2019

jerboaa Jan 2, 2019

sophia-guo Jan 2, 2019

sophia-guo Jan 2, 2019

sophia-guo Jan 3, 2019

jerboaa Jan 4, 2019

jerboaa commented Jan 3, 2019

sophia-guo left a comment

smlambert commented Jan 3, 2019

jerboaa commented Jan 4, 2019

Add examples to running tests user guide. #770

Add examples to running tests user guide. #770

Conversation

jerboaa commented Dec 21, 2018

smlambert commented Dec 21, 2018

smlambert left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jerboaa commented Jan 3, 2019

sophia-guo left a comment

Choose a reason for hiding this comment

smlambert commented Jan 3, 2019

jerboaa commented Jan 4, 2019