Publish openjdk.test.jlm project into openjdk-systemtest Adopt repository #89

Mesbah-Alam · 2018-03-22T15:57:16Z

Publish openjdk.test.jlm project into openjdk-systemtest Adopt repository. This project can be built and run against both OpenJDK and OpenJDK-OpenJ9 SDKs.

Related Issue: #88

smlambert · 2018-03-27T14:25:30Z

Thanks Mesbah, I am going to look at this over the next few days.

Mesbah-Alam · 2018-03-28T18:52:09Z

Openj9 Issues reported via tests pertaining to this PR so far:
eclipse-openj9/openj9#1566
eclipse-openj9/openj9#1520

smlambert · 2018-03-28T23:08:22Z

openjdk.test.jlm/build.xml

+	<!-- Projects which need to be built before this one. -->
+		<!-- dir must be set on the ant task otherwise the basedir property is not set to a new value in the subant task. -->
+		<target name="build-dependencies">
+			<!-- <ant antfile="${stf_root}/stf.build/build.xml" dir="${stf_root}/stf.build" inheritAll="false"/> -->


Line is commented out, is it not needed?

We don't need this build-dependencies target. This code needs to be cleaned up. Just discovered that, the same clean up needs to happen in other openjdk-systemtest projects too, where the same commented out code remains.

We don't need to build stf as part of building each individual test project, as stf is built from openjdk.build prior to building the test projects.

smlambert · 2018-03-28T23:22:24Z

openjdk.test.jlm/src/test.jlm/net/adoptopenjdk/test/jlm/local/VMLogger.java

+                i++;                
+            }
+       } catch (UnsupportedOperationException uoe) {
+            Message.logOut("One of the operations you tried is not supported");


General comment for the logged messages in this file, that many of them do not seem very useful. Do the stacktraces or some other info get automatically logged?

AttributeNotFoundException log message "Attribute does not exist on the MBean" does not really add value. Based on how some of these tests were written, where try catch blocks around large pieces of code, that it may not be possible how it is structure to include more info (like what attribute, as there are 6 setAttribute calls in this block), but if logging is supposed to help debugging, we may have to revisit this.

Agreed. The reporting and error logging in this test is rather messy. They generate "CSV" data files where some info is logged. The rest go into standard err and out. There are also some tests (e.g. VMLogger, where a log file is used to log detailed results of what the invoked Bean API's return. The output handling is not standardized in general. However, it is quite a lot of work to be honest to standardize the reporting mechanism of these tests. So, may be we could cover it in a separate enhancement work item, as you indicated in your next comment too (just noticed). :)

smlambert · 2018-03-28T23:25:56Z

openjdk.test.jlm/src/test.jlm/net/adoptopenjdk/test/jlm/remote/ClassProfiler.java

+			this.csv.println(df.format(upTime / 1000) + "," + classCount + "," + 
+							loadedCount + "," + unloadedCount);
+		} catch (ConnectException ce) {
+			Message.logOut("Problem with the MBean access");


Same comment as previous about the logging story applies across many test classes.

smlambert · 2018-03-28T23:31:50Z

openjdk.test.jlm/src/test.jlm/net/adoptopenjdk/test/jlm/remote/ServerConnector.java

+				ManagementFactory.RUNTIME_MXBEAN_NAME,
+				RuntimeMXBean.class);
+		String vendor = runtimeBean.getVmVendor();
+		if (vendor.contains("IBM")) {


Is this true for OpenJ9, I thought the vendor name changed for that SDK, as did java -version output? We should check.

Also, I guess if it is true (and you are testing against an IBM SDK, what does this check and information do for the test?

Thanks for the catch! It is indeed not true for OpenJ9. For OpenJ9, runtimeBean.getVMVendor() returns Eclipse OpenJ9. I will update the code to reflect this.

smlambert · 2018-03-28T23:39:26Z

I recognize that this code is migrated, and so do not expect that we can address the inconsistent logging, asserting, and println approach at this time, but we may want an epic/issue at some point to consolidate it.

Mesbah-Alam · 2018-03-29T18:05:45Z

@smlambert I created the following issue to track the work to standardize the error reporting approach of JLM tests : #94

smlambert

Thanks Mesbah, I know these tests run, and have been 'soaked' at other build farms. I would not hold this up for the logging cleanup, it is more appropriate to get these in and running (barring any other comments from reviewers), to vet out any other issues, and later come back to item #94

mamatha-jv · 2018-04-04T10:09:22Z

openjdk.test.jlm/doc/jlm.md

+
+A full list of the java.lang.management functions can be found in the Java 6 API documentation. As mentioned above, the functions are provided by a set of Platform MBeans: 
+> - ClassLoadingMXBean
+  - CompilationMXBean


In the bullets
First item alone is seen in grey color and with different indentation where as second item onward its black color.

I guess you have to remove that angle bracket i.e >

This is there in more than one place, remove them as well.

This is still there in 2 more places

stf.pl -test=TestJlmLocal

stf.pl -test=jlm_remote_test_name

mamatha-jv · 2018-04-04T10:43:29Z

openjdk.test.jlm/doc/jlm.md

+  - MemoryMXBean
+  - MemoryPoolMXBean
+  - OperatingSystemMXBean
+  - RuntimeMXBean<


Remove open angle bracket above i.e <

I see this still

Must have pushed the wrong branch! It should be there now.

mamatha-jv · 2018-04-04T10:48:00Z

openjdk.test.jlm/doc/jlm.md

+
+Once you have set up STF and compiled the test source in your workstation, you can run any test locally from the command line.
+
+####LM Local Tests 


'J' is missing above

Thanks, fixed.

mamatha-jv · 2018-04-04T10:49:12Z

openjdk.test.jlm/doc/jlm.md

+
+###How are the tests implemented? 
+
+The test cases are all implemented in Java, using the *STF*. Please see [STF test framework manual](../../../stf/stf.core/docs/STF-Manual.md).  


I guess this link is broken.

mamatha-jv · 2018-04-04T10:50:05Z

openjdk.test.jlm/doc/jlm.md

+> - STF 
+- Testcase Material 
+
+Since JLM tests are written based on STF, they will require the same set up as STF in general. Please follow the [STF Getting Started Guide](../../../stf/stf.core/docs/STF-GettingStarted.md) and install everything that is required to run STF tests. 


I guess this link as well broken.

mamatha-jv · 2018-04-04T10:56:18Z

openjdk.test.jlm/doc/jlm.md

+5. How do I know whether it worked or failed?
+6. Is there anything else I should know?
+7. Are there any references?
+


Looks like href is not added properly as I don't see hyper links for these when opened from your branch https://github.com/Mesbah-Alam/openjdk-systemtest/blob/addJlmSystemTestToAdopt/openjdk.test.jlm/doc/jlm.md

Missing in-page anchors have been added.

mamatha-jv · 2018-04-13T11:20:52Z

Hi Mesbah
I am not looking into .java files for now as they are migrated ones.
Please share test execution links if any.

Mesbah-Alam · 2018-04-13T15:27:13Z

@mamatha-jv : I had posted links to some of the test runs in personal builds in the pr: #88

llxia · 2018-04-17T16:16:05Z

openjdk.test.jlm/build.xml

+		<javac srcdir="${src_dir}"
+			   destdir="${bin_dir}"
+			   fork="true"
+			   executable="${java8_compiler}"


We should not hard code ${java8_compiler}, what about java9, 10, 11?
I think the code works because in top.xml, it happens to set java8_compiler and java9_compiler to java.home if it is not defined. https://github.com/AdoptOpenJDK/openjdk-systemtest/blob/0b730da942494ff5c4aea03593dffd8cf9252ae2/openjdk.build/include/top.xml#L57
For example, the output for jdk10 testing is very confusing:

17:55:24 [echo] java8_home set to /home/jenkins/workspace/openjdk10_j9_systemtest_x86-64_linux/openjdkbinary/j2sdk-image/bin/.. 17:55:24 [echo] java8_compiler set to /home/jenkins/workspace/openjdk10_j9_systemtest_x86-64_linux/openjdkbinary/j2sdk-image/bin/javac

https://ci.adoptopenjdk.net/view/System%20tests/job/openjdk10_j9_systemtest_x86-64_linux/6/consoleFull

This is may not be the scope of this PR, but top.xml should be fixed. There is no need to have separate variables for java compiler.

Thanks! A new PR has been created for the issue @llxia : #100

llxia · 2018-04-17T16:19:06Z

I understand this code is migrated. Please create issue about mentioned problem. We can fix it in the future.

Mesbah-Alam · 2018-04-17T19:26:54Z

Related PR : adoptium/openj9-systemtest#19

Mesbah-Alam self-assigned this Mar 22, 2018

Mesbah-Alam requested review from smlambert and llxia March 22, 2018 15:57

Mesbah-Alam mentioned this pull request Mar 22, 2018

Open-source JLM SVT test project into Adopt #88

Closed

Mesbah-Alam force-pushed the addJlmSystemTestToAdopt branch 2 times, most recently from 6f3c017 to 5ff466e Compare March 22, 2018 18:32

karianna added the enhancement label Mar 22, 2018

Mesbah-Alam force-pushed the addJlmSystemTestToAdopt branch from 5ff466e to 9fac0ad Compare March 23, 2018 16:14

Mesbah-Alam requested a review from mamatha-jv March 28, 2018 14:22

smlambert reviewed Mar 28, 2018

View reviewed changes

Mesbah-Alam force-pushed the addJlmSystemTestToAdopt branch from 9fac0ad to 16477e3 Compare March 29, 2018 18:02

Mesbah-Alam force-pushed the addJlmSystemTestToAdopt branch from 16477e3 to d928320 Compare March 29, 2018 18:12

smlambert approved these changes Mar 29, 2018

View reviewed changes

Mesbah-Alam force-pushed the addJlmSystemTestToAdopt branch 2 times, most recently from 21306f1 to d839d87 Compare March 29, 2018 19:19

mamatha-jv reviewed Apr 4, 2018

View reviewed changes

Mesbah-Alam force-pushed the addJlmSystemTestToAdopt branch 4 times, most recently from dc8bcd4 to add416e Compare April 6, 2018 14:49

Mesbah-Alam force-pushed the addJlmSystemTestToAdopt branch 3 times, most recently from 8161be4 to a924def Compare April 6, 2018 14:57

Add openjdk.test.jlm project to openjdk-systemtest in Adopt

1c5582a

Mesbah-Alam force-pushed the addJlmSystemTestToAdopt branch from a924def to 1c5582a Compare April 13, 2018 15:22

mamatha-jv approved these changes Apr 16, 2018

View reviewed changes

llxia reviewed Apr 17, 2018

View reviewed changes

llxia approved these changes Apr 17, 2018

View reviewed changes

Mesbah-Alam mentioned this pull request Apr 17, 2018

Fix top.xml and remove hard coded value for ${java8_compiler} #100

Closed

Mesbah-Alam merged commit 22b0a29 into adoptium:master Apr 17, 2018

Mesbah-Alam mentioned this pull request Apr 17, 2018

Add openjdk.test.jlm test playlists to systemtest adoptium/aqa-tests#385

Merged

Mesbah-Alam deleted the addJlmSystemTestToAdopt branch May 31, 2018 20:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Publish openjdk.test.jlm project into openjdk-systemtest Adopt repository #89

Publish openjdk.test.jlm project into openjdk-systemtest Adopt repository #89

Mesbah-Alam commented Mar 22, 2018 •

edited

Loading

smlambert commented Mar 27, 2018

Mesbah-Alam commented Mar 28, 2018

smlambert Mar 28, 2018

Mesbah-Alam Mar 29, 2018

smlambert Mar 28, 2018

Mesbah-Alam Mar 29, 2018 •

edited

Loading

smlambert Mar 28, 2018

smlambert Mar 28, 2018

Mesbah-Alam Mar 29, 2018

smlambert commented Mar 28, 2018

Mesbah-Alam commented Mar 29, 2018

smlambert left a comment

mamatha-jv Apr 4, 2018

mamatha-jv Apr 4, 2018

mamatha-jv Apr 4, 2018

mamatha-jv Apr 13, 2018

mamatha-jv Apr 4, 2018

Mesbah-Alam Apr 6, 2018

mamatha-jv Apr 13, 2018

Mesbah-Alam Apr 13, 2018 •

edited

Loading

mamatha-jv Apr 4, 2018

Mesbah-Alam Apr 6, 2018

mamatha-jv Apr 4, 2018

Mesbah-Alam Apr 6, 2018

mamatha-jv Apr 4, 2018

mamatha-jv Apr 4, 2018

Mesbah-Alam Apr 6, 2018

mamatha-jv commented Apr 13, 2018

Mesbah-Alam commented Apr 13, 2018

llxia Apr 17, 2018

Mesbah-Alam Apr 17, 2018 •

edited

Loading

llxia commented Apr 17, 2018

Mesbah-Alam commented Apr 17, 2018


		Once you have set up STF and compiled the test source in your workstation, you can run any test locally from the command line.

		####LM Local Tests


		###How are the tests implemented?

		The test cases are all implemented in Java, using the STF. Please see [STF test framework manual](../../../stf/stf.core/docs/STF-Manual.md).

Publish openjdk.test.jlm project into openjdk-systemtest Adopt repository #89

Publish openjdk.test.jlm project into openjdk-systemtest Adopt repository #89

Conversation

Mesbah-Alam commented Mar 22, 2018 • edited Loading

smlambert commented Mar 27, 2018

Mesbah-Alam commented Mar 28, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mesbah-Alam Mar 29, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smlambert commented Mar 28, 2018

Mesbah-Alam commented Mar 29, 2018

smlambert left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mesbah-Alam Apr 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mamatha-jv commented Apr 13, 2018

Mesbah-Alam commented Apr 13, 2018

Choose a reason for hiding this comment

Mesbah-Alam Apr 17, 2018 • edited Loading

Choose a reason for hiding this comment

llxia commented Apr 17, 2018

Mesbah-Alam commented Apr 17, 2018

Mesbah-Alam commented Mar 22, 2018 •

edited

Loading

Mesbah-Alam Mar 29, 2018 •

edited

Loading

Mesbah-Alam Apr 13, 2018 •

edited

Loading

Mesbah-Alam Apr 17, 2018 •

edited

Loading