JVM annotations

🚩 Table of contents

One JVM by test method

How to get the JVM options added by QuickPerf?

Configure your test JVM
    @HeapSize  |  @Xms  | @Xmx  |  @UseGC  |  @EnableGcLogging  |  @JvmOptions

Verify heap allocation
    @MeasureHeapAllocation  | @ExpectMaxHeapAllocation  |  @ExpectNoHeapAllocation

Dump the heap

Verify resident set size (RSS)
    @MeasureRSS  |  @ExpectMaxRSS

Profile or check your JVM
    @ProfileJvm  |  @ExpectNoJvmIssue

Global JVM annotations

Test examples

One JVM by test method

⚠️ If you use one of the JVM annotations, the test method is executed in a dedicated JVM.

How to get the JVM options added by QuickPerf?

Some JVM annotations configure JVM options.

You can use @DebugQuickPerf to get the JVM options added by QuickPerf.

🔎 Example

 @DebugQuickPerf

JVM OPTIONS
-Xms20m
-Xmx20m
-XX:+UnlockExperimentalVMOptions
-XX:+AlwaysPreTouch
-XX:+UseEpsilonGC
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=C:\Users\JEANBI~1\AppData\Local\Temp\QuickPerf-16618400885811126911\heapDump.hprof

Configure your test JVM

@HeapSize

This annotation makes the test executed in a specific JVM with the given heap size.

🔧 Elements

Name	Type	Meaning
value	long	Heap size value (Xms=Xmx)
unit	AllocationUnit	Allocation unit

🔎 Example

  @HeapSize(value = 20, unit = AllocationUnit.MEGA_BYTE)

💡 Fixing maximum heap size as a threshold test

@Xms

This annotation makes the test executed in a specific JVM having the given initial and minimum heap size value.

🔧 Elements

Name	Type	Meaning
value	long	Initial and minimum heap size value
unit	AllocationUnit	Allocation unit

🔎 Example

 @Xms(value = 20, unit = AllocationUnit.MEGA_BYTE)

@Xmx

This annotation annotation makes the test executed in a specific JVM having the given maximum heap size value.

🔧 Elements

Name	Type	Meaning
value	long	Maximum heap size value
unit	AllocationUnit	Allocation unit

🔎 Example

 @Xmx(value = 20, unit = AllocationUnit.MEGA_BYTE)

💡 Fixing maximum heap size as a threshold test

@UseGC

This annotation makes the test executed in a specific JVM with the given Garbage Collector (GC).

🔧 Elements

Name	Type	Meaning	Default value
value	org.quickperf.jvm.gc.GC	GC type	GC.DEFAULT

The following GC types are available:

• GC.EPSILON_GC: Epsilon GC - Doc 1, Epsilon GC - Doc 2
• GC.ZGC: ZGC - Doc
• GC.SHENANDOAH: Shenandoah - Doc

🔎 Example

@UseGC(GC.EPSILON_GC)

@EnableGcLogging

This annotation enables GC logging.

The path of the GC log file is displayed on the console:

GC log file: C:\Users\JEANBI~1\AppData\Local\Temp\QuickPerf-1127741195724299919\gc.log

This file can be analysed with the help of a GC log analyzer:

• GCViewer, GCViewer Download 1, GCViewer Download 2
• Censum
• GCeasy, the GC log file can be uploaded
• ...

@JvmOptions

This annotation makes the test executed in a specific JVM having the given JVM options.

A tool developed by Chris Newland can be used to explore the available JVM options.

🔧 Elements

Name	Type	Meaning
value	String	JVM options

Verify heap allocation

The following annotations use ByteWatcher under the hood:

• https://github.com/danielshaya/ByteWatcher
• https://www.javaspecialists.eu/archive/Issue232.html

⚠️ They measure heap allocation of the test method thread.

For example, you can use MeasureHeapAllocation and ExpectMaxHeapAllocation annotations to check the heap allocation cost of a large data structure (e.g., 1 000 000 elements).

You can use ExpectNoHeapAllocation annotation to verify that the tested code does not allocate on the heap.

@MeasureHeapAllocation

This annotation measures the heap allocation of the test method thread and displays it on the console.

🔧 Elements

Name	Type	Meaning	Default value
format	java.lang.String	Provides the format used to print the measured heap allocation on the console. This format will be called with a preformatted allocation as a String. So the only element you can use in this format is %s.	[QUICK PERF] Measured heap allocation (test method thread): %s
writerFactory	Class<? extends WriterFactory>	Allows you to provide a way to build a Writer instance to print your messages. The WriterFactoryclass is used to built this Writer. This WriterFactory class is constructed using reflection, so it should have an empty constructor. If it does not an exception will be raised and the default Writer will be used. The default value DefaultWriterFactory builds a Writerthat writes to System.out. In case an exception is raised in the use of a provided factory, the system falls back on this default value.	DefaultWriterFactory.class

🔎 Example 1

@RunWith(QuickPerfJUnitRunner.class)
public class ClassWithMethodAnnotatedWithMeasureAllocation {

    @MeasureHeapAllocation
    @JvmOptions("-XX:+UseCompressedOops -XX:+UseCompressedClassPointers")
    @Test
    public void array_list_with_size_100_should_allocate_440_bytes() {
        // java.util.ArrayList: 24 bytes
        //            +
        //  Object[]: 16 + 100 x 4 = 416
        //       = 440 bytes
        ArrayList<Object> data = new ArrayList<>(100);
    }

}

Console:

[QUICK PERF] Measured heap allocation (test method thread): 440 bytes

🔎 Example 2 (writerFactory and format)

@QuickPerfTest
public class TestClass {


    @MeasureHeapAllocation(writerFactory = FileWriterBuilder.class, format = "Heap allocation: %s\n")
    @Test
    public void test_method() {
        //...
    }

    public static class FileWriterBuilder implements WriterFactory {

        @Override
        public Writer buildWriter() throws IOException {
            return new FileWriter("C:\\Users\\UserName\\Allocation.txt", true);
        }
    }

}

@ExpectMaxHeapAllocation

This annotation makes the test fail if the heap allocation of the test method thread is greater than expected.

🔧 Elements

Name	Type	Meaning
value	long	Allocation value
unit	AllocationUnit	Allocation unit

🔎 Example

   @ExpectMaxHeapAllocation(value = 440, unit = AllocationUnit.BYTE)
   @Test
   public void array_list_with_size_100_should_allocate_440_bytes() {
       ArrayList<Object> data = new ArrayList<>(100);
   }

@ExpectNoHeapAllocation

This annotation makes the test fail if the test thread allocates on the heap.

Dump the heap

HeapDumper

You can use the two following methods of org.quickperf.jvm.heap.HeapDumper class to dump the Java heap:

• public static void dumpHeap(String fileName)
• public static void dumpHeapWithOnlyLiveObjects(String fileName)

💡 How to analyze the heap dump?

Several tools can be used to analyze the heap dump:

• Memory Analyzer Tool (MAT)
• JOverflow JMC plugin
• VisualVM
• YourKit (commercial)
• JProfiler (commercial)
• ...

🔎 Example

@HeapSize(value = 50, unit = AllocationUnit.MEGA_BYTE)
@Test
public void do_something_and_dump_heap() {

  IntegerAccumulator integerAccumulator = new IntegerAccumulator();
  integerAccumulator.accumulateInteger(3_000_000);

  HeapDumper.dumpHeap("C:\\Users\\UserName\heap-dump.hprof");

}

Console:

[QUICK PERF] Heap dump file 
👉 C:\Users\UserName\heap-dump.hprof

Verify resident set size (RSS)

@MeasureRSS

You can measure the Resident Set Size (RSS) with this annotation.

The measured RSS is displayed on the console.

⚠️ Today this annotation only woks on Linux. You can work on this issue to make the RSS annotations work on MacOS.

🔎 Example

[QUICK PERF] Measured RSS (process 5227): 46.64 Mega bytes (48 902 144 bytes)

@ExpectMaxRSS

This annotation will make the test fail if the Resident Set Size (RSS) is greater than expected.

⚠️ Today this annotation only woks on Linux. You can work on this issue to make the RSS annotations work on MacOS.

🔧 Elements

Name	Type	Meaning
value	long	value
unit	AllocationUnit	RAM unit

Profile or check your JVM

To be able to use the ProfileJvm and ExpectNoJvmIssue annotations, you must add the following dependency:

<dependency>
   <groupId>org.quickperf</groupId>
   <artifactId>quick-perf-jfr-annotations</artifactId>
   <version>1.1.0</version>
   <scope>test</scope>
</dependency>

If you get an org.openjdk.jmc:flightrecorder.rules.jdk:pom:7.0.0 not found issue, read this to fix it.

With these annotations, the JVM is profiled with the JDK Flight Recorder (JFR), an event recorder built into the JVM.

The profiling content is saved in a .jfr file.

⚠️ JFR profiling works with

• OpenJDK JDK >= 11
• OpenJDK JDK 8 with a version greater than u262/u272 (following vendors)
  👉 Article from Marcus Hirt giving details: http://hirt.se/blog/?p=1235
• Oracle JDK >= 1.7u40

@ProfileJvm

To profile the JVM with the JDK Flight Recorder (JFR).

The console displays the path to the recording file. You can open this file with JDK Mission Control.

ProfileJvm annotation also displays some JVM profiling data (GC times, heap allocation estimation, exception numbers, ...) in standard output.

💡 Where to find JDK Mission Control (JMC)?

⚠️ From JDK 16, to see the allocation metrics, you need to enable the TLAB and outside TLAB JFR events. To do this, update the profile.jfc file contained into thejdk_folder\lib\jfr folder:

    <event name="jdk.ObjectAllocationInNewTLAB">
      <setting name="enabled" control="memory-profiling-enabled-medium">true</setting>
      <setting name="stackTrace">true</setting>
    </event>

    <event name="jdk.ObjectAllocationOutsideTLAB">
      <setting name="enabled" control="memory-profiling-enabled-medium">true</setting>
      <setting name="stackTrace">true</setting>
    </event>

🔎 Example

[QUICK PERF] JVM was profiled with JDK Flight Recorder (JFR).
The recording file is available here: C:\Users\JEANBI~1\AppData\Local\Temp\QuickPerf-9292511997956298899\jvm-profiling.jfr
You can open it with JDK Mission Control (JMC).
Where to find JDK Mission Control? 👉 https://tinyurl.com/find-jmc

------------------------------------------------------------------------------
 ALLOCATION (estimations)     |   GARBAGE COLLECTION           |  THROWABLE
 Total       : 3,68 GiB       |   Total pause     : 1,264 s    |  Exception: 0
 Inside TLAB : 3,67 GiB       |   Longest GC pause: 206,519 ms |  Error    : 36
 Outside TLAB: 12,7 MiB       |   Young: 13                    |  Throwable: 36
 Allocation rate: 108.1 MiB/s |   Old  : 3                     |
------------------------------------------------------------------------------
 COMPILATION                  |   CODE CACHE
 Number : 157                 |   The number of full code cache events: 0
 Longest: 1,615 s             |   
------------------------------------------------------------------------------
 JVM
 Name     : OpenJDK 64-Bit Server VM
 Version  : OpenJDK 64-Bit Server VM (11.0.1+13) for windows-amd64 JRE (11.0.1+13), built on Oct  6 2018 13:18:13 by "mach5one" with MS VC++ 15.5 (VS2017)
 Arguments: -Xms6g -Xmx6g -XX:+FlightRecorder -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=C:\Users\JEANBI~1\AppData\Local\Temp\QuickPerf-1155358826815951142\heap-dump.hprof -DquickPerfToExecInASpecificJvm=true -DquickPerfWorkingFolder=C:\Users\JEANBI~1\AppData\Local\Temp\QuickPerf-1155358826815951142
------------------------------------------------------------------------------
 HARDWARE
 Hardware threads: 8
 Cores           : 4
 Sockets         : 1
 CPU
		Brand: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz, Vendor: GenuineIntel
		Family: <unknown> (0x6), Model: <unknown> (0x8e), Stepping: 0xa
		Ext. family: 0x0, Ext. model: 0x8, Type: 0x0, Signature: 0x000806ea
		Features: ebx: 0x03100800, ecx: 0xfedaf387, edx: 0xbfebfbff
		Ext. features: eax: 0x00000000, ebx: 0x00000000, ecx: 0x00000121, edx: 0x2c100800
		Supports: On-Chip FPU, Virtual Mode Extensions, Debugging Extensions, Page Size Extensions, Time Stamp Counter, Model Specific Registers, Physical Address Extension, Machine Check Exceptions, CMPXCHG8B Instruction, On-Chip APIC, Fast System Call, Memory Type Range Registers, Page Global Enable, Machine Check Architecture, Conditional Mov Instruction, Page Attribute Table, 36-bit Page Size Extension, CLFLUSH Instruction, Debug Trace Store feature, ACPI registers in MSR space, Intel Architecture MMX Technology, Fast Float Point Save and Restore, Streaming SIMD extensions, Streaming SIMD extensions 2, Self-Snoop, Hyper Threading, Thermal Monitor, Streaming SIMD Extensions 3, PCLMULQDQ, 64-bit DS Area, Enhanced Intel SpeedStep technology, Thermal Monitor 2, Supplemental Streaming SIMD Extensions 3, Fused Multiply-Add, CMPXCHG16B, xTPR Update Control, Perfmon and Debug Capability, Process-context identifiers, Streaming SIMD extensions 4.1, Streaming SIMD extensions 4.2, MOVBE, Popcount instruction, AESNI, XSAVE, OSXSAVE, AVX, F16C, LAHF/SAHF instruction support, Advanced Bit Manipulations: LZCNT, SYSCALL/SYSRET, Execute Disable Bit, RDTSCP, Intel 64 Architecture, Invariant TSC
------------------------------------------------------------------------------

@ExpectNoJvmIssue

Today we consider this annotation as experimental.

With this annotation, JVM is profiled with Java Flight Recorder (JFR).

Based on the profiling, some JMC rules are evaluated. For each rule a score is attributed. The maximum score value is 100. The test will fail if one rule has a score greater than this expected (by default 60)

Things like significant primitives to object conversions can be detected:

[PERF] JMC rules are expected to have score less than <50>.

Rule: Primitive To Object Conversion
Severity: INFO
Score: 74
Message: 79 % of the total allocation (45,6 MiB) is caused by conversion from primitive 
types to object types.

The most common object type that primitives are converted into is 
'java.lang.Integer', which causes 45,6 MiB to be allocated. The most common 
call site is 
'org.quickperf.jvm.jmc.JmcJUnit4Tests$ClassWithFailingJmcRules$IntegerAccumulator.accumulateInteger(int):40'.

💡 With this annotation you can also detect that most of the time is spent to do garbage collection in your test.

💡 If you have the following message in the console

Rule: Stackdepth Setting
Severity: WARNING
Score: 97
Message: Some stack traces were truncated in this recording.

then you can increase the stack depth value in this way:

@JvmOptions("-XX:FlightRecorderOptions=stackdepth=128")

🔧 Elements

Name	Type	Meaning	Default value
score	int	Rule score (<=100)	60

Global JVM annotations

Annotations with a global scope apply to each test.

org.quickperf.jvm.annotation.JvmAnnotationBuilder and org.quickperf.jvm.jfr.annotation.JfrAnnotationBuilder help to configure JVM annotations with a global scope.

Test examples

With JUnit 4

With JUnit 5

With TestNG