Skip to content

Commit

Permalink
1. Add Java Instrumentation
Browse files Browse the repository at this point in the history
2. Update Javassist
  • Loading branch information
T5750 committed Feb 8, 2020
1 parent 79def54 commit 804907d
Show file tree
Hide file tree
Showing 17 changed files with 482 additions and 8 deletions.
1 change: 1 addition & 0 deletions doc/source/jdk8/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Java™ Platform Standard Ed. 8
jdkJre
Lambda
javap
instrumentation
asm
javassistTutorial
javassistLog
73 changes: 73 additions & 0 deletions doc/source/jdk8/instrumentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Java Instrumentation

## Introduction
[Java Instrumentation API](https://docs.oracle.com/javase/7/docs/api/java/lang/instrument/Instrumentation.html) provides the ability to add byte-code to existing compiled Java classes.

## Key Components
- Agent – is a jar file containing agent and transformer class files.
- Agent Class – A java class file, containing a method named `premain`.
- Manifest – `manifest.mf` file containing the `Premain-Class` property.
- Transformer – A Java class file implementing the interface `ClassFileTransformer`.

## What Is a Java Agent
In general, a java agent is just a specially crafted jar file. **It utilizes the [Instrumentation API](https://docs.oracle.com/javase/7/docs/api/java/lang/instrument/Instrumentation.html) that the JVM provides to alter existing byte-code that is loaded in a JVM.**

For an agent to work, we need to define two methods:
- `premain` – will statically load the agent using -javaagent parameter at JVM startup
- `agentmain` – will dynamically load the agent into the JVM using the [Java Attach API](https://docs.oracle.com/javase/7/docs/jdk/api/attach/spec/com/sun/tools/attach/package-summary.html)

### Instrumentation Activity Sequence
![Java-Instrumentation-Activity-Flow](https://s1.wailian.download/2020/02/08/Java-Instrumentation-Activity-Flow-min.jpg)

## Loading a Java Agent
We have two types of load:
- static – makes use of the `premain` to load the agent using `-javaagent` option
- dynamic – makes use of the `agentmain` to load the agent into the JVM using the [Java Attach API](https://docs.oracle.com/javase/7/docs/jdk/api/attach/spec/com/sun/tools/attach/package-summary.html)

### Static Load
Loading a Java agent at application startup is called static load. Static load modifies the byte-code at startup time before any code is executed.

Keep in mind that the static load uses the premain method, which will run before any application code runs, to get it running we can execute:
```
java -javaagent:agent.jar -jar application.jar
```

`Launcher` -> args: `StartMyAtmApplication 2 7 8`

### Dynamic Load
**The procedure of loading a Java agent into an already running JVM is called dynamic load.** The agent is attached using the [Java Attach API](https://docs.oracle.com/javase/7/docs/jdk/api/attach/spec/com/sun/tools/attach/package-summary.html).

A more complex scenario is when we already have our ATM application running in production and we want to add the total time of transactions dynamically without downtime for our application.
```
VirtualMachine jvm = VirtualMachine.attach(jvmPid);
jvm.loadAgent(agentFile.getAbsolutePath());
jvm.detach();
```

- Starting the Application: `java -jar application.jar StartMyAtmApplication`
- Attaching Java Agent: `java -jar application.jar LoadAgent`
* `Launcher` -> args: `LoadAgent`
- Check Application Logs

### APIs
- `addTransformer` – adds a transformer to the instrumentation engine
- `getAllLoadedClasses` – returns an array of all classes currently loaded by the JVM
- `retransformClasses` – facilitates the instrumentation of already loaded classes by adding byte-code
- `removeTransformer` – unregisters the supplied transformer
- `redefineClasses` – redefine the supplied set of classes using the supplied class files, meaning that the class will be fully replaced, not modified as with `retransformClasses`

## Creating a Java Agent
1. Create the `premain` and `agentmain` Methods: `Premain-transformClass(String className, Instrumentation instrumentation)`
2. Defining Our `ClassFileTransformer`: `AtmTransformer`
3. Creating an Agent Manifest File
- We can find the full list of manifest attributes in the [Instrumentation Package](https://docs.oracle.com/javase/7/docs/api/java/lang/instrument/package-summary.html) official documentation.
```
Agent-Class: com.baeldung.instrumentation.agent.MyInstrumentationAgent
Can-Redefine-Classes: true
Can-Retransform-Classes: true
Premain-Class: com.baeldung.instrumentation.agent.MyInstrumentationAgent
```

## References
- [Guide to Java Instrumentation](https://www.baeldung.com/java-instrumentation)
- [Java Instrumentation](https://javapapers.com/core-java/java-instrumentation/)
79 changes: 78 additions & 1 deletion doc/source/jdk8/javassistLog.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Javassist Audit Log
# Javassist/ASM Audit Log

With Spring and Hibernate on your stack, your application’s bytecode is likely enhanced or manipulated at runtime. Bytecode is the instruction set of the Java Virtual Machine (JVM), and all languages that run on the JVM must eventually compile down to bytecode. Bytecode is manipulated for a variety of reasons:

Expand Down Expand Up @@ -100,10 +100,87 @@ As you write Java code using Javassist, be wary of the following gotchas:
java -javaagent:jdk8.jar -cp .;C:\Users\hero\.m2\repository\org\javassist\javassist\3.26.0-GA\javassist-3.26.0-GA.jar t5750.module.log.BankTransactions
```

Debug: IDE -> VM options: `-javaagent:jdk8.jar`

### Summary
`ImportantLogClassTransformer`
- On the positive side, the amount of code written is pretty minimal and we did not actually have to write bytecode to use Javassist.
- The big drawback is that writing Java code in quotes can become tedious.

## How do we modify the bytes using ASM?
[ASM](http://asm.ow2.org/) is a bytecode manipulation framework that has a small memory footprint and is relatively fast. I consider ASM to be the industry standard for bytecode manipulation, as even Javassist uses ASM under the hood. ASM provides both object and event-based libraries, but here I’ll focus on the event-based model.

In ASM’s event-based model, all of these class components can be considered events.

![asm_Classes](https://s1.wailian.download/2020/02/07/asm_Classes-min.png)

The class events for ASM can be found on a `ClassVisitor`. In order “see” these events, you must create a classVisitor that overrides the desired components you want to see.

![asm_ClassVisitor](https://s1.wailian.download/2020/02/08/asm_ClassVisitor-min.png)

![asm_diagram](https://s1.wailian.download/2020/02/08/asm_diagram-min.png)

![asm_diagram_BankTrans](https://s1.wailian.download/2020/02/07/asm_diagram_BankTrans-min.png)

In addition to a class visitor, we need something to parse the class and generate events.
- ASM provides an object called a `ClassReader` for this purpose. The reader parses the class and produces events.
- After the class has been parsed, we need a `ClassWriter` to consume the events, converting them back to a class byte array.

```
public byte[] transform(ClassLoader loader, String className,
Class<?> classBeingRedefined, ProtectionDomain protectionDomain,
byte[] classfileBuffer) throws IllegalClassFormatException {
ClassReader cr = new ClassReader(classfileBuffer);
ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES);
ClassVisitor cv = new LogMethodClassVisitor(cw, className);
cr.accept(cv, 0);
return cw.toByteArray();
}
```
The `accept` call to the `ClassReader` says parse the class.
```
public class LogMethodClassVisitor extends ClassVisitor {
private String className;
public LogMethodClassVisitor(ClassVisitor cv, String pClassName) {
super(Opcodes.ASM6, cv);
className = pClassName;
}
@Override
public MethodVisitor visitMethod(int access, String name, String desc,
String signature, String[] exceptions) {
//put logic in here
}
}
```
Note that `visitAnnotation` returns an `AnnotationVisitor`.
```
public class PrintMessageMethodVisitor extends MethodVisitor {
@Override
public AnnotationVisitor visitAnnotation(String desc, boolean visible) {
// 1. check method for annotation @ImportantLog
// 2. if annotation present, then get important method param indexes
}
@Override
public void visitCode() {
// 3. if annotation present, add logging to beginning of the method
}
}
```

### Tips
As you write Java code using ASM, be wary of the following gotchas:
- In the event-model, the events for a class or method will always occur in a particular order. For example, the annotations on a method will always be visited before the actual code.
- When referencing method parameter values using `$1`, `$2`, etc., know that `$0` is reserved for `this`. This means the value of the first parameter to your method is `$1`.

```
java -javaagent:jdk8.jar -cp .;C:\Users\hero\.m2\repository\org\ow2\asm\asm\6.0\asm-6.0.jar;C:\Users\hero\.m2\repository\org\ow2\asm\asm\6.0\asm-util-6.0.jar t5750.module.log.BankTransactions
```

## Differences
One of the major differences between Javassist and ASM can be seen above. With ASM, you have to write code at the bytecode level when modifying methods, meaning you need to have a good understanding of how the JVM works. You need to know exactly what is on your stack and the local variables at a given moment of time. While writing at the bytecode level opens up the door in terms of functionality and optimization, it does mean ASM has a long developer ramp up time.

## References
- [Diving Into Bytecode Manipulation: Creating an Audit Log With ASM and Javassist](https://blog.newrelic.com/engineering/diving-bytecode-manipulation-creating-audit-log-asm-javassist/)
3 changes: 2 additions & 1 deletion jdk8/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
- [Lambda](../doc/source/jdk8/Lambda.md)
- [JDK, JRE, JVM, JSE, JEE, JME](../doc/source/jdk8/jdkJre.md)
- [javap](../doc/source/jdk8/javap.md)
- [Java Instrumentation](../doc/source/jdk8/instrumentation.md)

### ASM
- [ASM](../doc/source/jdk8/asm.md)
Expand All @@ -21,7 +22,7 @@
9. Boxing/Unboxing
10. Debug

[Javassist Audit Log](../doc/source/jdk8/javassistLog.md)
[Javassist/ASM Audit Log](../doc/source/jdk8/javassistLog.md)

## Runtime Environment
- [Java 8](http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html)
Expand Down
10 changes: 10 additions & 0 deletions jdk8/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
<java.version>1.8</java.version>
<javassist.version>3.26.0-GA</javassist.version>
<asm.version>6.0</asm.version>
<sun.tools.version>1.8.0</sun.tools.version>
<junit.version>4.11</junit.version>
<maven-jar-plugin.version>3.2.0</maven-jar-plugin.version>
</properties>
Expand All @@ -34,6 +35,13 @@
<artifactId>asm-util</artifactId>
<version>${asm.version}</version>
</dependency>
<dependency>
<groupId>com.sun</groupId>
<artifactId>tools</artifactId>
<version>${sun.tools.version}</version>
<scope>system</scope>
<systemPath>${java.home}/../lib/tools.jar</systemPath>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
Expand Down Expand Up @@ -68,6 +76,8 @@
<manifestEntries>
<Premain-Class>t5750.instrument.Premain</Premain-Class>
<Can-Retransform-Classes>true</Can-Retransform-Classes>
<Agent-Class>t5750.instrument.Premain</Agent-Class>
<Can-Redefine-Classes>true</Can-Redefine-Classes>
</manifestEntries>
</archive>
</configuration>
Expand Down
22 changes: 22 additions & 0 deletions jdk8/src/main/java/t5750/asm/visitor/LogMethodClassVisitor.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package t5750.asm.visitor;

import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.MethodVisitor;
import org.objectweb.asm.Opcodes;

public class LogMethodClassVisitor extends ClassVisitor {
private String className;

public LogMethodClassVisitor(ClassVisitor cv, String pClassName) {
super(Opcodes.ASM6, cv);
className = pClassName;
}

@Override
public MethodVisitor visitMethod(int access, String name, String desc,
String signature, String[] exceptions) {
MethodVisitor mv = super.visitMethod(access, name, desc, signature,
exceptions);
return new PrintMessageMethodVisitor(access, mv, name, className);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
package t5750.asm.visitor;

import static org.objectweb.asm.Opcodes.ASM6;

import java.util.ArrayList;
import java.util.List;

import org.objectweb.asm.AnnotationVisitor;
import org.objectweb.asm.MethodVisitor;
import org.objectweb.asm.Opcodes;

import t5750.module.log.util.LogUtil;

public class PrintMessageMethodVisitor extends MethodVisitor {
private String methodName;
private String className;
private boolean isAnnotationPresent = false;
private List parameterIndexes = new ArrayList<>();

public PrintMessageMethodVisitor(int access, MethodVisitor mv,
String methodName, String className) {
super(Opcodes.ASM6, mv);
this.methodName = methodName;
this.className = className;
}

@Override
public AnnotationVisitor visitAnnotation(String desc, boolean visible) {
if (LogUtil.ASM_METHOD_ANNOTATION.equals(desc)) {
isAnnotationPresent = true;
return new AnnotationVisitor(Opcodes.ASM6,
super.visitAnnotation(desc, visible)) {
public AnnotationVisitor visitArray(String name, Object value) {
if ("fields".equals(name)) {
return new AnnotationVisitor(ASM6,
super.visitArray(name)) {
public void visit(String name, Object value) {
parameterIndexes.add((String) value);
super.visit(name, value);
}
};
} else {
return super.visitArray(name);
}
}
};
}
return super.visitAnnotation(desc, visible);
}

@Override
public void visitCode() {
if (isAnnotationPresent) {
// create string builder
mv.visitFieldInsn(Opcodes.GETSTATIC, "java/lang/System", "out",
"Ljava/io/PrintStream;");
mv.visitTypeInsn(Opcodes.NEW, "java/lang/StringBuilder");
mv.visitInsn(Opcodes.DUP);
// add everything to the string builder
mv.visitLdcInsn("A call was made to method \"");
mv.visitMethodInsn(Opcodes.INVOKESPECIAL, "java/lang/StringBuilder",
"", "(Ljava/lang/String;)V", false);
mv.visitLdcInsn(methodName);
mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, "java/lang/StringBuilder",
"append", "(Ljava/lang/String;)Ljava/lang/StringBuilder;",
false);
}
}
}
23 changes: 23 additions & 0 deletions jdk8/src/main/java/t5750/instrument/AsmLogClassTransformer.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package t5750.instrument;

import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.security.ProtectionDomain;

import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.ClassWriter;

import t5750.asm.visitor.LogMethodClassVisitor;

public class AsmLogClassTransformer implements ClassFileTransformer {
public byte[] transform(ClassLoader loader, String className,
Class classBeingRedefined, ProtectionDomain protectionDomain,
byte[] classfileBuffer) throws IllegalClassFormatException {
ClassReader cr = new ClassReader(classfileBuffer);
ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES);
ClassVisitor cv = new LogMethodClassVisitor(cw, className);
cr.accept(cv, 0);
return cw.toByteArray();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,9 @@
import javassist.bytecode.annotation.ArrayMemberValue;
import javassist.bytecode.annotation.MemberValue;
import javassist.bytecode.annotation.StringMemberValue;
import t5750.module.log.util.LogUtil;

public class ImportantLogClassTransformer implements ClassFileTransformer {
private static final String METHOD_ANNOTATION = "t5750.module.log.annotation.ImportantLog";
private static final String ANNOTATION_ARRAY = "fields";
private ClassPool pool;

public ImportantLogClassTransformer() {
Expand Down Expand Up @@ -60,14 +59,14 @@ private Annotation getAnnotation(CtMethod method) {
.getAttribute(AnnotationsAttribute.invisibleTag);
if (attInfo != null) {
// this is the type name meaning use dots instead of slashes
return attInfo.getAnnotation(METHOD_ANNOTATION);
return attInfo.getAnnotation(LogUtil.METHOD_ANNOTATION);
}
return null;
}

private List getParamIndexes(Annotation annotation) {
ArrayMemberValue fields = (ArrayMemberValue) annotation
.getMemberValue(ANNOTATION_ARRAY);
.getMemberValue(LogUtil.ANNOTATION_ARRAY);
if (fields != null) {
MemberValue[] values = (MemberValue[]) fields.getValue();
List parameterIndexes = new ArrayList();
Expand Down

0 comments on commit 804907d

Please sign in to comment.