Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 188 lines (135 sloc) 7.794 kb
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
1 # HeapAudit
2
3 HeapAudit is a java agent which audits heap allocations for JVM processes.
4
1988a1c0 » norberthu
2012-02-22 Add support for hybrid mode
5 HeapAudit runs in three modes:
36e1443d » norberthu
2011-12-15 Implement support for dynamically injecting HeapAudit to JVM processes
6
7 - STATIC: This requires a simple integration hook to be implemented by the java
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
8 process of interest. The callback hook defines how the allocations are recorded
9 and the callback code is only executed when the java agent is loaded.
36e1443d » norberthu
2011-12-15 Implement support for dynamically injecting HeapAudit to JVM processes
10 - DYNAMIC: This injects HeapQuantile recorders to all matching methods and dumps
5488f16f » norberthu
2012-02-07 Add documentation
11 heap allocations to stdout when removed. Be aware, a lot of recorders, including
12 nested ones, may be injected if the supplied matching pattern is not restrictive
13 enough.
1988a1c0 » norberthu
2012-02-22 Add support for hybrid mode
14 - HYBRID: This launches like the static use case but dynamically determins where
15 to inject recorders.
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
16
c8fcc688 » norberthu
2012-03-05 Update documentations
17 ### Building and testing the HeapAudit java agent
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
18
19 Build project with Maven:
20
21 $ mvn clean package
22
23 The built jar will be in 'target/'.
24
9f517f7d » norberthu
2012-02-23 Add help message for command line
25 NOTE: The built jar package references the file name with version suffix in its
26 [manifest](https://github.com/foursquare/heapaudit/blob/master/pom.xml) boot
27 class path. If the built jar package were to be renamed, the path may be
28 required to be specified on the java command line.
29
aa5a4c8c » norberthu
2012-02-09 Update README with testing instructions
30 Because the included tests must be executed with the java agent attached, they
31 must run in the verify phase instead of in the test phase as unit tests:
32
33 $ mvn verify
34
c8fcc688 » norberthu
2012-03-05 Update documentations
35 ### Implementing the HeapAudit hook
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
36
37 Currently, two recorders are provided with HeapAudit:
38
5488f16f » norberthu
2012-02-07 Add documentation
39 - [HeapActivity](https://github.com/foursquare/heapaudit/blob/master/src/main/java/com/foursquare/heapaudit/HeapActivity.java)
40 prints each heap allocation to stdout as they occur
41 - [HeapQuantile](https://github.com/foursquare/heapaudit/blob/master/src/main/java/com/foursquare/heapaudit/HeapQuantile.java)
42 accumulates allocations and dumps out summary at the end
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
43
6433caad » norberthu
2011-12-14 Update README.md
44 Both of the above inherit from the base class HeapRecorder. Additional recording
45 behavior can be extended by implementing the record method in [HeapRecorder](https://github.com/foursquare/heapaudit/blob/master/src/main/java/com/foursquare/heapaudit/HeapRecorder.java).
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
46
6433caad » norberthu
2011-12-14 Update README.md
47 class MyRecorder extends HeapRecorder {
48 @Override public void record(String name,
49 int count,
50 long size) {
84580e3c » norberthu
2012-03-16 Add hybrid example
51 System.out.println("Allocated " + name + "[" + count + "] " + size + " bytes");
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
52 }
6433caad » norberthu
2011-12-14 Update README.md
53 }
54
c8fcc688 » norberthu
2012-03-05 Update documentations
55 ### Registering the HeapAudit recorder
5488f16f » norberthu
2012-02-07 Add documentation
56
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
57 Recording starts when it is registered and stops when it is unregistered. Each
58 recorder can be registered globally across all threads or local to the current.
5488f16f » norberthu
2012-02-07 Add documentation
59 The following example shows how to register the HeapActivity recorder across all
60 threads. The output will display as allocations occur.
61
62 HeapActivity r = new HeapActivity();
63 HeapRecorder.register(r, true);
64 MyObject o = new MyObject();
65 HeapRecorder.unregister(r, true);
66
67 The HeapQuantile recorder requires an extra step at the end to tally up the
68 results. The following example shows how to register the HeapQuantile recorder
69 only on the current thread and displays the summary at the end.
70
71 HeapQuantile r = new HeapQuantile();
6433caad » norberthu
2011-12-14 Update README.md
72 HeapRecorder.register(r, false);
73 MyObject o = new MyObject();
74 HeapRecorder.unregister(r, false);
5488f16f » norberthu
2012-02-07 Add documentation
75 for (HeapQuantile.Stats s: r.tally(false, true)) System.out.println(s);
76
c8fcc688 » norberthu
2012-03-05 Update documentations
77 ### Launching the HeapAudit java agent
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
78
36e1443d » norberthu
2011-12-15 Implement support for dynamically injecting HeapAudit to JVM processes
79 Launch HeapAudit statically along with the process of interest (requires MyTest
80 to implement the integration hook to register heap recorders).
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
81
82 $ java -javaagent:heapaudit.jar MyTest
83
5488f16f » norberthu
2012-02-07 Add documentation
84 Launch HeapAudit dynamically by injecting to the process of interest (does not
472ca259 » norberthu
2012-02-14 Properly cleanup instrumentation upon detaching
85 require MyTest to have any prior intrumentations). The recorder data is dumped
86 to the console upon exiting.
36e1443d » norberthu
2011-12-15 Implement support for dynamically injecting HeapAudit to JVM processes
87
8cd2c1fb » norberthu
2011-12-20 Re-implement HeapAudit args syntax
88 $ java -jar heapaudit.jar 999 -Icom/foursquare/test/MyTest@test.+
56364edb » norberthu
2012-02-16 Add interactive menu
89 Press <enter> to exit HeapAudit...
36e1443d » norberthu
2011-12-15 Implement support for dynamically injecting HeapAudit to JVM processes
90
8cd2c1fb » norberthu
2011-12-20 Re-implement HeapAudit args syntax
91 The JDK's tools.jar library is required to launch HeapAudit dynamically. If
92 launching within JRE, specify the -Xbootclasspath command line arg to point to
93 the tools.jar file.
94
95 $ java -Xbootclasspath/a:/usr/local/lib/tools.jar -jar heapaudit.jar 999 -Icom/foursquare/test/MyTest@test.+
56364edb » norberthu
2012-02-16 Add interactive menu
96 Press <enter> to exit HeapAudit...
97
98 Alternatively, an interactive menu will show if options are left off of the
99 command line.
100
101 $ java -jar heapaudit.jar
102 999 MyTest
103 PID: 999
104 OPTIONS: -Icom/foursquare/test/MyTest@test.+
105 Press <enter> to exit HeapAudit...
36e1443d » norberthu
2011-12-15 Implement support for dynamically injecting HeapAudit to JVM processes
106
1988a1c0 » norberthu
2012-02-22 Add support for hybrid mode
107 Another hybrid way is to statically instrument the process of interest at launch
108 time with dynamically injected recorders. This will provide an inclusive
109 collection from the moment the targeted process starts to the moment it exits
110 while not requiring prior code changes.
111
112 $ java -javaagent:heapaudit.jar=-Icom/foursquare/test/MyTest@test.+ MyTest
113
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
114 Additional options can be passed to HeapAudit to customize which classes and/or
115 methods are not to be instrumented for recording allocations. For additional
9f517f7d » norberthu
2012-02-23 Add help message for command line
116 information on how to specify the options, see [HeapAudit.java](https://github.com/foursquare/heapaudit/blob/master/src/main/java/com/foursquare/heapaudit/HeapAudit.java).
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
117
8cd2c1fb » norberthu
2011-12-20 Re-implement HeapAudit args syntax
118 $ java -javaagent:heapaudit.jar="-Acom/foursquare/test/.+" MyTest
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
119
c8fcc688 » norberthu
2012-03-05 Update documentations
120 ### Understanding HeapQuantile output
56364edb » norberthu
2012-02-16 Add interactive menu
121
122 The HeapQuantile recorder is intended to collect a concise summary of all the
123 allocations while still providing a meaningful breakdown of types, sizes and
124 occurrences.
125
126 For instance, given the following code:
127
128 public void Test() {
129 int array1D = new int[9];
130 int array3D = new int[8][29][5];
131 HashMap<Integer, Integer> map = new HashMap<Integer, Integer>();
132 for (int i = 0; i < 25; ++i) map.put(i, i);
133 }
134
135 the HeapQuantile recorder collects the following output:
136
137 HEAP: MyTest@test()V
138 - int[9] (56 bytes) x1
139 - int[1160] (9464 bytes) x1
140 - java.util.HashMap (48 bytes) x1
141 - java.util.HashMap$Entry (32 bytes) x25
142 - java.util.HashMap$Entry[16] (80 bytes) x1
143 - java.util.HashMap$Entry[32] (144 bytes) x1
144
145 The output starts with the HEAP prefix followed by the method signature in which
146 the recorder was injected. The subsequent lines each show a bucket of captured
1988a1c0 » norberthu
2012-02-22 Add support for hybrid mode
147 allocations with the format: \<TYPE\>\[\<AVG_ELEMENTS\>\] (\<AVG_BYTES\> bytes)
148 x\<OCCURRENCES\>
56364edb » norberthu
2012-02-16 Add interactive menu
149
150 You will notice that multiple lines of the same type may appear. This is because
151 each array type is further broken into separate buckets by element count over
152 power of 2s.
153
c8fcc688 » norberthu
2012-03-05 Update documentations
154 ### Troubleshooting
db26bb07 » norberthu
2012-02-08 Redirect console output to injector's console
155
156 Some libraries may not be instrumentable. This often manifests in some useless
157 error while instrumenting the target code, i.e. some generic error in the form
158 of a java.lang.NoClassDefFoundError exception. While some of the errors may be
159 attributed to bugs in HeapAudit, this should not block you from auditing the
160 rest of your code.
161
162 To identify the offending library, run with "-D.+" and search for the last line
163 of the debug output that starts with the word CLASS. You can subsequently avoid
164 instrumenting the identified class via the -A flag.
165
166 For instance, we recently switched to use jrockit JVM and it would not run with
167 HeapAudit attached. Upon troubleshooting with "-D.+", we noticed that many of
168 the classes under the jrockit/ namespace causes exception to be thrown during
169 startup. We subsequently ran HeapAudit with "-Ajrockit/.+" and everything
170 returned back to normal. See [HeapSettings.java](https://github.com/foursquare/heapaudit/blob/master/src/main/java/com/foursquare/heapaudit/HeapSettings.java)
171 for list of namespaces avoided by default.
172
3bbbb435 » norberthu
2012-03-15 Add sample tutorials
173 ### Tutorials
174
175 Check out some of the sample [tutorials](https://github.com/foursquare/heapaudit/blob/master/src/test/java/com/foursquare/heapaudit/tutorials/)
176 and play around with the examples!
177
c8fcc688 » norberthu
2012-03-05 Update documentations
178 ### Dependencies
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
179
180 - [ASM](http://asm.ow2.org/)
181
c8fcc688 » norberthu
2012-03-05 Update documentations
182 ### Other
9f517f7d » norberthu
2012-02-23 Add help message for command line
183
184 - [HeapAudit - JVM Memory Profiler for the Real World](http://engineering.foursquare.com/2012/02/02/heapaudit-jvm-memory-profiler-for-the-real-world)
185
c8fcc688 » norberthu
2012-03-05 Update documentations
186 ### Maintainers
89c1f94a » norberthu
2011-12-14 Add LICENSE and README
187
188 - Norbert Y. Hu norberthu@foursquare.com
Something went wrong with that request. Please try again.