Report how much memory is allocated per rep

[GoogleCodeExporter]

Users need to understand if one variant of their benchmark allocates more 
memory than another. I believe we cannot assume that simply letting GC happen 
during the timing loop will properly account for this cost.

We would almost certainly use this:
http://code.google.com/p/java-allocation-instrumenter/

And we'd do it as a separate test, not while actual benchmarking is happening.

We may or may not want to try to conceive this as a "pluggable measurer" as 
described in http://code.google.com/p/caliper/issues/detail?id=4 ... what 
matters is that we offer the feature somehow.

Original issue reported on code.google.com by kevinb@google.com on 8 Jul 2010 at 4:02

[GoogleCodeExporter]

We might as well record both the bytes allocated and the number of instances 
allocated.

Should we just make a single measurement with reps=1?  Or sanity-check that by 
also trying reps=3 and reps=10 or whatever?  How should we react if we don't 
get linear results?

How should these numbers be presented in the web app?  For one thing the three 
statistics (bytes, instances, runtime) should probably be governed by 
checkboxes the way parameters and runs are.  In fact, one or more off them 
might even want to be off by default. (?)

(Side note when using the allocation instrumenter: the 'count' parameter is 
misdocumented.  It's simply the array.length if the object is an array, and -1 
otherwise.  We can ignore it; we get everything we need from just the size 
parameter.)

Original comment by kevinb@google.com on 13 Jul 2010 at 6:30

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

I'd prefer to do a small number of checks to make sure things grow linearly. If 
they don't, perhaps we could do the same thing that we do with runtime: just 
report a bunch of measurements.

Original comment by limpbizkit on 14 Jul 2010 at 12:36

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by kevinb@google.com on 16 Jul 2010 at 7:46

• Added labels: Milestone-1.0
• Removed labels: ****

[GoogleCodeExporter]

woooo.

Original comment by kevinb@google.com on 19 Mar 2011 at 2:39

• Changed state: Fixed
• Added labels: ****
• Removed labels: ****