Penciller Memory and slicing binaries

[martinsumner]

The metadata for a riak object is directly extracted from the binary version of the riak object.

However, this is may mean that when the metadata is being held in the Penciller, it might have a reference to a shadow of the old object.

If, for example 10K 2MB objects are added (with 1KB metadata per object). When those 10K objects are in the penciller's memory, the penciller will have 10MB of binary data referenced - however, the reference is to part of a bigger binary, and although the rest of the bigger binary is not referenced, the whole of the bigger binary cannot be GC'd. Therefore, in effect the penciller has references to 20GB of binary heap memory.

[martinsumner]

#286

[martinsumner]

This is resolved by forcing a binary copy in the case where a section of the main object is binary is extracted to be the binary metadata.

Volume tests reveal a small improvement in throughput, and a reduction in the memory overhead of running Riak/leveled even when not bombarding Riak with large objects.