serialize to bytebuffer

[richardstartin]

No description provided.

[lemire]

This solves #322

[lemire]

Looks like great work to me. Thorough.

[richardstartin]

Thanks. I haven't benchmarked it but I'm sure this will soften the CPU spikes I see during serialisation of many sparse bitmaps so I am keen to pick up the next release. Please ping me when you get time to issue a release.

[lemire]

@richardstartin

Working on it right now. I just want to update the documentation before I issue a release.

Even if the performance was the same, the usage is now so much nicer...

[lemire]

I have release 0.8.0. I want with a new minor version because I think we have a much better API.

[richardstartin]

I notice that Spark may be picking up 0.7.45 for Kryo serialisation. It may be worth their while to use ByteBuffer in preference to DataOutput.

[richardstartin]

I can't share more details but this reduced a "CPU utilisation" spike from 50% to under 15%. It was a big efficiency gain.

[lemire]

@richardstartin

I fear that this long lasting inefficiency is largely my fault. I was assuming that if there were any performance differences with such functions, it would be small. I held to my assumption for too long without checking.