Implement kvitems in the C extension for increased performance.

[rtobar]

Currently the new kvitems method is implemented in python, which is what all backends use. The C backend however should see performance benefits by seeing this method implemented in C, with an expected increase of something around ~3x, 4x, depending on the use case.

See #18 (comment) and #18 (comment) for reference.

[rtobar]

@ltalirz could you check this one? As expected I'm getting about ~4x improvement on kvitems times for the C backend compared to the previous method, and (also as expected) kvitems seems to be slightly faster than items.

I also sped up a bit the python implementation of kvitems, which now seems to be ~10, 20% faster depending on the use case, as far as I could measure.

[ltalirz]

Hi @rtobar - thanks a lot for your efforts!

A few observations:

• Parsing the first 1M items of my JSON now takes ~16s (0.02ms/pair). That is within a factor of 2 of my revised timings for json.load, which is great.
• It seems like the C implementation leaks memory (it grows up to 2GB in my test)
• I checked that the pure python implementation uses ~13MB of memory throughout, i.e. the problem seems to be in the C extension

[rtobar]

@ltalirz oops, bad reference counting there... I just pushed some further fixes to the kvitems branch (which also boosts performance by yet another tiny bit, at least in my local benchmarks), please give it another nudge when possible.

[ltalirz]

Thanks - I'm now at 13s (0.01ms/pair), memory issues seem gone.

Looking great!

[rtobar]

Code merged to the master branch, closing issue now.