Performance improvements #196

merged 5 commits into from Sep 29, 2015


None yet

4 participants


This PR contains performance improvements which are especially useful for high performance applications (I use json-c in rsyslog, among others). This PR will probably be amended by more improvements (depending on when it is merged). Please let me know any concerns you may have.

rgerhards added some commits Sep 22, 2015
@rgerhards rgerhards reduce duplicate hash computation in json_object_object_add()
This can be a very considerable performance saver.
@rgerhards rgerhards add json_object_object_add_ex() API
This provides more control over some detail aspects, many
of which are performance related.
@rgerhards rgerhards add perllike hash function for strings
This also adds a new API json_global_set_string_hash() which permits
to select the hash function. The default one is the only one that was
previously present. So there are no changes to existing apps, and the
new hash function needs to be explicitely be opted in. Especially for
smaller strings, the perllike functions seems to be around twice as
fast as the other one, with similarly good results in value distribution.
@rgerhards rgerhards remove unneeded data items from hashtable code
These items were used for statistics tracking, but no code at all
exists to consume them. By removing them we save

a) space
   because they counters required space, and did so in each and every
   json object

b) performance
   because calloc() needs to write less data and the counters are
   no longer maintained; cache performance can be better, load
   on OS main memory is lighter

We could conditionally enable/disable these counters, but I have not
done this they were really nowhere used and it looked more like a
left-over from the import of hashtable code.
@rgerhards rgerhards more efficient handling for smalls strings inside json_object
smalls strings inside json_objects had a high overhead because dynamic
memory allocation was needed for each of them. This also meant that the
pointer needed to be updated. This is now changed so that small strings
can directly be stored inside the json_object. Note that on the regular
64 bit machines a pointer takes 8 bytes. So even without increasing
memory, we could store string up to 7 bytes directly inside the object.
The max size is configurable. I have selected up to 31 bytes (which
means a buffer of 32 including the NUL byte). This brings a 24-bytes
memory overhead, but I consider that still useful because the memory
allocator usually also has quite some overhead (16 bytes) for
dyn alloced memory blocks. In any case, the max buffer size can be
tweaked via #define.
@hawicz hawicz merged commit 12916e2 into json-c:master Sep 29, 2015

Hi @rgerhards, for some reason this function is not included in latest release json-c-0.12.1-20160607 (despite the commit date), however rsyslog versions after 8.16.0 use it, leading to building failure with message "error: implicit declaration of function 'json_global_set_string_hash'". Could you, please, add a release version with this function included?



It's not included because the change was made on the master branch, and the 0607 release just pulled in selected changes to the (rather old) 0.12 branch. Yes, we need to cut another release at some point here. I've been waiting until I get a few more things done from my rather nebulous todo list, but I should probably push the rest off to a subsequent release.

Thank you @hawicz, cherry-picked this commit to 0607 release and it works fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment