High object churn in DB Reader

[DigitalGoldfish]

We are using the GeoIP2-java library to read from the GeoIP2 database file. Our throughput is rather high with thousands of IPs resolved per second and we noticed that there is a high object churn in the Maxmind APIs with huge numbers of strings created only to be thrown away soon after again.

Are you aware of this issue? Any plans to improve the API in this aspect or any idea how we could improve this behaviour ourselves?

[oschwald]

Lookups create a significant number of string when the read the structure out of the database. Potentially an object pool could be created in the Decoder and used to cache objects rather than reading them out of the database on each lookup. This would reduce churn, although it might not actually speed up the lookups significantly unless you allowed it to use a very large amount of memory.

We currently do not have plans to change this behavior.

[oschwald]

Probably the easiest way to reduce churn is to cache key names, although we would want to limit the cache size as there is no limit on number of keys in an arbitrary database.

[DigitalGoldfish]

Thx for the input ... we haven't dealt with that issue yet. It's still in our backlog and gets occasionally remarked on by our monitoring guys but so far it is not enough pain to get it bumped to the top.

[phraktle]

+1 for this. The design of this API is not good enough for high-performance scenarios.

It's very wasteful that it dynamically creates an object tree of everything for a node (which then in the higher level API is mapped to actual objects), only to get the values for the few keys you really needed.

[phraktle]

Caching canonical string instances for map keys would indeed be a useful optimization which would not require a major redesign, so worth implementing.

However, an API redesign is definitely needed to improve things significantly. Two major gripes:

1. avoid reading all fields all the time (most requests really only need a few fields).
2. use proper objects / primitives, instead of bloating things unnecessarily with a JSON API (this should not even be a dependency, since there's no need for JSON here).

And another implementation detail that should be changed in the internal design: every decode operation allocates a new Decoder.Result instance (instead the offset could be advanced during decoding). This is the second largest source of garbage creation after String decoding.

[oschwald]

Thanks for your feedback. I largely agree with your two gripes for very high performance scenarios. Jackson was used for ease of deserializing the data to our GeoIP2 model classes, which are shared with the web service and already used Jackson. Although this design has the problems outlined here in terms of object churn, it does seem to meet the requirements of most of our customers.

[phraktle]

I would urge you to consider a redesign of this API. Yes, most people in the "enterprise" java space have such bloated libraries and code that this library is not a bottleneck. That said, it could be an order of magnitude faster with proper design. In our case, the one geoip lookup per request is about a third of all garbage created (and we do have a service solving a rather more complex problem :)

[phraktle]

For starters, would returning plain Java objects be a good direction? This eliminates the need for a dependency, but Jackson should still be able to bind to objects just as easily. But this would also reduce the need for the wrapper objects (i.e. a TextNode around every String, an ObjectNode around each Map, etc). (This is of course a breaking change for API clients, so this is more about discussing longer-term design directions)

[oschwald]

Could you expand on what you mean by plain Java objects here? The structure of the data record in the database isn't known ahead of time in the general case. Do you just mean a Map<String, Object> or an Object[]? I am not sure I like that change enough to do a major release around it. (In fact, if I recall correctly, a very early version did just that.)

I suspect for your particular case, based on you mentioning needing a "few keys", some sort of lookup-path method would be the best solution similar to the data lookup functions in the C API.

The Go API takes a different approach and only decodes what is necessary to fill out the struct passed to it. I suppose this could also be done with the Java API, but it would likely require a reasonable amount of work.

[phraktle]

Yes, I meant Map<String, Object> vs ObjectNode, String vs TextNode, etc. But indeed, this in itself is not going to yield great perf gains, so probably not worth bothering (just for the sake of a cleaner API :)

[phraktle]

Based on a quick experiment, it seems that introducing a reasonably sized shared cache for pointer objects would yield "good enough" results. With a few megabytes, you can get a 5x performance increase (due to the reduced memory churn and less pointer-chasing). You can take a look here and run the benchmark: https://github.com/phraktle/MaxMind-DB-Reader-java/tree/caching

For a production quality implementation, one would need a bounded cache of course. After a million random lookups, the cache has around 32k objects.

[phraktle]

Update: added some abstractions for pluggable caching and a very basic bounded cache (no eviction, just holds the first N items – this works perfectly well here, since the first few thousand items have all the hot items anyway). Let me know your thoughts on this, if it seems like a reasonable approach to you, I can pretty it up and submit a pull request. (Bottom line: ~5x perf improvement with ~2Mb cache footprint)

[oschwald]

Overall, this seems good. I am somewhat uncomfortable caching mutable objects, and I am not sure about lack of eviction policy. Have you tested this in comparison to just caching the keys? I assume that in addition to the keys, some hot countries are being cached, but so are many things that might be very uncommon.