Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
feat(clustering) retiring Serf - new cache invalidation mechanism #2561
This rewrite makes Kong independent when dealing with cache invalidation events. This opens the way to #2355 and to a simpler installation and usage experience for Kong.
As we know, Kong caches datastore entities (configuration entities mostly, such as APIs, Consumers, Plugins, Credentials...) to prevent unnecessary overhead and database round trips on a per-request basis. As of Kong 0.10, Serf has always been the dependency Kong relies upon to broadcast invalidation events to other members of the cluster. This approach is not without compromises, the main ones being:
TLDR: database polling of a time-series entity called "cluster events".
Cluster events are implemented in both PostgreSQL and Cassandra. A Kong node can
Here is a high-level, non-exhaustive list of changes:
This should be reviewed and heavily tested. There should be some more tests written, especially around the
Reviews and comments are welcome, questions as well! Local testing of this would be extremely helpful.
Regarding CI, the Travis builds might still fail at this point, partly for unrelated reasons (non-deterministic tests), partly because this is still in a "polishing" state.
Here is a small list of leftovers todos:
First round of comments, not enough time to walk deeply through everything today but wanted to get some thoughts on paper. Also, I notice several integration tests are failing consistently in a local env based on commit 689ae8c: https://gist.github.com/p0pr0ck5/6b31ddbcdf3a941aaccd3bb9314758e5
referenced this pull request
Jun 1, 2017
another incomplete review. But it's a start.
Unless I'm mistaken we're using a single cache, right? Will we be making entity specific caches (on Lua land) later? eg. grabbing an api from an lru cache that also holds 10000 consumers takes longer than necessary if we would split them up.
Using multiple caches might require a different cache-key setup, as it would also need to carry the info for which entity (which cache) it is about. If so, we must make sure that the new cache-key supports that way of working.
Similar for wrapping entity specific code. Eg. consumer fetching is duplicated all over the codebase, whereas we could implement a single module for consumers that would have
get_consumer_by_uuid as method and would hide all underlying invalidation, events, caches, etc. But I guess that's another refactor after this, but maybe good to agree on here.
@Tieske can you help me understand your meaning behind this comment:
LRU cache operations are O(1). There is no performance gain by using multiple LRU caches (if anything, it's just more complexity in management).
Jun 16, 2017
referenced this pull request
Nov 19, 2017
@olderwei In 0.11 you cannot: all Kong nodes are purely stateless. The