Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor lifecycle for lookup adapters and caches #3873

Merged
merged 16 commits into from Jun 12, 2017
Merged

Refactor lifecycle for lookup adapters and caches #3873

merged 16 commits into from Jun 12, 2017

Conversation

@kroepke
Copy link
Member

@kroepke kroepke commented May 29, 2017

we use auxiliary servicemanagers to start lookup tables and use listeners to maintain the dependent services' state

Graylog2/graylog-plugin-map-widget#48 needs to be merged once this one is in.

@kroepke kroepke added this to the 2.3.0 milestone May 29, 2017
@kroepke kroepke requested a review from bernd May 29, 2017
@bernd bernd self-assigned this May 29, 2017
@bernd bernd added this to In Progress in Lookup Tables Jun 7, 2017
@bernd bernd force-pushed the cache-lifecycle branch from bdca888 to cc9dca9 Jun 9, 2017
@bernd bernd force-pushed the cache-lifecycle branch from d52a6f8 to 9d61bbb Jun 10, 2017
@bernd bernd changed the title [WIP] add initial lifecycle to caches and make them services Refactor lifecycle for lookup adapters and caches Jun 12, 2017
kroepke and others added 15 commits May 29, 2017
we use auxiliary servicemanagers to start lookup tables and use listeners to maintain the dependent services' state
caches and data adapters now run on their own, without being lazily loaded when a lookup table uses them
this greatly simplfies the update and creation logic and makes it trivial to query the system state

not all update/delete events are filled in the resource yet
expose errors in errorstate api endpoint
avoid race condition by waiting for a running service before re-creating lookup tables
The live tables might not be populated yet.
fix wrong state in logging service listener
since we are sharing instances across lookup tables, the caches and adapters no longer have a back reference to the table
this means the lookup table needs to orchestrate loading and caching

only loading caches can use refreshAfterWrite, so we had to remove that option from the config

make sure restarting caches and adapters is properly synchronized by waiting for all instances to be running
The LookupCacheKey consists of the actual key object and a prefix String
to make it possible to cache the same key value for a different data
adapter. Since cache instances can be shared between data adapters,
cache values might be overwritten for different data adapters otherwise.
This decouples the data adapters from the caches and handles the refresh
logic outside of the data adapter.
@bernd bernd force-pushed the cache-lifecycle branch from 1e56cdb to f94127d Jun 12, 2017
@bernd
Copy link
Member

@bernd bernd commented Jun 12, 2017

@kroepke I would like to merge this PR so it doesn't get bigger and bigger.

Please review the two commits I added to move the refresh handling out of the adapters.

@kroepke

This comment has been minimized.

Do we need this to be @JsonCreator in case we want to serialize it ever?

This comment has been minimized.

Copy link
Member Author

@bernd bernd replied Jun 12, 2017

I thought we didn't need it because we do not deser it yet. But I will add @JsonCreator and @JsonProperty annotations to make sure it gets deser correctly. 👍

This comment has been minimized.

Copy link
Member Author

@bernd bernd replied Jun 12, 2017

@kroepke Added in 4fd2020

@kroepke
Copy link
Member Author

@kroepke kroepke commented Jun 12, 2017

lgtm!

@bernd
bernd approved these changes Jun 12, 2017
@bernd bernd merged commit 51157da into master Jun 12, 2017
1 of 4 checks passed
1 of 4 checks passed
@garybot2
graylog-project/pr Jenkins build graylog-project-pr-snapshot 209 has failed
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build is in progress
Details
@garybot2
ci-web-linter Jenkins build graylog-pr-linter-check 1731 has succeeded
Details
@bernd bernd deleted the cache-lifecycle branch Jun 12, 2017
@bernd bernd moved this from In Progress to Done in Lookup Tables Jun 14, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants