StreamRouter interval should be configurable #2391
Closed
Comments
bernd
added a commit
that referenced
this issue
Jul 14, 2016
- Extractor filter - Stream router - Rules filter - Static field filter - In memory role permission resolver Fixes #2391
bernd
added a commit
that referenced
this issue
Jul 18, 2016
Previously every StreamRouter instance was polling for stream changes every second. Now it's listening on the server event bus for updates. Stream related REST endpoints are now posting events when updating stream configuration. Fixes #2391
bernd
added a commit
that referenced
this issue
Jul 18, 2016
Before this, every ExtractorFilter was using a cache which expired after 1 second for the extractors. So it was basically polling the extractors every second. Refs #2391
bernd
added a commit
that referenced
this issue
Jul 18, 2016
Before this, every StaticFieldFilter instance was using a cache which expired after 1 second. It was basically polling the static fields configuration every second. Refs #2391
bernd
added a commit
that referenced
this issue
Jul 18, 2016
Before this, every RulesFilter instance was using a cache which expired after 1 second so it was basically polling for new rules every second. Refs #2391
joschi
added a commit
that referenced
this issue
Jul 22, 2016
…2496) * Switch stream router from polling to subscription for updates Previously every StreamRouter instance was polling for stream changes every second. Now it's listening on the server event bus for updates. Stream related REST endpoints are now posting events when updating stream configuration. * Switch extractor filter from polling to subscription for updates Before this, every ExtractorFilter was using a cache which expired after 1 second for the extractors. So it was basically polling the extractors every second. * Switch static fields filter from polling to subscription for updates Before this, every StaticFieldFilter instance was using a cache which expired after 1 second. It was basically polling the static fields configuration every second. * Switch rules filter from polling to subscription for updates Before this, every RulesFilter instance was using a cache which expired after 1 second so it was basically polling for new rules every second. Fixes #2391
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
With hundreds of streams and dozens of graylog nodes, mongo load starts to be a (big) problem.
Obviously, this is mainly due to the hits done by StreamRouter every second for getting fresh rules.
As you can see on my Ganglia graph, switching this interval from 1 second to 1 minute saves me a lot of Mongo CPU.
I'm aware that this is introducing a minute of lag between stream creation and real routing and so probably need an explicit configuration directive in server.conf
The text was updated successfully, but these errors were encountered: