/graylog2-server

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Index range handling slow with ES 1.x #1465

Closed
kroepke opened this Issue Oct 6, 2015 · 0 comments

Comments

Assignees

@joschi joschi

Labels
bug
Projects
None yet
Milestone
  1.2.2
2 participants
@kroepke @joschi
@kroepke
Member

kroepke commented Oct 6, 2015

Storing the index range information in Elasticsearch can lead to excessive cluster state exchanges, which can seriously impact performance in certain situations.

Compare https://groups.google.com/forum/#!topic/graylog2/obgyM74FMsE

@kroepke kroepke added the bug label Oct 6, 2015

@kroepke kroepke added this to the 1.2.2 milestone Oct 6, 2015

@kroepke kroepke modified the milestones: 1.3.0, 1.2.2 Oct 13, 2015

joschi added a commit that referenced this issue Oct 19, 2015

@joschi
Implement MongoIndexRangeService and make it new default IndexRangeSe… 
…rvice

Index range information is now stored in MongoDB again, but in the format being used in Elasticsearch
and not in the legacy format (which only included partial information).

The IndexRangesMigrationPeriodical has been extended to now migrate old index ranges from MongoDB
(Graylog 1.1.x and earlier) and the current index ranges from Elasticsearch into MongoDB and will
only run the migration once.

EsIndexRangeService is now deprecated and shouldn't be used anymore (except for the migration of
old index range information from Elasticsearch in IndexRangesMigrationPeriodical).

Fixes #1465
1cc0583

@joschi joschi referenced this issue Oct 19, 2015

Merged

Move index ranges back into MongoDB #1493

joschi added a commit that referenced this issue Oct 19, 2015

@joschi
Implement MongoIndexRangeService and make it new default IndexRangeSe… 
…rvice

Index range information is now stored in MongoDB again, but in the format being used in Elasticsearch
and not in the legacy format (which only included partial information).

The IndexRangesMigrationPeriodical has been extended to now migrate old index ranges from MongoDB
(Graylog 1.1.x and earlier) and the current index ranges from Elasticsearch into MongoDB and will
only run the migration once.

EsIndexRangeService is now deprecated and shouldn't be used anymore (except for the migration of
old index range information from Elasticsearch in IndexRangesMigrationPeriodical).

Fixes #1465
Loading status checks…
22486d4

@kroepke kroepke added the 0 - Backlog label Oct 21, 2015

@joschi joschi self-assigned this Oct 26, 2015

@joschi joschi closed this Oct 26, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment