New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Multi Search in Searches to prevent large HTTP request headers #4149

Merged
merged 1 commit into from Sep 20, 2017

Conversation

Projects
None yet
2 participants
@joschi
Contributor

joschi commented Sep 14, 2017

Instead of artificially restricting the number of indices in a search request and resorting to using index wildcards if the list of indices threatens to hit the default HTTP header size limit of Elasticsearch (4 KB), the Searches class is now using the Multi Search API of Elasticsearch which allows specifying the list of indices in the request body instead of the URI path.

Refs https://www.elastic.co/guide/en/elasticsearch/reference/5.6/search-multi-search.html
Refs elastic/elasticsearch#26360
Refs #4054

@joschi joschi added this to the 2.4.0 milestone Sep 14, 2017

@bernd bernd self-assigned this Sep 19, 2017

@bernd

This comment has been minimized.

Member

bernd commented Sep 19, 2017

@joschi Can you please rebase against master? Thanks!

Use Multi Search in Searches to prevent large HTTP request headers
Instead of artificially restricting the number of indices in a search request and
resorting to using index wildcards if the list of indices threatens to hit the default
HTTP header size limit of Elasticsearch (4 KB), the `Searches` class is now using the
Multi Search API of Elasticsearch which allows specifying the list of indices in the
request body instead of the URI path.

Refs https://www.elastic.co/guide/en/elasticsearch/reference/5.6/search-multi-search.html
Refs elastic/elasticsearch#26360
Refs #4054

@joschi joschi force-pushed the elasticsearch-multi-search branch from f84f078 to 9f05fd1 Sep 19, 2017

@bernd

bernd approved these changes Sep 20, 2017

LGTM 👍 Way better than the index length limit hack. Good work!

@bernd bernd merged commit ec7f57e into master Sep 20, 2017

5 checks passed

ci-web-linter Jenkins build graylog-pr-linter-check 1925 has succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
graylog-project/pr Jenkins build graylog-project-pr-snapshot 473 has succeeded
Details
license/cla Contributor License Agreement is signed.
Details

@bernd bernd deleted the elasticsearch-multi-search branch Sep 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment