Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to elasticsearch 6.x compatibility #5020

Merged
merged 11 commits into from Sep 4, 2018
Merged

Update to elasticsearch 6.x compatibility #5020

merged 11 commits into from Sep 4, 2018

Conversation

@kroepke
Copy link
Member

@kroepke kroepke commented Aug 24, 2018

Primarily adapt index mapping template changes for 6.x backends.
Updated jest+jackson dependency for minor URL compatibility with 6.x servers.

Add elasticsearch request logger for debugging request/response layer: enable with TRACE on org.graylog2.indexer.cluster.jest.RequestResponseLogger

fixes #4957

kroepke added 8 commits Jul 31, 2018
location is based on bounding boxes of the world's countries to get a bit more realistic looking data
otherwise a third is going to be in the pacific...
add request logging to elasticsearch client for easier debugging
@kroepke
Copy link
Member Author

@kroepke kroepke commented Aug 24, 2018

This also enhances the random http input to provide geolocation information for easier testing of map features.

@kroepke kroepke requested a review from bernd Aug 24, 2018
@bernd bernd self-assigned this Aug 24, 2018
final Map<String, Object> settings = ImmutableMap.of("analysis", analysis);
// mapping types are deprecated, 7.0 and later will only allow `_doc` here, so we start using that
// see https://www.elastic.co/guide/en/elasticsearch/reference/6.3/removal-of-types.html#_schedule_for_removal_of_mapping_types
final Map<String, Object> mappings = ImmutableMap.of(IndexMapping.TYPE_MESSAGE, messageMapping(analyzer));

This comment has been minimized.

@bernd

bernd Aug 24, 2018
Member

Is the comment or the code wrong here? The comment says that we are using _doc but the code is actually still using message.

@bernd
Copy link
Member

@bernd bernd commented Aug 24, 2018

@kroepke Please add the missing license headers.

@bernd
Copy link
Member

@bernd bernd commented Aug 24, 2018

I found this log message:

[2018-08-24T18:54:25,832][WARN ][o.e.d.a.a.i.t.p.PutIndexTemplateRequest] Deprecated field [template] used, replaced by [index_patterns]

Are we ignoring this warning for now?

@kroepke
Copy link
Member Author

@kroepke kroepke commented Aug 28, 2018

Yes, that deprecation warning we'll likely ignore for now because we'll fix it in 3.0 anyway.

@bernd bernd added this to the 2.5.0 milestone Aug 29, 2018
@bernd
bernd approved these changes Sep 4, 2018
Copy link
Member

@bernd bernd left a comment

I did some basic testing with ES 2.4, ES 5.6 and ES 6.4.

  • Message ingest
  • Using some widgets
  • Manual index rotation
  • Archiving
@bernd bernd merged commit 5f00f3d into 2.5 Sep 4, 2018
2 of 3 checks passed
2 of 3 checks passed
@garybot2
ci-web-linter Jenkins build graylog-pr-linter-check 2703 has failed
Details
@garybot2
graylog-project/pr Jenkins build graylog-project-pr-snapshot 1786 has succeeded
Details
license/cla Contributor License Agreement is signed.
Details
@bernd bernd deleted the es6-mapping branch Sep 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants