Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Elasticsearch 6.x clusters #623

Closed
rhaist opened this issue Jun 27, 2018 · 4 comments

Comments

Projects
None yet
4 participants
@rhaist
Copy link

commented Jun 27, 2018

Support the current Elasticsearch 6.x stack/clusters

Request Type

Bug

Work Environment

Question Answer
OS version (server) Debian
OS version (client) Stretch (stable)
TheHive version / git hash 3.x
Package Type DEB

Problem Description

During a test installation of the hive with our Elastic 6.x cluster the migration failed at version 13.

Steps to Reproduce

  1. Install thehive from the official repo on debian stretch
  2. Point to an external ES6 cluster
  3. Read logs.

Possible Solutions

The following issue upstream at Elastic might give further hints: https://discuss.elastic.co/t/unable-to-create-index-with-more-that-1-type-in-6-x/106089

Complementary information

Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Create a new empty database
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 2
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 3
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 4
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 5
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 6
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 7
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 8
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 9
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 10
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 11
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 12
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.s.MigrationSrv - Migrate database from version 0, add operations for version 13
Jun 27 10:04:03 hive authbind[16095]: [error] o.e.s.MigrationSrv - Migration fail
Jun 27 10:04:03 hive authbind[16095]: org.elasticsearch.transport.RemoteTransportException: [castleblack][10.0.0.1:9300][indices:admin/create]
Jun 27 10:04:03 hive authbind[16095]: Caused by: org.elasticsearch.transport.RemoteTransportException: [shadowtower][10.0.0.2:9300][indices:admin/create]
Jun 27 10:04:03 hive authbind[16095]: Caused by: java.lang.IllegalArgumentException: Rejecting mapping update to [the_hive_13] as the final mapping would have more than 1 type: [dblist, data, case_artifact_job, caseTemplate, case_task, reportTemplate, case_task_log, alert, audit, case_artifact, user, case, dashboard]
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:408)
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:356)
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:280)
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$IndexCreationTask.execute(MetaDataCreateIndexService.java:443)
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45)
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:630)
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:267)
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:197)
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:132)
Jun 27 10:04:03 hive authbind[16095]:         at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150)
Jun 27 10:04:03 hive authbind[16095]: [info] o.e.ErrorHandler - POST /api/maintenance/migrate returned 400

@saadkadhi saadkadhi closed this Jun 27, 2018

@To-om To-om added the question label Jul 31, 2018

@romans8

This comment has been minimized.

Copy link

commented Mar 22, 2019

Any movement on this? Elasticsearch doesn't do anything with 5.x anymore - it's only in critical bugfix mode

@romans8

This comment has been minimized.

Copy link

commented Apr 26, 2019

I know it was stated thing are moving to graphDB a couple months back in the next major release 4.0.

ES7 just launched and Kubernetes seem to like 6+.

How can I help things move forward and contribute? What's the best way?

@saadkadhi

This comment has been minimized.

Copy link
Contributor

commented May 6, 2019

We found out indeed and very recently that ES 5.6 is dead. We have published a blog post that I invite you to read at: https://blog.thehive-project.org/2019/05/06/an-apology/.

TL;DR we are currently working on having a supported ES version in TheHive & Cortex. We will come up with a concrete action plan in the upcoming days.

@To-om To-om self-assigned this May 15, 2019

@To-om To-om added enhancement and removed question labels May 15, 2019

@To-om To-om added this to the 3.4.0-RC1 milestone May 15, 2019

To-om added a commit that referenced this issue May 20, 2019

To-om added a commit that referenced this issue Jun 5, 2019

nadouani added a commit that referenced this issue Jun 5, 2019

To-om added a commit that referenced this issue Jul 9, 2019

To-om added a commit that referenced this issue Jul 9, 2019

To-om added a commit that referenced this issue Jul 10, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.