New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tasks Tab Elasticsearch exception: Fielddata is disabled on text fields by default. Set fielddata=true on [title] #311

Closed
functure opened this Issue Sep 18, 2017 · 1 comment

Comments

Projects
None yet
2 participants
@functure

functure commented Sep 18, 2017

Request Type

Bug

Work Environment

Question Answer
OS version (server) Ubuntu Server 16.04
OS version (client) Ubuntu
TheHive version / git hash 2.13.0-1
Package Type DEB
Browser type & version If applicable
ElasticSearch version 5.6.0

Problem Description

Assume we have one case containing one or more tasks. When user clicks Tasks part in Case page, Elasticsearch throws exception and user is unable to browse tasks. User still can see or take any task from Waiting Tasks page.

Steps to Reproduce

  1. Click on New Case
  2. Fill mandatory fields, create one or more tasks
  3. Click Create Case and wait for redirection to Case page
  4. Click on Tasks tab. See red box on upper left.

Complementary information

Logs & Traceback:

[2017-09-18T17:35:42,519][DEBUG][o.e.a.s.TransportSearchAction] [DiQW4zf] [the_hive_10][4], node[DiQW4zfkSxi5i-IWW0Ij3A], [P], s[STARTED], a[id=SH7amUIAQjSBZ-QSip5jEw]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[the_hive_10], indicesOptions=IndicesOptions[id=38, ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_alisases_to_multiple_indices=true, forbid_closed_indices=true], types=[case_task], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=5, batchedReduceSize=512, preFilterShardSize=128, source={
  "from" : 0,
  "size" : 30,
  "query" : {
    "bool" : {
      "must" : [
        {
          "bool" : {
            "must" : [
              {
                "has_parent" : {
                  "query" : {
                    "ids" : {
                      "type" : [ ],
                      "values" : [
                        "AV6Van-gA3Pv3sSafPeV"
                      ],
                      "boost" : 1.0
                    }
                  },
                  "parent_type" : "case",
                  "score" : false,
                  "ignore_unmapped" : false,
                  "boost" : 1.0
                }
              },
              {
                "bool" : {
                  "must_not" : [
                    {
                      "term" : {
                        "status" : {
                          "value" : "Cancel",
                          "boost" : 1.0
                        }
                      }
                    }
                  ],
                  "disable_coord" : false,
                  "adjust_pure_negative" : true,
                  "boost" : 1.0
                }
              }
            ],
            "disable_coord" : false,
            "adjust_pure_negative" : true,
            "boost" : 1.0
          }
        }
      ],
      "disable_coord" : false,
      "adjust_pure_negative" : true,
      "boost" : 1.0
    }
  },
  "stored_fields" : [
    "_source",
    "_routing",
    "_parent"
  ],
  "sort" : [
    {
      "flag" : {
        "order" : "desc"
      }
    },
    {
      "order" : {
        "order" : "asc"
      }
    },
    {
      "startDate" : {
        "order" : "asc"
      }
    },
    {
      "title" : {
        "order" : "asc"
      }
    },
    {
      "_uid" : {
        "order" : "desc"
      }
    }
  ]
}}]
org.elasticsearch.transport.RemoteTransportException: [DiQW4zf][127.0.0.1:9300][indices:data/read/search[phase/query]]
Caused by: java.lang.IllegalArgumentException: Fielddata is disabled on text fields by default. Set fielddata=true on [title] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
	at org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:336) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.index.fielddata.IndexFieldDataService.getForField(IndexFieldDataService.java:111) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.index.query.QueryShardContext.getForField(QueryShardContext.java:166) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.sort.FieldSortBuilder.build(FieldSortBuilder.java:277) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.sort.SortBuilder.buildSort(SortBuilder.java:156) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.SearchService.parseSource(SearchService.java:634) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.SearchService.createContext(SearchService.java:485) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:461) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:340) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:337) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:644) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.0.jar:5.6.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
[2017-09-18T17:35:42,521][DEBUG][o.e.a.s.TransportSearchAction] [DiQW4zf] All shards failed for phase: [query]
org.elasticsearch.ElasticsearchException$1: Fielddata is disabled on text fields by default. Set fielddata=true on [title] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
	at org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:618) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:126) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:241) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:90) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.search.InitialSearchPhase.access$100(InitialSearchPhase.java:46) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.search.InitialSearchPhase$1.onFailure(InitialSearchPhase.java:169) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1067) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1171) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1149) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.transport.TransportService$7.onFailure(TransportService.java:655) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:623) [elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39) [elasticsearch-5.6.0.jar:5.6.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: java.lang.IllegalArgumentException: Fielddata is disabled on text fields by default. Set fielddata=true on [title] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
	at org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:336) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.index.fielddata.IndexFieldDataService.getForField(IndexFieldDataService.java:111) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.index.query.QueryShardContext.getForField(QueryShardContext.java:166) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.sort.FieldSortBuilder.build(FieldSortBuilder.java:277) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.sort.SortBuilder.buildSort(SortBuilder.java:156) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.SearchService.parseSource(SearchService.java:634) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.SearchService.createContext(SearchService.java:485) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:461) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:340) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:337) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:644) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) ~[elasticsearch-5.6.0.jar:5.6.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.6.0.jar:5.6.0]
	... 3 more
@To-om

This comment has been minimized.

Show comment
Hide comment
@To-om

To-om Sep 18, 2017

Contributor

This bug exists only for newly created index. If you create the ElasticSearch index with TheHive 2.12 and migration to TheHive 2.13, you should not have this issue.

With ElasticSearch 5.x, fielddata is disabled by default for text fields. Then field can't be used for aggregation.

This issue've just been fixed in (TheHive-Project/elastic4play#31) which is in TheHive 2.13.1

Contributor

To-om commented Sep 18, 2017

This bug exists only for newly created index. If you create the ElasticSearch index with TheHive 2.12 and migration to TheHive 2.13, you should not have this issue.

With ElasticSearch 5.x, fielddata is disabled by default for text fields. Then field can't be used for aggregation.

This issue've just been fixed in (TheHive-Project/elastic4play#31) which is in TheHive 2.13.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment