Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Casting errors under load #120

Closed
clintongormley opened this issue Apr 6, 2010 · 4 comments
Closed

Casting errors under load #120

clintongormley opened this issue Apr 6, 2010 · 4 comments

Comments

@clintongormley
Copy link

Hiya

I am unable to replicate this, but I've seen this issue a lot, especially when the servers are under load.

I was in the process of indexing a lot of new docs. Sometimes the CPU would be running at 100% plus.

{
  request   => {
       cmd => "/ia_object/notice/_search?search_type=query_then_fetch",
       data => {
       fields => [],
       from   => 1332200,
       query  => {
           filtered => {
             filter => {
               bool => {
                 must => [
                   { term => { status => "active" } },
                   { term => { location_id => 23 } },
                 ],
                 },
               },
             query  => { match_all => {} },
           },
           },
       size   => 100,
       "sort" => [{ publish_date => { "reverse" => 1 } }],
       },
       method => "GET",
     },
  response  => {
       error => "ReduceSearchPhaseException[Failed to execute [query] [reduce] ]; nested: ClassCastException[java.lang.Float cannot be cast to java.lang.Long]; ",
     },
  server  => "http://192.168.10.41:9200",
  status_code => 500,
  status_msg  => "Internal Server Error",
}

Another:
{
request => {
cmd => "/ia_object/notice/_search?search_type=dfs_query_then_fetch",
data => {
fields => [],
from => 0,
query => {
filtered => {
filter => {
bool => {
must => [
{ term => { sub_type => "obit" } },
{ term => { status => "active" } },
{ term => { location_id => 28 } },
],
},
},
query => {
dis_max => {
queries => [
{
query_string => { boost => 1, fields => ["name"], query => ""Sammy Brian Happer"~8" },
},
{
filtered => {
filter => { term => { has_name => 0 } },
query => {
query_string => { boost => "1.3", fields => ["text"], query => ""Sammy Brian Happer"~8" },
},
},
},
{ query_string => { boost => 1, query => "Sammy Brian Happer" } },
],
tie_breaker => "0.7",
},
},
},
},
size => 100,
"sort" => ["score"],
},
method => "GET",
},
response => {
error => "ReduceSearchPhaseException[Failed to execute [query] [reduce] ]; nested: ClassCastException[java.lang.Float cannot be cast to java.lang.Long]; ",
},
server => "http://192.168.10.41:9200",
status_code => 500,
status_msg => "Internal Server Error",
}

@clintongormley
Copy link
Author

I enabled debugging, and put the server under load, and am seeing logs like this:

[12:57:56,059][DEBUG][action.search.type       ] [Dragonwing] Failed to execute query phase
org.elasticsearch.search.query.QueryPhaseExecutionException: [ia_object_1270224642][0]: query[filtered((name:"ray woolston"~7 | filtered(text:"ray woolston"~7^1.3)->FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@3e3f4f7) | (_all:ray _all:woolston))~0.7)->BooleanFilter( +FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@3a1bb115) +FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@8d6b4fd5) +FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@5d5d3c26) +publish_date:[1267833600000 TO *])],from[0],size[300],sort[<score>]: Query Failed []
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:98)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:156)
    at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:128)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeQuery(TransportSearchDfsQueryThenFetchAction.java:140)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$200(TransportSearchDfsQueryThenFetchAction.java:64)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$1.run(TransportSearchDfsQueryThenFetchAction.java:113)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalArgumentException: df for term ray not available
    at org.elasticsearch.search.dfs.CachedDfSource.docFreq(CachedDfSource.java:51)
    at org.apache.lucene.search.Similarity.idfExplain(Similarity.java:769)
    at org.apache.lucene.search.PhraseQuery$PhraseWeight.<init>(PhraseQuery.java:122)
    at org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:264)
    at org.apache.lucene.search.DisjunctionMaxQuery$DisjunctionMaxWeight.<init>(DisjunctionMaxQuery.java:107)
    at org.apache.lucene.search.DisjunctionMaxQuery.createWeight(DisjunctionMaxQuery.java:184)
    at org.apache.lucene.search.FilteredQuery.createWeight(FilteredQuery.java:63)
    at org.apache.lucene.search.FilteredQuery.createWeight(FilteredQuery.java:63)
    at org.apache.lucene.search.Query.weight(Query.java:101)
    at org.elasticsearch.search.internal.ContextIndexSearcher.createWeight(ContextIndexSearcher.java:64)
    at org.apache.lucene.search.Searcher.search(Searcher.java:49)
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:92)
    ... 8 more
[12:57:56,060][DEBUG][action.search.type       ] [Dragonwing] Failed to execute query phase
org.elasticsearch.search.query.QueryPhaseExecutionException: [ia_object_1270224642][4]: query[filtered((name:"ray woolston"~7 | filtered(text:"ray woolston"~7^1.3)->FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@3e3f4f7) | (_all:ray _all:woolston))~0.7)->BooleanFilter( +FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@3a1bb115) +FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@8d6b4fd5) +FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@5d5d3c26) +publish_date:[1267833600000 TO *])],from[0],size[300],sort[<score>]: Query Failed []
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:98)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:156)
    at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:128)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeQuery(TransportSearchDfsQueryThenFetchAction.java:140)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$200(TransportSearchDfsQueryThenFetchAction.java:64)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$1.run(TransportSearchDfsQueryThenFetchAction.java:113)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalArgumentException: df for term ray not available
    at org.elasticsearch.search.dfs.CachedDfSource.docFreq(CachedDfSource.java:51)
    at org.apache.lucene.search.Similarity.idfExplain(Similarity.java:769)
    at org.apache.lucene.search.PhraseQuery$PhraseWeight.<init>(PhraseQuery.java:122)
    at org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:264)
    at org.apache.lucene.search.DisjunctionMaxQuery$DisjunctionMaxWeight.<init>(DisjunctionMaxQuery.java:107)
    at org.apache.lucene.search.DisjunctionMaxQuery.createWeight(DisjunctionMaxQuery.java:184)
    at org.apache.lucene.search.FilteredQuery.createWeight(FilteredQuery.java:63)
    at org.apache.lucene.search.FilteredQuery.createWeight(FilteredQuery.java:63)
    at org.apache.lucene.search.Query.weight(Query.java:101)
    at org.elasticsearch.search.internal.ContextIndexSearcher.createWeight(ContextIndexSearcher.java:64)
    at org.apache.lucene.search.Searcher.search(Searcher.java:49)
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:92)
    ... 8 more
[13:03:46,546][DEBUG][action.search.type       ] [Dragonwing] Failed to execute query phase
org.elasticsearch.search.SearchContextMissingException: No search context found for id [8144], timed out
    at org.elasticsearch.search.SearchService.findContext(SearchService.java:212)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:150)
    at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:128)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeQuery(TransportSearchDfsQueryThenFetchAction.java:140)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$200(TransportSearchDfsQueryThenFetchAction.java:64)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$1.run(TransportSearchDfsQueryThenFetchAction.java:113)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
[13:03:46,546][DEBUG][action.search.type       ] [Dragonwing] Failed to execute fetch phase
org.elasticsearch.search.SearchContextMissingException: No search context found for id [8142], timed out
    at org.elasticsearch.search.SearchService.findContext(SearchService.java:212)
    at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:200)
    at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:278)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchDfsQueryThenFetchAction.java:224)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$700(TransportSearchDfsQueryThenFetchAction.java:64)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$4.run(TransportSearchDfsQueryThenFetchAction.java:197)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
[13:03:46,547][DEBUG][action.search.type       ] [Dragonwing] Failed to execute fetch phase
org.elasticsearch.search.SearchContextMissingException: No search context found for id [8145], timed out
    at org.elasticsearch.search.SearchService.findContext(SearchService.java:212)
    at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:200)
    at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:278)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchDfsQueryThenFetchAction.java:224)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$700(TransportSearchDfsQueryThenFetchAction.java:64)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$4.run(TransportSearchDfsQueryThenFetchAction.java:197)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
[13:03:46,547][DEBUG][action.search.type       ] [Dragonwing] Failed to execute query phase
org.elasticsearch.search.SearchContextMissingException: No search context found for id [8145], timed out
    at org.elasticsearch.search.SearchService.findContext(SearchService.java:212)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:150)
    at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:128)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeQuery(TransportSearchDfsQueryThenFetchAction.java:140)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$200(TransportSearchDfsQueryThenFetchAction.java:64)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$1.run(TransportSearchDfsQueryThenFetchAction.java:113)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
[13:03:46,548][DEBUG][action.search.type       ] [Dragonwing] Failed to execute fetch phase
org.elasticsearch.search.SearchContextMissingException: No search context found for id [8144], timed out
    at org.elasticsearch.search.SearchService.findContext(SearchService.java:212)
    at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:200)
    at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:278)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchDfsQueryThenFetchAction.java:224)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$700(TransportSearchDfsQueryThenFetchAction.java:64)
    at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$4.run(TransportSearchDfsQueryThenFetchAction.java:197)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)

@clintongormley
Copy link
Author

Just had another casting error:

ReduceSearchPhaseException[Failed to execute [query] [reduce] ]; nested: ClassCastException[java.lang.Float cannot be cast to java.lang.Long]; 

on this query:

curl -XGET 'http://192.168.10.42:9200/ia_object/notice/_search?search_type=dfs_query_then_fetch'  -d '
{
   "sort" : [
      "score"
   ],
   "fields" : [],
   "from" : 0,
   "query" : {
      "filtered" : {
         "filter" : {
            "bool" : {
               "must" : [
                  {
                     "term" : {
                        "notice_type" : "all_memorial"
                     }
                  },
                  {
                     "term" : {
                        "status" : "active"
                     }
                  },
                  {
                     "term" : {
                        "location_id" : "30"
                     }
                  }
               ]
            }
         },
         "query" : {
            "dis_max" : {
               "queries" : [
                  {
                     "query_string" : {
                        "fields" : [
                           "name"
                        ],
                        "boost" : 1,
                        "query" : "\"Carmela Hitchings\"~7"
                     }
                  },
                  {
                     "filtered" : {
                        "filter" : {
                           "term" : {
                              "has_name" : "0"
                           }
                        },
                        "query" : {
                           "query_string" : {
                              "fields" : [
                                 "text"
                              ],
                              "boost" : "1.3",
                              "query" : "\"Carmela Hitchings\"~7"
                           }
                        }
                     }
                  },
                  {
                     "query_string" : {
                        "boost" : 1,
                        "query" : "Carmela Hitchings"
                     }
                  }
               ],
               "tie_breaker" : "0.7"
            }
         }
      }
   },
   "size" : "300"
}
'

with this in the logs of one node:

[11:54:07,863][DEBUG][action.search.type       ] [Longshot] [ia_object_1271065315][4], Node[idb2-11081], [P], S[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@54ed10ea]
org.elasticsearch.transport.RemoteTransportException: None remote transport exception
Caused by: org.elasticsearch.transport.ResponseHandlerFailureTransportException: Failed to handler response
        at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:104)
        at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:76)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:391)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:345)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:323)
        at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:275)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:196)
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
        at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.util.NoSuchElementException: No active shard found
        at org.elasticsearch.cluster.routing.IndexShardRoutingTable$IndexShardsIterator.nextActive(IndexShardRoutingTable.java:159)
        at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:194)
        at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.access$100(TransportSearchTypeAction.java:78)
        at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onResult(TransportSearchTypeAction.java:178)
        at org.elasticsearch.search.action.SearchServiceTransportAction$1.handleResponse(SearchServiceTransportAction.java:92)
        at org.elasticsearch.search.action.SearchServiceTransportAction$1.handleResponse(SearchServiceTransportAction.java:85)
        at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:102)
        ... 21 more
[12:21:20,202][DEBUG][action.search.type       ] [Longshot] Failed to execute query phase
org.elasticsearch.search.query.QueryPhaseExecutionException: [ia_object_1271065315][2]: query[filtered((name:"stephanie gregory"~7 | filtered(text:"stephanie gregory"~7^1.3)->FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@3e3f4f7) | (_all:stephanie _all:gregory))~0.7)->BooleanFilter( +FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@fdb5bf46) +FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@3a1bb115) +FilterCacheFilterWrapper(org.elasticsearch.util.lucene.search.TermFilter@8d6b4fd7))],from[0],size[300],sort[<score>]: Query Failed []
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:98)
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:197)
        at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:139)
        at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeQuery(TransportSearchDfsQueryThenFetchAction.java:140)
        at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$200(TransportSearchDfsQueryThenFetchAction.java:64)
        at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$1.run(TransportSearchDfsQueryThenFetchAction.java:113)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalArgumentException: df for term stephanie not available
        at org.elasticsearch.search.dfs.CachedDfSource.docFreq(CachedDfSource.java:51)
...skipping...
        at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:289)
        at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchDfsQueryThenFetchAction.java:227)
        at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$700(TransportSearchDfsQueryThenFetchAction.java:64)
        at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$4.run(TransportSearchDfsQueryThenFetchAction.java:198)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
[12:23:42,706][DEBUG][action.search.type       ] [Longshot] Failed to execute fetch phase
org.elasticsearch.search.SearchContextMissingException: No search context found for id [348292], timed out
        at org.elasticsearch.search.SearchService.findContext(SearchService.java:278)
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:261)
        at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:289)
        at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchDfsQueryThenFetchAction.java:227)
        at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.access$700(TransportSearchDfsQueryThenFetchAction.java:64)
        at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$4.run(TransportSearchDfsQueryThenFetchAction.java:198)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
[12:23:42,756][DEBUG][action.search.type       ] [Longshot] Failed to execute fetch phase
org.elasticsearch.transport.RemoteTransportException: [Gemini][inet[/192.168.10.41:9300]][search/phase/fetch/id]
Caused by: org.elasticsearch.search.SearchContextMissingException: No search context found for id [354851], timed out
        at org.elasticsearch.search.SearchService.findContext(SearchService.java:278)
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:261)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchFetchByIdTransportHandler.messageReceived(SearchServiceTransportAction.java:424)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchFetchByIdTransportHandler.messageReceived(SearchServiceTransportAction.java:415)
        at org.elasticsearch.transport.netty.MessageChannelHandler$3.run(MessageChannelHandler.java:155)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
[12:27:53,074][DEBUG][action.search.type       ] [Longshot] Failed to execute fetch phase
org.elasticsearch.transport.RemoteTransportException: [Gemini][inet[/192.168.10.41:9300]][search/phase/fetch/id]
Caused by: org.elasticsearch.search.SearchContextMissingException: No search context found for id [355847], timed out
        at org.elasticsearch.search.SearchService.findContext(SearchService.java:278)
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:261)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchFetchByIdTransportHandler.messageReceived(SearchServiceTransportAction.java:424)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchFetchByIdTransportHandler.messageReceived(SearchServiceTransportAction.java:415)
        at org.elasticsearch.transport.netty.MessageChannelHandler$3.run(MessageChannelHandler.java:155)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
[12:27:53,148][DEBUG][action.search.type       ] [Longshot] Failed to execute fetch phase
org.elasticsearch.transport.RemoteTransportException: [Magnum, Moses][inet[/192.168.10.33:9300]][search/phase/fetch/id]
Caused by: org.elasticsearch.search.SearchContextMissingException: No search context found for id [342239], timed out
        at org.elasticsearch.search.SearchService.findContext(SearchService.java:277)
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:260)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchFetchByIdTransportHandler.messageReceived(SearchServiceTransportAction.java:424)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchFetchByIdTransportHandler.messageReceived(SearchServiceTransportAction.java:415)
        at org.elasticsearch.transport.netty.MessageChannelHandler$3.run(MessageChannelHandler.java:155)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

@clintongormley
Copy link
Author

btw, these nodes were built from master on 2010-04-08, so not sure if you've made more changes that should have fixed this already

@clintongormley
Copy link
Author

Seems to work in recent releases

rmuir pushed a commit to rmuir/elasticsearch that referenced this issue Nov 8, 2015
Related to elastic#10157

BytesStreamInput does not support anymore `BytesStreamInput(byte[], boolean)`

Closes elastic#120.
C-Bish pushed a commit to C-Bish/elasticsearch that referenced this issue Mar 23, 2017
ClaudioMFreitas pushed a commit to ClaudioMFreitas/elasticsearch-1 that referenced this issue Nov 12, 2019
Fix for template fileglob not being defined + initial shield files
mindw pushed a commit to mindw/elasticsearch that referenced this issue Sep 5, 2022
…equest elastic#120)

MPC-1943: Create config/local.yml only if it doesn't exist

Approved-by: Rich Nessel
cbuescher pushed a commit to cbuescher/elasticsearch that referenced this issue Oct 2, 2023
With this commit we set the environment variable JAVA13_HOME so Rally
can pick it up in order to compile and run with JDK 13.
emilykmarx pushed a commit to emilykmarx/elasticsearch that referenced this issue Dec 26, 2023
some log messages were referencing @host field, even when
the field was nil and not applicable (when protocol was node).

Closes elastic#120.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant