Error when deserializing aggregations on es 1.4

[jensbengtsson]

Hi!

We just upgraded from es 1.3.4 to 1.4 but when our MVC app now tries to execute a query with aggregations we get the following error:

[JsonSerializationException: Unexpected token when deserializing object: StartObject. Path 'aggregations.Deleted.buckets[0]', line 1, position 14629.]
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateDictionary(IDictionary dictionary, JsonReader reader, JsonDictionaryContract contract, JsonProperty containerProperty, String id) +2082
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) +2798
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) +189
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) +597
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) +1896
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) +1212
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) +189
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) +748
Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) +354
Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType) +47
Nest.NestSerializer.DeserializeUsingSettings(Stream stream, JsonSerializerSettings settings) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:109
Nest.NestSerializer.DeserializeInternal(Stream stream, JsonConverter converter) in c:\code\elasticsearch-net\src\Nest\ExposedInternals\NestSerializer.cs:100
Nest.ElasticClient.FieldsSearchDeserializer(IElasticsearchResponse response, Stream stream, ISearchRequest d) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:118
Nest.<>c__DisplayClass2a82.<CreateSearchDeserializer>b__2a7(IElasticsearchResponse r, Stream s) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:130 Elasticsearch.Net.Connection.Transport._deserializeToResponse(ElasticsearchResponse1 typedResponse, Stream responseStream, ITransportRequestState requestState) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\Transport.cs:810
Elasticsearch.Net.Connection.Transport.StreamToTypedResponse(ElasticsearchResponse1 streamResponse, ITransportRequestState requestState) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\Transport.cs:717 Elasticsearch.Net.Connection.Transport.DoRequest(TransportRequestState1 requestState) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\Transport.cs:376
Elasticsearch.Net.Connection.Transport.DoRequest(String method, String path, Object data, IRequestParameters requestParameters) in c:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\Transport.cs:309
Elasticsearch.Net.ElasticsearchClient.DoRequest(String method, String path, Object data, IRequestParameters requestParameters) in c:\code\elasticsearch-net\src\Elasticsearch.Net\ElasticsearchClient.cs:65
Elasticsearch.Net.ElasticsearchClient.Search(String index, String type, Object body, Func2 requestParameters) in c:\code\elasticsearch-net\src\Elasticsearch.Net\ElasticsearchClient.Generated.cs:32683 Nest.RawDispatch.SearchDispatch(ElasticsearchPathInfo1 pathInfo, Object body) in c:\code\elasticsearch-net\src\Nest\RawDispatch.generated.cs:3412
Nest.ElasticClient.Search(ISearchRequest request) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:52
Nest.ElasticClient.Search(ISearchRequest request) in c:\code\elasticsearch-net\src\Nest\ElasticClient-Search.cs:41
ElasticsearchIntegration.SearchHelpers.SearchHelper.Search(SearchRequest searchrequest, IEnumerable1 types) in c:\code\elasticsearch-gui\ElasticsearchIntegration\SearchHelpers\SearchHelper.cs:45 ElasticsearchGUI.Controllers.MultipleController.Search(String q, Int32 p, String u) in c:\code\elasticsearch-gui\ElasticsearchGUI\Controllers\MultipleController.cs:22 lambda_method(Closure , ControllerBase , Object[] ) +301 System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) +59 System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary2 parameters) +435
System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary2 parameters) +60 System.Web.Mvc.Async.ActionInvocation.InvokeSynchronousActionMethod() +76 System.Web.Mvc.Async.AsyncControllerActionInvoker.<BeginInvokeSynchronousActionMethod>b__39(IAsyncResult asyncResult, ActionInvocation innerInvokeState) +36 System.Web.Mvc.Async.WrappedAsyncResult2.CallEndDelegate(IAsyncResult asyncResult) +73
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +136 System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +102 System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult) +49 System.Web.Mvc.Async.AsyncInvocationWithFilters.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3d() +117 System.Web.Mvc.Async.<>c__DisplayClass46.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3f() +323 System.Web.Mvc.Async.<>c__DisplayClass33.<BeginInvokeActionMethodWithFilters>b__32(IAsyncResult asyncResult) +44 System.Web.Mvc.Async.WrappedAsyncResult1.CallEndDelegate(IAsyncResult asyncResult) +47
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +136 System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +102 System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult) +50 System.Web.Mvc.Async.<>c__DisplayClass2b.<BeginInvokeAction>b__1c() +72 System.Web.Mvc.Async.<>c__DisplayClass21.<BeginInvokeAction>b__1e(IAsyncResult asyncResult) +185 System.Web.Mvc.Async.WrappedAsyncResult1.CallEndDelegate(IAsyncResult asyncResult) +42
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +133 System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +56 System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult) +40 System.Web.Mvc.Controller.<BeginExecuteCore>b__1d(IAsyncResult asyncResult, ExecuteCoreState innerState) +34 System.Web.Mvc.Async.WrappedAsyncVoid1.CallEndDelegate(IAsyncResult asyncResult) +70
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +139 System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +59 System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +40 System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult) +44 System.Web.Mvc.Controller.<BeginExecute>b__15(IAsyncResult asyncResult, Controller controller) +39 System.Web.Mvc.Async.WrappedAsyncVoid1.CallEndDelegate(IAsyncResult asyncResult) +62
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +139 System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +59 System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +40 System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult) +39 System.Web.Mvc.Controller.System.Web.Mvc.Async.IAsyncController.EndExecute(IAsyncResult asyncResult) +39 System.Web.Mvc.MvcHandler.<BeginProcessRequest>b__5(IAsyncResult asyncResult, ProcessRequestState innerState) +39 System.Web.Mvc.Async.WrappedAsyncVoid1.CallEndDelegate(IAsyncResult asyncResult) +70
System.Web.Mvc.Async.WrappedAsyncResultBase`1.End() +139
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +59
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +40
System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult) +40
System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result) +38
System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +9657896
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +155

The base method for the request looks like this

        public ISearchResponse<object> Search(SearchRequest searchrequest, IEnumerable<TypeNameMarker> types)
        {
            LoggingHelper.LogSearchRequest(searchrequest);

            var filteredQuery = new QueryContainer(new FilteredQuery
            {
                Filter = GetQueryFilter(searchrequest),
                Query = QueryGenerator.GenerateQuery(searchrequest.Query)
            });
            var nestrequest = new SearchRequest<object>
            {
                Types = types,
                From = (searchrequest.Page - 1) * PageSize,
                Size = PageSize,
                Query = filteredQuery,
                Aggregations = FacetGenerator.GetAggregationRequest()
            };
            return _client.Search<object>(nestrequest);
        }

[gmarz]

Hi @jensbengtsson, this is a known compatibility issue between NEST and ES 1.4. We've already made a fix (#1041) and plan on releasing a new version shortly. Stay tuned.