diff --git a/docs/client-concepts/high-level/serialization/custom-serialization.asciidoc b/docs/client-concepts/high-level/serialization/custom-serialization.asciidoc index 59ca6695413..6c2502ef5a8 100644 --- a/docs/client-concepts/high-level/serialization/custom-serialization.asciidoc +++ b/docs/client-concepts/high-level/serialization/custom-serialization.asciidoc @@ -15,32 +15,38 @@ please modify the original csharp file found at the link and submit the PR with [[custom-serialization]] === Custom Serialization -NEST 6.x ships with a _shaded_ Json.NET dependency, meaning that all of Json.NET's types are -internalized and IL merged into the NEST assembly, and their namespace has been changed -from `Newtonsoft.Json` to `Nest.Json`. - -Why would we do this, you may ask? Well, NEST has always isolated its dependency on Json.NET as best as it could, -but this meant that we had to mandate how certain things were in the client. For instance, -NEST heavily relied on the fact that the `IContractResolver` used by the configured serializer was -an instance of `ElasticContractResolver`, so if you wanted to deserialize your `_source` or `_fields` -using your own resolver, you were out of luck. In addition, continued improvements to NEST's serialization pipeline -was stymied by this dependency and as client authors, we wanted to unhinder ourselves from this in order to explore the myriad -of exciting developments happening in the .NET ecosystem around performance with the likes of `Span`, -`ArrayPool` and a more compact UTF-8 string representation. - -So what did we do in 6.x and how does it affect you? - -The `NEST` nuget package from 6.0.0 onwards will use the internal Json.NET serializer and will in effect, behave the same -as it did in previous releases. If you previously relied on custom Json.NET serialization and configuration with custom -`JsonSerializerSettings` and `JsonConverter` types for example, things have changed a bit. The following section explains -how to continue working with these with NEST 6.x. +After internalizing the serialization routines, and IL-merging the Newtonsoft.Json package in 6.x, we are pleased to +announce that the next stage of serialization improvements have been completed in 7.0. + +Both SimpleJson and Newtonsoft.Json have been completely removed and replaced with an implementation of Utf8Json, a fast +serializer that works directly with UTF-8 binary. + +With the move to Utf8Json, we have removed some features that were available in the previous JSON libraries that have +proven too onerous to carry forward at this stage. + +* JSON in the request is never indented, even if SerializationFormatting.Indented is specified. The serialization +routines generated by Utf8Json never generate an IJsonFormatter that will indent JSON, for performance reasons. +We are considering options for exposing indented JSON for development and debugging purposes. + +* NEST types cannot be extended by inheritance. With NEST 6.x, additional properties can be included for a type by deriving from +that type and annotating these new properties. With the current implementation of serialization with Utf8Json, this approach will not work. + +* Serializer uses Reflection.Emit. Utf8Json uses Reflection.Emit to generate efficient formatters for serializing types that it sees. +Reflection.Emit is not supported on all platforms, for example, UWP, Xamarin.iOS, and Xamarin.Android. + +* Elasticsearch.Net.DynamicResponse deserializes JSON arrays to List. SimpleJson deserialized JSON arrays to object[], +but Utf8Json deserializes them to List. This change is preferred for allocation and performance reasons. + +* Utf8Json is much stricter when deserializing JSON object field names to C# POCO properties. With the internal Json.NET serializer +in 6.x, JSON object field names would attempt to be matched with C# POCO property names first by an exact match, falling back to a +case insensitive match. With Utf8Json in 7.x however, JSON object field names must match exactly the name configured for the +C# POCO property name. [float] ==== Injecting a new serializer -Starting with NEST 6.x you can inject a serializer that is isolated to only be called -for the (de)serialization of `_source`, `_fields`, or wherever a user provided value is expected -to be written and returned. +You can inject a serializer that is isolated to only be called for the (de)serialization of `_source`, `_fields`, or +wherever a user provided value is expected to be written and returned. Within NEST, we refer to this serializer as the `SourceSerializer`. @@ -109,8 +115,11 @@ back to NEST's built-in serializer. We ship a separate {nuget}/NEST.JsonNetSerializer[NEST.JsonNetSerializer] package that helps in composing a custom `SourceSerializer` using `Json.NET`, that is smart enough to delegate the serialization of known NEST types back to the built-in -`RequestResponseSerializer`. This package is also useful if you want to control how your documents and values are stored -and retrieved from Elasticsearch using `Json.NET`, without interfering with the way NEST uses `Json.NET` internally. +`RequestResponseSerializer`. This package is also useful if + +. You want to control how your documents and values are stored and retrieved from Elasticsearch using `Json.NET` + +. You want to use `Newtonsoft.Json.Linq` types such as `JObject` within your documents The easiest way to hook this custom source serializer up is as follows diff --git a/docs/query-dsl.asciidoc b/docs/query-dsl.asciidoc index be9da05bd0e..a9a75ac229d 100644 --- a/docs/query-dsl.asciidoc +++ b/docs/query-dsl.asciidoc @@ -49,13 +49,13 @@ NEST exposes all of the full text queries available in Elasticsearch * <> -* <> +* <> -* <> +* <> * <> -* <> +* <> * <> @@ -71,13 +71,13 @@ include::query-dsl/full-text/common-terms/common-terms-usage.asciidoc[] include::query-dsl/full-text/intervals/intervals-usage.asciidoc[] -include::query-dsl/full-text/match-bool-prefix/match-bool-prefix-usage.asciidoc[] +include::query-dsl/full-text/match/match-usage.asciidoc[] -include::query-dsl/full-text/match-phrase-prefix/match-phrase-prefix-usage.asciidoc[] +include::query-dsl/full-text/match-bool-prefix/match-bool-prefix-usage.asciidoc[] include::query-dsl/full-text/match-phrase/match-phrase-usage.asciidoc[] -include::query-dsl/full-text/match/match-usage.asciidoc[] +include::query-dsl/full-text/match-phrase-prefix/match-phrase-prefix-usage.asciidoc[] include::query-dsl/full-text/multi-match/multi-match-usage.asciidoc[] @@ -122,14 +122,14 @@ NEST exposes all of the term queries available in Elasticsearch * <> -* <> - * <> * <> * <> +* <> + * <> See the Elasticsearch documentation on {ref_current}/term-level-queries.html[Term level queries] for more details. @@ -160,14 +160,14 @@ include::query-dsl/term-level/regexp/regexp-query-usage.asciidoc[] include::query-dsl/term-level/term/term-query-usage.asciidoc[] -include::query-dsl/term-level/terms-set/terms-set-query-usage.asciidoc[] - include::query-dsl/term-level/terms/terms-list-query-usage.asciidoc[] include::query-dsl/term-level/terms/terms-lookup-query-usage.asciidoc[] include::query-dsl/term-level/terms/terms-query-usage.asciidoc[] +include::query-dsl/term-level/terms-set/terms-set-query-usage.asciidoc[] + include::query-dsl/term-level/wildcard/wildcard-query-usage.asciidoc[] [[compound-queries]] @@ -283,10 +283,10 @@ Specialized types of queries that do not fit into other groups * <> -* <> - * <> +* <> + * <> See the Elasticsearch documentation on {ref_current}/specialized-queries.html[Specialized queries] for more details. @@ -305,10 +305,10 @@ include::query-dsl/specialized/pinned/pinned-query-usage.asciidoc[] include::query-dsl/specialized/rank-feature/rank-feature-query-usage.asciidoc[] -include::query-dsl/specialized/script-score/script-score-query-usage.asciidoc[] - include::query-dsl/specialized/script/script-query-usage.asciidoc[] +include::query-dsl/specialized/script-score/script-score-query-usage.asciidoc[] + include::query-dsl/specialized/shape/shape-query-usage.asciidoc[] [[span-queries]] diff --git a/tests/Tests/ClientConcepts/HighLevel/Serialization/CustomSerialization.doc.cs b/tests/Tests/ClientConcepts/HighLevel/Serialization/CustomSerialization.doc.cs index 4cdad9b5068..b45fe630ac7 100644 --- a/tests/Tests/ClientConcepts/HighLevel/Serialization/CustomSerialization.doc.cs +++ b/tests/Tests/ClientConcepts/HighLevel/Serialization/CustomSerialization.doc.cs @@ -21,34 +21,40 @@ namespace Tests.ClientConcepts.HighLevel.Serialization /**[[custom-serialization]] * === Custom Serialization * - * NEST 6.x ships with a _shaded_ Json.NET dependency, meaning that all of Json.NET's types are - * internalized and IL merged into the NEST assembly, and their namespace has been changed - * from `Newtonsoft.Json` to `Nest.Json`. + * After internalizing the serialization routines, and IL-merging the Newtonsoft.Json package in 6.x, we are pleased to + * announce that the next stage of serialization improvements have been completed in 7.0. * - * Why would we do this, you may ask? Well, NEST has always isolated its dependency on Json.NET as best as it could, - * but this meant that we had to mandate how certain things were in the client. For instance, - * NEST heavily relied on the fact that the `IContractResolver` used by the configured serializer was - * an instance of `ElasticContractResolver`, so if you wanted to deserialize your `_source` or `_fields` - * using your own resolver, you were out of luck. In addition, continued improvements to NEST's serialization pipeline - * was stymied by this dependency and as client authors, we wanted to unhinder ourselves from this in order to explore the myriad - * of exciting developments happening in the .NET ecosystem around performance with the likes of `Span`, - * `ArrayPool` and a more compact UTF-8 string representation. + * Both SimpleJson and Newtonsoft.Json have been completely removed and replaced with an implementation of Utf8Json, a fast + * serializer that works directly with UTF-8 binary. * - * So what did we do in 6.x and how does it affect you? + * With the move to Utf8Json, we have removed some features that were available in the previous JSON libraries that have + * proven too onerous to carry forward at this stage. * - * The `NEST` nuget package from 6.0.0 onwards will use the internal Json.NET serializer and will in effect, behave the same - * as it did in previous releases. If you previously relied on custom Json.NET serialization and configuration with custom - * `JsonSerializerSettings` and `JsonConverter` types for example, things have changed a bit. The following section explains - * how to continue working with these with NEST 6.x. + * - JSON in the request is never indented, even if SerializationFormatting.Indented is specified. The serialization + * routines generated by Utf8Json never generate an IJsonFormatter that will indent JSON, for performance reasons. + * We are considering options for exposing indented JSON for development and debugging purposes. + * + * - NEST types cannot be extended by inheritance. With NEST 6.x, additional properties can be included for a type by deriving from + * that type and annotating these new properties. With the current implementation of serialization with Utf8Json, this approach will not work. + * + * - Serializer uses Reflection.Emit. Utf8Json uses Reflection.Emit to generate efficient formatters for serializing types that it sees. + * Reflection.Emit is not supported on all platforms, for example, UWP, Xamarin.iOS, and Xamarin.Android. + * + * - Elasticsearch.Net.DynamicResponse deserializes JSON arrays to List. SimpleJson deserialized JSON arrays to object[], + * but Utf8Json deserializes them to List. This change is preferred for allocation and performance reasons. + * + * - Utf8Json is much stricter when deserializing JSON object field names to C# POCO properties. With the internal Json.NET serializer + * in 6.x, JSON object field names would attempt to be matched with C# POCO property names first by an exact match, falling back to a + * case insensitive match. With Utf8Json in 7.x however, JSON object field names must match exactly the name configured for the + * C# POCO property name. */ public class GettingStarted { /**[float] * ==== Injecting a new serializer * - * Starting with NEST 6.x you can inject a serializer that is isolated to only be called - * for the (de)serialization of `_source`, `_fields`, or wherever a user provided value is expected - * to be written and returned. + * You can inject a serializer that is isolated to only be called for the (de)serialization of `_source`, `_fields`, or + * wherever a user provided value is expected to be written and returned. * * Within NEST, we refer to this serializer as the `SourceSerializer`. * @@ -114,8 +120,10 @@ public class MyPercolationDocument * * We ship a separate {nuget}/NEST.JsonNetSerializer[NEST.JsonNetSerializer] package that helps in composing a custom `SourceSerializer` * using `Json.NET`, that is smart enough to delegate the serialization of known NEST types back to the built-in - * `RequestResponseSerializer`. This package is also useful if you want to control how your documents and values are stored - * and retrieved from Elasticsearch using `Json.NET`, without interfering with the way NEST uses `Json.NET` internally. + * `RequestResponseSerializer`. This package is also useful if + * + * . You want to control how your documents and values are stored and retrieved from Elasticsearch using `Json.NET` + * . You want to use `Newtonsoft.Json.Linq` types such as `JObject` within your documents * * The easiest way to hook this custom source serializer up is as follows */