Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IndicesCreatePost not working with JSON string #2656

Closed
GaryTurpin opened this issue Mar 7, 2017 · 1 comment
Closed

IndicesCreatePost not working with JSON string #2656

GaryTurpin opened this issue Mar 7, 2017 · 1 comment

Comments

@GaryTurpin
Copy link

NEST/Elasticsearch.Net version:
2.5.2

Elasticsearch version:
2.4.4

Description of the problem including expected versus actual behavior:
I am upgrading from ES 1.X to ES 2.X and I am having a problem creating an index with mappings/settings from JSON.

In ES 1.X Nest, I would do the following and everything would work perfectly.

public bool CreateIndex(string json)
{
  var client = new Nest.ElasticClient(new ConnectionSettings(this.ESIndex.ClusterUri));
  var resp = client.Raw.IndicesCreatePost(this.ESIndex.SyncIndex, json);
  return resp.Success;
}

In ES 2.X Nest, I do the following and the mappings/settings never get added to the blank index.

public bool CreateIndex<T>(string json) where T : class
{
  var client = new Nest.ElasticClient(this.ESIndex.ClusterUri);
  var body = new PostData<string>(json);
  var resp = client.LowLevel.IndicesCreatePost<T>(this.ESIndex.SyncIndex, body);
  return resp.Success;
}

If I look at fiddler, it looks like the only thing in the body of the request is
{"type":1}

But, I expect to have the JSON that have for my mappings/settings. Any idea what I am doing incorrectly so I can make this work?

@Mpdreamz
Copy link
Member

This is a little confusing but its because of PostData<string>

  var body = new PostData<object>(json);
var resp = client.LowLevel.IndicesCreatePost<string>("someindex5", body);

Will work or just sending json directly as the second argument.

It happens because that PostData<string> gets passed internally to argument taking PostData<object> and the implicit operator for T kicks in wrapping it.

Mpdreamz added a commit that referenced this issue Nov 23, 2017
`PostData<object>` was was introduced as a union'esque type that
`byte[]/string/object/IEnumerable<string>/<IEnumerable<object>` could
all implicitly converter to automagically making the lowel client easy
to use without exploding the already massive method list on
ILowLevelClient.

This abstraction meant that we always boxed for NEST types to object.

In preparation for all the serialization/perf work we want to do in 6.x
it would be great if we can take `T` that we we want to serialize all
the way to the `IElasticsearchSerializer` which we can do with this PR
now.

We sacrifice on the implicit operators in the low level client (for the
most part: byte[] and string still do). TBF most of the time I've seen
people use the low level client they would instantiate
`PostData<object>` directly already. Or worse see: #2656. We weren't
really helping anyone here.

`PostData<object>` now has a base class `PostData` that we use
throughout. There is also `SerializableData<T>` subclass that we now use
for anything that needs serializing.

This dispatcher is now aware of what generics an `IRequest`
implementation has. Because of this our hack to make `Index(document)`
and `CreateIndex(document)` work when document == IIndexRequest or not
no longer works. Rather than fighting C# resolution of overloads (and
its lack of not constraints :() the methods are now called
`IndexDocument()` and `CreateDocument()`.
Mpdreamz added a commit that referenced this issue Nov 25, 2017
`PostData<object>` was was introduced as a union'esque type that
`byte[]/string/object/IEnumerable<string>/<IEnumerable<object>` could
all implicitly converter to automagically making the lowel client easy
to use without exploding the already massive method list on
ILowLevelClient.

This abstraction meant that we always boxed for NEST types to object.

In preparation for all the serialization/perf work we want to do in 6.x
it would be great if we can take `T` that we we want to serialize all
the way to the `IElasticsearchSerializer` which we can do with this PR
now.

We sacrifice on the implicit operators in the low level client (for the
most part: byte[] and string still do). TBF most of the time I've seen
people use the low level client they would instantiate
`PostData<object>` directly already. Or worse see: #2656. We weren't
really helping anyone here.

`PostData<object>` now has a base class `PostData` that we use
throughout. There is also `SerializableData<T>` subclass that we now use
for anything that needs serializing.

This dispatcher is now aware of what generics an `IRequest`
implementation has. Because of this our hack to make `Index(document)`
and `CreateIndex(document)` work when document == IIndexRequest or not
no longer works. Rather than fighting C# resolution of overloads (and
its lack of not constraints :() the methods are now called
`IndexDocument()` and `CreateDocument()`.
Mpdreamz added a commit that referenced this issue Nov 28, 2017
`PostData<object>` was was introduced as a union'esque type that
`byte[]/string/object/IEnumerable<string>/<IEnumerable<object>` could
all implicitly converter to automagically making the lowel client easy
to use without exploding the already massive method list on
ILowLevelClient.

This abstraction meant that we always boxed for NEST types to object.

In preparation for all the serialization/perf work we want to do in 6.x
it would be great if we can take `T` that we we want to serialize all
the way to the `IElasticsearchSerializer` which we can do with this PR
now.

We sacrifice on the implicit operators in the low level client (for the
most part: byte[] and string still do). TBF most of the time I've seen
people use the low level client they would instantiate
`PostData<object>` directly already. Or worse see: #2656. We weren't
really helping anyone here.

`PostData<object>` now has a base class `PostData` that we use
throughout. There is also `SerializableData<T>` subclass that we now use
for anything that needs serializing.

This dispatcher is now aware of what generics an `IRequest`
implementation has. Because of this our hack to make `Index(document)`
and `CreateIndex(document)` work when document == IIndexRequest or not
no longer works. Rather than fighting C# resolution of overloads (and
its lack of not constraints :() the methods are now called
`IndexDocument()` and `CreateDocument()`.
Mpdreamz added a commit that referenced this issue Nov 28, 2017
* fixed last remaining failing unit tests

* Split serializer into request/response AND source serializer. Scrub json.net related classes from public API

* Writing test for source serialization, making sure it respects the custom user serializer

* formatting

* validated more API's are using source serializer correctly for writing

* we now randomly seed when running on the commandline and we show replay command at the end of the run always

* test framework can now run with or without source serializer this is random on the commandline and pinnable on both commandline and vanilla test runners. unit tests succeeds for both with and without source serializer, onwards to integration tests

* Don't bleed Union<T,T> into JoinField, since its expected to be on folks documents it does not play well with SourceSerializer's, had it working in a previous commit but this limits the ammount of places that need patching.

* clearly separate Source and RequestResponse Serializers by name

* We can now offer a custom json net serializer that will fallback to our serializer for known NEST types in _source e.g JoinField, or QueryContainer when indexing percolation queries

* RevertBackToBuiltInConverter now works as expected

* fixed failing unit test due to new property on Project

* Several improvements

LazyDocument is now source serializer aware.

Explain() Search() MultiGet() SimulatePipeline() Get() now all have
integration tests for custom source deserialization

* Suggest() is now source serializer aware

* fixed all unit tests, reverted changes to source converter and documentjson converter since the proposed optimization still used the internal writer and thus the internal serializer settings

* ApiTestBase now uses source convverter by default as well, more like this query document now uses source converter

* TopHits aggregations now support custom source serializer

* Scripted metric now uses source serializer

* FieldsValues is now also source serializer aware

* update unit tests now that we have a non nullable int in Project

* update paket targets file

* ILRepack internalizes JSON.NET, Mono.Cecil rewrites its namespace

* updated Mono.Cecil to the latest beta, this one does not lock the assembly file and explicitly allows overwriting existing files

* Introduced new nuget package NEST.JsonNetSerializer which folks can reference should they need to customize the way their _source and field values are (de)serialized using Json.NET

* added new documentation around customizing serialization'

* attempt 1

* attempt 2

* All unit tests now pass when testing against local nuget package for NEST.

Simplified Serialization testbase

Return of Gimme.Lock started seeing Bogus blocking on Populate again.

* canary builds now test the nuget package at the end of its build run.

* make appveyor run the canary build and include appveyor.yml as solution item

* ILRepack and mono do not mix, prevent calling canary/release on mono systems for now

* move to async token read in json.net 10.0.1, idea being that its less bad to buffer in a JObject than to read the stream in a blocking fashion. cc @codebrain @russcam

* Add SerializeAsync to interface

* Prevent boxing on PostData<object>

`PostData<object>` was was introduced as a union'esque type that
`byte[]/string/object/IEnumerable<string>/<IEnumerable<object>` could
all implicitly converter to automagically making the lowel client easy
to use without exploding the already massive method list on
ILowLevelClient.

This abstraction meant that we always boxed for NEST types to object.

In preparation for all the serialization/perf work we want to do in 6.x
it would be great if we can take `T` that we we want to serialize all
the way to the `IElasticsearchSerializer` which we can do with this PR
now.

We sacrifice on the implicit operators in the low level client (for the
most part: byte[] and string still do). TBF most of the time I've seen
people use the low level client they would instantiate
`PostData<object>` directly already. Or worse see: #2656. We weren't
really helping anyone here.

`PostData<object>` now has a base class `PostData` that we use
throughout. There is also `SerializableData<T>` subclass that we now use
for anything that needs serializing.

This dispatcher is now aware of what generics an `IRequest`
implementation has. Because of this our hack to make `Index(document)`
and `CreateIndex(document)` work when document == IIndexRequest or not
no longer works. Rather than fighting C# resolution of overloads (and
its lack of not constraints :() the methods are now called
`IndexDocument()` and `CreateDocument()`.

* update travis to dotnet 2.0.3

* walkthrough review comments from @codebrain and @russcam, updated travis.yml to support netcore 2 building netcoreapp1.x on linux

* suspicion skipping first time experience on the dotnet cli breaks setting up nuget caches properly

* put back DOTNET_SKIP_FIRST_EXPERIENCE, as it did not solve travis build

* build.sh was alway doing an install because it was checking for  add an empty paket file in src to prevent paket from adding targets to csproj file on paket install

* make sure dotnet SDK 1.0.4 is available

* 1.0.5 not 1.0.4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants