Prevent indexing from changing the mapping

[mausch]

I'm seeing some data in my index that belongs to some property in the object I'm indexing but I didn't define in my mapping. Here's a small repro where I see this unexpected behavior:

using System;
using System.Collections.Generic;
using Nest;
using Nest.DSL.Visitor;

namespace NESTMapping
{
    public class UnwantedObject
    {
        public int DoNotWant { get; set; }
    }

    public class SomeObject
    {
        public UnwantedObject Unwanted { get; set; }

        public string Name;
    }

    class MappingVisitor: IMappingVisitor
    {
        private readonly int id;

        public MappingVisitor(int id)
        {
            this.id = id;
        }

        public int Depth {
            get { throw new NotImplementedException(); }
            set { throw new NotImplementedException(); }
        }

        public void Visit(AttachmentMapping mapping) {
            throw new NotImplementedException();
        }

        public void Visit(GeoShapeMapping mapping)
        {
            throw new NotImplementedException();
        }

        public void Visit(GeoPointMapping mapping)
        {
            throw new NotImplementedException();
        }

        public void Visit(IPMapping mapping)
        {
            throw new NotImplementedException();
        }

        public void Visit(MultiFieldMapping mapping)
        {
            throw new NotImplementedException();
        }

        public void Visit(NestedObjectMapping mapping)
        {
            throw new NotImplementedException();
        }

        public void Visit(ObjectMapping mapping) {
            throw new Exception("Object mapping detected " + id);
        }

        public void Visit(BinaryMapping mapping){
            throw new NotImplementedException();
        }

        public void Visit(BooleanMapping mapping) {
            throw new NotImplementedException();
        }

        public void Visit(DateMapping mapping) {
            throw new NotImplementedException();
        }

        public void Visit(NumberMapping mapping){
            throw new NotImplementedException();
        }

        public void Visit(StringMapping mapping) {}

        public void Visit(RootObjectMapping mapping){}
    }

    class Program
    {
        static void Main(string[] args)
        {
            const string indexName = "someindex";
            var connectionSettings = new ConnectionSettings(uri: null, defaultIndex: indexName);
            var client = new ElasticClient(connectionSettings);
            client.DeleteIndex(x => x.AllIndices());
            client.CreateIndex(indexName);
            client.Map<SomeObject>(m => m.Type(indexName).Properties(p => p.String(d => d.Name(x => x.Name))));

            var mapping = client.GetMapping<SomeObject>(x => x.Type(indexName));
            mapping.Accept(new MappingVisitor(1));

            client.Index(new SomeObject
            {
                Name = "John Doe",
                Unwanted = new UnwantedObject { DoNotWant = 1000 },
            }, x => x.Type(indexName));

            mapping = client.GetMapping<SomeObject>(x => x.Type(indexName));
            mapping.Accept(new MappingVisitor(2));

        }
    }
}

This throws Object mapping detected 2, i.e. there is no object mapping when I define the mapping, but the mapping is then modified when I index an object.

I can see this too when requesting the mapping directly from Elasticsearch, e.g. after defining the mapping but before indexing, http://localhost:9200/someindex/_mapping shows this:

{"someindex":{"mappings":{"someindex":{"properties":{"name":{"type":"string"}}}}}}

And after indexing http://localhost:9200/someindex/_mapping is:

{"someindex":{"mappings":{"someindex":{"properties":{"name":{"type":"string"},"unwanted":{"properties":{"doNotWant":{"type":"long"}}}}}}}}

Is there any way to tell NEST/Elasticsearch to never ever modify mappings, i.e. only send to the server what has been defined in the mapping?

[Mpdreamz]

This is an Elasticsearch behaviour and not NEST.

The initial mapping does not include Unwanted as you've seen but the Index() introduces the field and elasticsearch's default dynamic mapping kicks in:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-dynamic-mapping.html

This is the schemaless bit in elasticsearch.

You can force the mapping to be strict though (as outlined in the linked documentation):

client.Map<SomeObject>(m => m
    .Dynamic(DynamicMappingOption.Strict)
    ...

which should cause the Index() to fail

[Mpdreamz]

To never have it send though you can apply a [JsonIgnore] or [ElasticsearchProperty(OptOut=true)] attributes, that you can't currently set in code as you can names: #1062, I could see If we can amend that PR.

[mausch]

Thanks. If I understand correctly the problem is that you'd still have to map both what you want to index and what you don't want to index?

[Mpdreamz]

Yes if you use strict mapping on a type Z with a property X, where X is not mapped in elasticsearch you need to tell NEST to not send X, or use class Y to more strictly resemble the type as it exists in elasticsearch and use automapper/manual mapping prior to sending documents of type Z.

[mausch]

Ok, thanks for confirming.

[niemyjski]

Can we get a visitor sample in future docs :)

[gmarz]

@niemyjski we have some docs for 2.x here https://www.elastic.co/guide/en/elasticsearch/client/net-api/2.x/auto-map.html#applying-conventions-through-the-visitor-pattern

For 1.x, check out the unit tests https://github.com/elastic/elasticsearch-net/blob/1.x/src/Tests/Nest.Tests.Integration/Mapping/MappingVisitorTests.cs