Not_analyzed has no effect

[TowardsDeath]

I'm trying to tell ElasticSearch to stop analyzing a certain property, to prevent values with a dash from being split. I tried the options I could find, but none of them work. Have I stumbled upon a bug in NEST or am I doing something wrong? Let me tell you what I've tried so far.

Mapping with ElasticProperty

The NEST docs tell me I can use the ElasticProperty attribute with a FieldIndexOption.

[ElasticType( Name = "product" )]
public class ElasticSearchProduct
{
    [ElasticProperty( OmitNorms = true, Index = FieldIndexOption.not_analyzed )]
    public string Format { get; set; }
}

When the Index method has been called, I can see the product has been correctly indexed and the mapping is created in ElasticSearch. However, when I perform a faceted search, the value is still split. (The facet value "E-book" becomes two: "E" and "Book".)

Manually call the mapper method

I found issue #19 that told me to persist the mapping, so I did:

this.ElasticSearch.Map<ElasticSearchProduct>( "products" );

It creates a mapping, but the value is still split.

Explicit type mapping

Then I tried to perform an explicit type mapping, as mentioned in issue #19.

var notAnalyzedProperty = new TypeMappingProperty
{
    Type = "string",
    Index = "not_analyzed"
};
var newMapping = new TypeMapping( "product" );
newMapping.Properties.Add( "format", notAnalyzedProperty );
this.ElasticSearch.Map( newMapping, index );

That doesn't prevent the value from being split either.

I'm running this with:

• the latest NEST package from NuGet: 0.9.1.0
• the latest ElasticSearch 0.19.7, with default configuration.
• search queries that use an index alias and a routing value, a construct that I've found on the ElasticSearch Google group. That should have nothing to do with this issue, but perhaps it's good to know to determine the cause.

If you need me further to debug this or provide any more information, just let me know.

[Mpdreamz]

I've updated the tests but all seems well:
7e39b76#src-nest-tests-mapping-maptests-cs-P6

Can you gist your GetMapping() response and your search request + response ?

if you also specify `SearchAnalyzer="keyword", IndexAnalyzer="keyword" are you getting better results ? Note that this should not be necessary though.

[Mpdreamz]

Hey Robert,

I've committed even more explicit tests:
https://github.com/Mpdreamz/NEST/blob/811311854d49ad15fde53a4969741120a57eea2b/src/Nest.Tests/Mapping/NotAnalyzedTest.cs

Which seem to indicate everything is working like it should. Can you run those test and confirm they are working for you too?

Can you gist your entity class, index code, search code ? Privately would be fine too just sent me an email at my github account name @ gmail

Perhaps we can even schedule a teamviewer (or something of the sort) session to debug this?

[TowardsDeath]

Thanks, I appreciate you taking the time to help. Here are my answers:

Can you gist your GetMapping() response and your search request + response ?

Of course: https://gist.github.com/0f9bdcdd7e444d7dedc5. (Don't pay attention to the double entries for some products in the response. That's just test data.)

if you also specify `SearchAnalyzer="keyword", IndexAnalyzer="keyword" are you getting better results?

No, the results stay the same.

Can you run those test and confirm they are working for you too?

I've ran the tests and they all pass.

Can you gist your entity class, index code, search code ? Privately would be fine too just sent me an email at my github account name @ gmail
Perhaps we can even schedule a teamviewer (or something of the sort) session to debug this?

The search code is part of a larger project. I'll try to reduce the code to a smaller case that reproduces this behaviour, so it's easier to debug. I'll let you know.

[Mpdreamz]

Can you also gist your class, code that does the mapping ?

[TowardsDeath]

I've created an MVC project with a simple piece of indexing and search code where not_analyzed does not work. If you run it and press the "Add"-button, it indexes a product, and you should see a facet with two values instead of the expected one.

[Mpdreamz]

Appreciate the effort in setting that up! Helped me find the issue quickly.

It seems ElasticSearch no longer creates an index if it doesn't exist on the mapping call but will do so on the add call. So the mapping call threw a 500 but the index call still happened creating a default mapping. On later add calls the mapping call still throws a 500 but now it complaints it cant update the mapping because it can't handle changing indexing options.

I've sent a pull request that creates the index on application_start and now all seems well.

Handling updated schema versions updates is probably the most dificult thing in search systems (not just elasticsearch's problem) and the only way to manage it a bit in Elasticsearch is using it's aliasing mechanism (which NEST supports) and reindexing on big schema changes. Elasticsearch can update and merge quite a lot of schema changes though, Analysis settings is not one of them though.

[TowardsDeath]

Thanks for your help. So the trick is to create the index before persisting the mapping. Seems like an easy fix.

I don't necessarily want to have lots of schema updates, and the ElasticSearch code in my original web app actually tries to avoid that. It's just that this example code demonstrates the issue.

I did try to only do a mapping when there's no mapping present, but GetMapping<ElasticSearchProduct>() throws a StackOverflowException, even if the mapping and index are both present, but I guess that's another issue. (When I run the test you have for that method, IndicesTest.PutMapping(), my NUnit instance crashes.) The problem seems to be that GetMapping<T> keeps calling itself, see https://github.com/Mpdreamz/NEST/blob/master/src/Nest/ElasticClient-MappingType.cs#L223. If you want, I can send you a pull request with a fix.

[Mpdreamz]

Yikes, pull request are always welcome!

[vorou]

@Mpdreamz it would be nice to add a remark to NEST docs, probably. I was looking at this page http://nest.azurewebsites.net/indices/put-mapping.html and didn't manage to understand what else should I do to make the attributes work.

But, on the other hand, nobody else has complained about that for 2 years, so, maybe I'm doing something wrong? Is creating the index explicitly considered a preferred way of working with NEST?

[mickdelaney]

i too have hit this problem (i.e. not being clear in the docs).

[emwalker]

I've run into this problem as well.

[rajatg64]

same problem here. anyone got the solution?

[russcam]

@rajatg64 it'd be best to open a new issue and describe what the problem you're seeing is.