Mapper: An analyzer mapper allowing to control the index analyzer of a document based on a document field, closed by 171fa4a.

Hi Shay,
I couldn't make this feature work, I think it is broken, take a look at: AnalyzerMapper
line 85: String value = context.doc().get(path);

When indexing a document with:
curl -XPUT 'http://sd:5100/test/s/ZZZ' -d '
{
"analyzer_field":"es_analyzer",
"description":"an mp3",
"title":"mp3"
}
'

value is null, because context.doc() doesn't contain the analyzer field in it.
I don't know exactly how to fix the problem though.

Sebastian.

Did you set a mapping for type s with _analyzer definition that points to analyzer_field? The analyzer_field was just an example for the field name that will control the analysis.

Maybe there should be a default for this, for example, if you place in the json _analyzer property, then it will be used, so you won't even need to define a mapping unless you want the analyzer to be driven by a different json field. I will change that.

Yes I did, the file looks like:

{
"s" : {
"_all" : {"enabled" : false},
"_source" : {"enabled" : false},
"_analyzer" : {"path" : "analyzer_field"},
"dynamic" : false,
"properties" : {
"title" : {
"type" : "string",
"store" : "yes",
"index" : "analyzed",
"analyzer" : "all_analyzer",
"term_vector" : "with_positions_offsets"
},
"description" : {
"type" : "string",
"store" : "yes",
"index" : "analyzed",
"term_vector" : "with_positions_offsets"
},
...
and I've used a field named "analyzer_field" in my JSON as you can see in my previous post.

I will try the "_analyzer" default one

Thanks

I see the problem, because the mapping you have is not dynamic, then you need to explicitly define the "analyzer_field" in the properties mapping so it will get added to the document. Same will happen with _analyzer you will need to add it to the properties part. Why don't you use dynamic?

I see, (well, sort of trying to).

I didn't use dynamic mappings as a safety measure, to rule out wrong mappings from my client code, so I won't push invalid unsearchable data to an index, with the overhead it involves taking into account the scale and scarcity of a SSD space for example. Do you think this is not a good idea?

So, what are my options? What do you mean by define an "_analyzer" or "analyzer_field" in the properties mapping? Adding another field like "title" but containing what?

Thanks
Sebastian.

add a mapping called in the same name (like analyzer_field) of type string.

Hi Shay,
I added the field and everything works now, also tried dynamic mapping and works too. The only thing I noticed that wasn't ok is that the "_analyzer" field gets added to Lucene as well (checked with Luke with dynamic and not dynamic settings), and it shouldn't because it's a "control" field, not data.
Is it feasible to remove it from being added?
Thanks,
Sebastian.

To be more precise I added:
"_analyzer" : {"type" : "string"},
I thought in adding index=no and store=no, but I didn't see any example with both options disabled, is that the way to prevent that field to be added?

Answering to myself, so somebody looking at the issue can use the information. I tried both index and store in "no" and it works:

        "_analyzer" : {"type" : "string","store" : "no","index" : "no"},

The past post wasn't accurate, I made a mistake interpreting the results, the one that works is:
"_analyzer" : {
"type" : "string",
"index" : "no",
"store" : "yes"
},
What happened in case of both "no" is that "_analyzer" is ignored, but Standard analyzer, with english stop words set kicks in and that's why I thought it was working.

Shay, is there a way to avoid adding that "_analyzer" stored field in every document?
Thanks,
Sebastian.

Pushed support for specifying index set to no (store defaults to no). I would argue that almost in all cases you would want it indexed (maybe set to not_analyzed) so later on you can query on it and see what docs where indexed how.