Whether synthetic source is enabled checks are inconsistent

[Kubik42]

The following line will evaluate to false when index.mapping.source.mode is missing but _source.mode is set.

_source.mode is the "old" way of enabling synthetic source. That said, some indices may still be using it.

This creates inconsistency as syntheticSourceSupport() gets invoked, all the while the field mapper indexes documents with the assumption that synthetic source is disabled.

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[Kubik42]

I was reading the code to try to understand why the deprecated _source.mode is still being used. This happens for BWC sake. Customers that have enabled synthetic source using _source.mode in indices prior to this version will have _source.mode enforced. Meanwhile, customers using _source.mode in new indices, will have that settings effectively ignored.

What happened in our case is the index was created prior to these changes, so _source.mode was still enforced. Meanwhile, in field mappers (ex. text and keyword), the isSyntheticSourceEnabled flag is set to false since it gets its value directly from settings.index.mapping.mode.

No longer enforcing _source.mode is not a good idea as it needs to be BWC with older indices. So the path forward is to have the isSyntheticSourceEnabled flag be aware of _source.mode somehow.

[Kubik42]

One, nearly complete, way of making the isSyntheticSourceEnabled flag aware of _source.mode is to forego using that flag entirely and instead rely on this. Luckily, this is set here, and takes its value directly from mode, so its not susceptible to the inconsistency issue.

The reason why this solution isn't 100% is because the Builders in TextFieldMapper and AnnotatedTextFieldMapper don't have access to this information and hence will still use the wrong value for isSyntheticSourceEnabled. This means, the following checks will fail:

• elasticsearch/server/src/main/java/org/elasticsearch/index/mapper/TextFieldMapper.java

  Line 331 in 12df171

  return isSyntheticSourceEnabled

• elasticsearch/server/src/main/java/org/elasticsearch/index/mapper/TextFieldMapper.java

  Line 335 in 12df171

  return isSyntheticSourceEnabled && multiFieldsBuilder.hasSyntheticSourceCompatibleKeywordField() == false;

• elasticsearch/plugins/mapper-annotated-text/src/main/java/org/elasticsearch/index/mapper/annotatedtext/AnnotatedTextFieldMapper.java

  Line 114 in 12df171

  return isSyntheticSourceEnabled() && multiFieldsBuilder.hasSyntheticSourceCompatibleKeywordField() == false;

However, given the fact that these checks currently don't work for legacy indices anyway, this might be ok.