Enterprise Search controller generates invalid configuration when multiple auth sources are used

[pebrc]

When users configure additional authentication sources in ECK for example Active Directory the operator generates an invalid configuration file.

For example following the instructions in https://www.elastic.co/guide/en/enterprise-search/7.16/active-directory-auth.html

Users would set up a configuration like the following in the Enterprise Search manifest:

  config:
    ent_search.auth:
      native1:
        source: elasticsearch-native
        order: 1
      my_ad:
        source: elasticsearch-active-directory
        order: 2

which leads to a generated configuration as follows:

ent_search:
  auth:
    my_ad:
      order: 2
      source: elasticsearch-active-directory
    native1:
      order: 1
      source: elasticsearch-native
    source: elasticsearch-native

which is invalid.

Omitting the native auth source in the Enterprise Search custom resource config does not help either. The configuration generated is still invalid.

Affected are versions of the Elastic Stack before 7.17 because of #5085

[pebrc]

Enterprise Search 7.10 introduced "namespaced" auth configuration, so instead of

ent_search.auth.source: elasticsearch-native you could write ent_search.auth.native1.source: elasticsearch_native

Enterprise Search 7.14 switched the default from standard which was authentication in Enterprise Search's own user management to elasticsearch-native. So technically since 7.14 we do not need to set this source: elasticsearch-native value anymore.

However Enterprise Search documentation recommended strongly not to change the auth configuration before upgrading to 7.14 because Enterprise Search would use the old configuration to do an automatic migration of users to the new standard elasticsearch-native method based on the existing auth config.

To fix this bug I would suggest to not generate the ent_search.auth.source as of 7.14 which should be fine as any overrides users have made should be preserved and the new default is aligned with what we were using in ECK all along.

Version	User config	Effective Config	Effective Auth Source
7.7-7.13	auth.source: standard	auth.source: standard	standard
7.7-7.13		auth.source: elasticsearch-native	elasticsearch-native
7.14-7.last	auth.source: standard	auth.source: standard	standard
7.14-7.last			elasticsearch-native
8.x	auth.source: standard	error	error
8.x			elasticsearch-native

In addition I would add code for versions 7.10-7.13 to detect namespaced auth configuration and not generate ent_search.auth.source in this case.