bug fix about elasticsearch.common.settings.Settings.processSetting

[kkewwei]

Relates #43791

[elasticmachine]

Pinging @elastic/es-core-infra

[williamrandolph]

@kkewwei Thanks for finding a subtle bug and submitting a solution.

We need to add some unit tests before we merge this PR, but first I'm going to describe what seems to me to be going on here in order to try to pin down what the exact problem is and make sure that the rest of the team agrees with us.

Discussion

The Settings#processSetting method is intended to take a setting map and add a setting to it, adjusting the keys as it goes in case of "conflicts" where the new setting implies an object where there is currently a string, or vice versa. We handle and have tests for cases like foo=setting1 foo.bar=setting2. Note that processSetting is called from getAsStructuredMap, which is iterating over a sorted list, meaning that keys are added to maps in "alphabetical" order. Thus I will use alphabetical keys in the examples below rather than the usual "foo", "bar", "baz" keys.

It looks to me like processSetting was failing in two cases:

1. Adding a setting two levels under a string:

ant: value1
ant.bee.cat: value2

…was being added as:

{
  "ant": "value1",
  "bee": {
    "cat": "value2"
  }
}

The change to line 147 fixes this bug. The ClassCast exception in the original bug report occurs when the map is already in this bad state, with settings like the following:

ant: value1
ant.bee.cat: value2
bee.cat: value3

2. Adding a setting two levels under a string and four levels under a map

ant: value1
ant.bee.cat: value2
ant.bee.cat.dog.ewe: value3

…was resulting in:

{
  "ant": "value1,
  "bee": {
    "cat": {
      "dog": {
        "ewe": "value3"
      }
    }
  }
}

In this case, it drops a value. That's no good! The change in line 153 fixes this case.

Unit Test Suggestions

It's pretty easy to reproduce this bug by copying and adjusting code from SettingsTests. In particular, I would suggest looking at SettingsTests#testToXContent and writing a unit test to cover the cases above. Will you be able to add some unit tests? I will be happy to help or share some of my scratch code.

[williamrandolph]

This is a great catch and a great fix. We need to add at least two unit tests before merging - see the comment I left on the ticket.

[kkewwei]

It's my pleasure to add some unit tests, and I am very grateful for your sharding code.

[williamrandolph]

I apologize for my delay in reviewing this code. I have been traveling.

The idea here is correct, but we can't access the processSetting method the way you've done it in this code. The processSetting method is private, and I don't believe we should change that. However, we can correctly test the code changes if we build our settings as follows:

    public void testProcessSetting() throws IOException {
        Settings test = Settings.builder()
            .put("ant", "value1")
            .put("ant.bee.cat", "value2")
            .put("bee.cat", "value3")
            .build();
        XContentBuilder builder = XContentBuilder.builder(XContentType.JSON.xContent());
        builder.startObject();
        test.toXContent(builder, new ToXContent.MapParams(Collections.emptyMap()));
        builder.endObject();
        assertEquals("{\"ant.bee\":{\"cat\":\"value2\"},\"ant\":\"value1\",\"bee\":{\"cat\":\"value3\"}}", Strings.toString(builder));

        test = Settings.builder()
            .put("ant", "value1")
            .put("ant.bee.cat", "value2")
            .put("ant.bee.cat.dog.ewe", "value3")
            .build();
        builder = XContentBuilder.builder(XContentType.JSON.xContent());
        builder.startObject();
        test.toXContent(builder, new ToXContent.MapParams(Collections.emptyMap()));
        builder.endObject();
        assertEquals("{\"ant.bee\":{\"cat.dog\":{\"ewe\":\"value3\"},\"cat\":\"value2\"},\"ant\":\"value1\"}", Strings.toString(builder));
    }

This is the approach that is used in the TestSettings#testToXContent method.

[williamrandolph]

@elasticmachine test this please

[kkewwei]

Thank you for your sharind code which I use in unit test, It is a good idea to test it.

[williamrandolph]

This looks very good. Thank you for your prompt revisions, and for taking the time to improve Elasticsearch!