[BUG] Missing Index Setting Options

[chludwig-haufe]

What is the bug?

org.opensearch.client.opensearch.indices.IndexSettings does not support all configuration options offered by the server. For instance, there is no way to set translog.sync_interval.

How can one reproduce the bug?

If one tries to construct the index settings object via its builder, there is the setter translogDurability(String) for translog.durability, but no corresponding setter for translog.sync_interval. (When I check the settings of an existing index with request parameter include_defaults=true, then I see even more attributes not available in IndexSettings.)

Since many developers are likely to read the index settings and type mappings from a file, the following test seems relevant:

    @Test
    public void createIndexSettingsFromJson() throws IOException {
        ObjectMapper mapper = new JsonMapper();
        JsonpMapper jsonpMapper = new JacksonJsonpMapper(mapper);

        String settingsJson = "{\"index\": {\"translog.durability\": \"async\", \"translog.sync_interval\": \"60s\"}}";
        IndexSettings settings;
        try (JsonParser settingsParser = new JacksonJsonpParser(mapper.createParser(settingsJson))) {
            settings = IndexSettings._DESERIALIZER.deserialize(settingsParser, jsonpMapper);
        }
        
        StringWriter writer = new StringWriter();
        try (JsonGenerator jsonGenerator = Json.createGenerator(writer)) {
            settings.serialize(jsonGenerator, jsonpMapper);
        }
        String roundTripJson = writer.toString();

        assertThat(roundTripJson, Matchers.containsString("durability"));
        assertThat(roundTripJson, Matchers.containsString("async"));
        assertThat(roundTripJson, Matchers.containsString("sync_interval"));
    }

The test fails with the message:

Expected: a string containing "sync_interval"
     but: was "{"index":{"translog.durability":"async"}}"

The attribute translog.sync_interval has been lost.

As an aside: I am surprised the deserializer only recognizes the string-valued attribute translog.durability, not an object-valued attribute translog whose value then contains an attribute durability.

What is the expected behavior?

All index configuration options should be available through the Java client.

What is your host/environment?

OpenSearch 2.4.1, Java client 2.1.0

Do you have any additional context?

I am migrating some code from Elasticsearch 7.10 + Rest high-level client to OpenSearch 2.x + Java client. The index in question will be filled in a long running bulk update. If the ingest of the document corpus fails for some reason, then the index is discarded anyway. Therefore, I prefer to set translog.durability to async and translog.sync_interval to a value in the minute range.

[ginkel]

Probably it would be a good idea to offer a more generic (but not type-safe) way of interacting with OpenSearch for endpoints/features that are not yet available in the client.

[wbeckler]

Feel free to PR this fix, and maybe open an issue about the semi-generic CURL capability

[Mstrutov]

Stumbled upon this problem during migration from Elasticsearch. Working on the PR at the moment, going to add translog.durability support via index settings builder.

[VachaShah]

Stumbled upon this problem during migration from Elasticsearch. Working on the PR at the moment, going to add translog.durability support via index settings builder.

Thank you @Mstrutov!