Cannot set indexing_pressure.memory.limit

[rafisnih]

Elasticsearch Version

8.5.2

Installed Plugins

our_custom_but_not_related

Java Version

bundled

OS Version

5.15.0-56-generic #62~20.04.1-Ubuntu SMP Tue Nov 22 21:24:20 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Problem Description

There are 2 problems setting indexing_pressure.memory.limit in index settings.

First, parseInt is used to parse the setting value which limits it to 2g.

Second, even if I use smaller value, it does not work.

Third, not an error, but it would be convenient to set the value as percentage, for example "40%".

Steps to Reproduce

First, parseInt is used to parse the setting value which limits it to 2g. My settings:

{
  "index": {
    "auto_expand_replicas": "0-all",
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "refresh_interval": -1,
    "codec": "best_compression"
  },
  "indexing_pressure": {
    "memory": {
      "limit": "10737418240"
    }
  }
}

Error:

Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.indices.IndexingPressureMemory: java.lang.NumberFormatException: For input string: "10737418248" (JSON path: indexing_pressure.memory.limit) (line no=11, column no=29, offset=242)
	at co.elastic.clients.json.JsonpMappingException.from0(JsonpMappingException.java:134)
	at co.elastic.clients.json.JsonpMappingException.from(JsonpMappingException.java:121)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:218)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:148)
	at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:76)
	at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
	at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
	at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:192)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:148)
	at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:76)
	at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
	at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
	at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:192)
	at co.elastic.clients.util.WithJsonObjectBuilderBase.withJson(WithJsonObjectBuilderBase.java:54)
	at co.elastic.clients.json.WithJson.withJson(WithJson.java:43)
        ...
Caused by: java.lang.NumberFormatException: For input string: "10737418240"
	at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
	at java.base/java.lang.Integer.parseInt(Integer.java:668)
	at java.base/java.lang.Integer.valueOf(Integer.java:999)
	at co.elastic.clients.json.JsonpDeserializerBase$2.deserialize(JsonpDeserializerBase.java:119)
	at co.elastic.clients.json.JsonpDeserializerBase$2.deserialize(JsonpDeserializerBase.java:115)
	at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:76)
	at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
	at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:192)
	... 36 common frames omitted

Second, even if I use smaller value, it does not work. My settings with 1g:

{
  "index": {
    "auto_expand_replicas": "0-all",
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "refresh_interval": -1,
    "codec": "best_compression"
  },
  "indexing_pressure": {
    "memory": {
      "limit": "1073741824"
    }
  }
}

Error:

Caused by: co.elastic.clients.elasticsearch._types.ElasticsearchException: [es/indices.create] failed: [illegal_argument_exception] unknown setting [index.indexing_pressure.memory.limit] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
	at co.elastic.clients.transport.rest_client.RestClientTransport.getHighLevelResponse(RestClientTransport.java:282)
	at co.elastic.clients.transport.rest_client.RestClientTransport.performRequest(RestClientTransport.java:148)
	at co.elastic.clients.elasticsearch.indices.ElasticsearchIndicesClient.create(ElasticsearchIndicesClient.java:266)
	at co.elastic.clients.elasticsearch.indices.ElasticsearchIndicesClient.create(ElasticsearchIndicesClient.java:282)

See the prefix index. above

Logs (if relevant)

No response

[DaveCTurner]

I think this is behaving as expected: the limit is node-wide but you're trying to set it on a specific index. This appears to be a user question, and we'd like to direct these kinds of things to the Elasticsearch forum. If you can stop by there, we'd appreciate it. This allows us to use GitHub for verified bug reports, feature requests, and pull requests.

There's an active community in the forum that should be able to help get an answer to your question. As such, I hope you don't mind that I close this.