How to remove shard allocation settings

[psaiz]

Hi,

I wanted to use curator to remove some of the shard allocation filtering that I had already enabled. So, basically, before running curator the index settings looked like:

  "index_test" : {
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "include" : {
              "my_key" : "my_value"
            }
          }
        },

and I want that after running curator they look like

  "index_test" : {
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "exclude" : {
              "my_key" : "my_value"
            }
          }
        },

If I define an action like

    options:
      key:  my_key
      value: my_value
      allocation_type: exclude
      ignore_empty_list: True

then, the settings will have both the include and the exclude. Then, if I define the action to remove it, something like

  options:
      key: my_key
      value: None
      allocation_type: include

then, I get the error:

Failed to complete action: allocation.  <class 'curator.exceptions.MissingArgument'>: No value for "value" provided

I've tried with value: "", or leaving it empty, and I got the same message. How can I remove it?

Thanks a lot,
pablo

[untergeek]

You can't "unset" it, as there are no examples of that in the documentation, but you can use an arbitrary value:

index.routing.allocation.exclude.{attribute}
Assign the index to a node whose {attribute} has none of the comma-separated values.

Set the exclude to an arbitrary value that none of your Elasticsearch will recognize. If you use an arbitrary value, none of the nodes should have that value, so the shard can be assigned there.

[psaiz]

Hi,

Thanks for your reply. The issue is that the one I want to 'unset' is the 'include' (not exclude). If I set an arbitrary value, it will not do the allocation correctly

From the command line, I can do something like

 curl -X PUT localhost:9200/my_index/_settings -d  '{"index.routing.allocation.include.my_key":""}'

and that works as expected.
Would it be possible to drop the restriction that 'value' has to be defined?

Cheers,
pablo

[untergeek]

That's worth considering. I'll look into it.

[untergeek]

@psaiz I did some testing. This is only possible in the 5.x branch. In the 2.x branch, it fails. The API does not allow for this in 2.x.

>>> client = elasticsearch.Elasticsearch()
>>> client.indices.create(index='alltest')
{u'acknowledged': True}
>>> client.indices.put_settings(index='alltest', body={'index.routing.allocation.exclude.foo': 'bar'})
{u'acknowledged': True}
>>> client.indices.put_settings(index='alltest', body={'index.routing.allocation.exclude.foo': None})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/buh/Library/Python/2.7/lib/python/site-packages/elasticsearch/client/utils.py", line 71, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/Users/buh/Library/Python/2.7/lib/python/site-packages/elasticsearch/client/indices.py", line 564, in put_settings
    '_settings'), params=params, body=body)
  File "/Users/buh/Library/Python/2.7/lib/python/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/Users/buh/Library/Python/2.7/lib/python/site-packages/elasticsearch/connection/http_urllib3.py", line 127, in perform_request
    self._raise_error(response.status, raw_data)
  File "/Users/buh/Library/Python/2.7/lib/python/site-packages/elasticsearch/connection/base.py", line 122, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'action_request_validation_exception', u'Validation Failed: 1: no settings to update;')

So I tried sending an empty string:

>>> client.indices.put_settings(index='alltest', body={'index.routing.allocation.exclude.foo':''})
{u'acknowledged': True}
>>> client.indices.get_settings(index='alltest')
{u'alltest': {u'settings': {u'index': {u'number_of_replicas': u'1', u'uuid': u'hT-yeFl6TAmIV7ILMofGTw', u'number_of_shards': u'5', u'creation_date': u'1489691896090', u'version': {u'created': u'2040299'}, u'routing': {u'allocation': {u'exclude': {u'foo': u''}}}}}}}

It still doesn't do any better than the arbitrary value approach I mentioned.

So, this will make it into Curator 5, but not Curator 4, as it will not support the 2.x versions at all.