Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cluster / Index level allocation filtering #1311

Closed
kimchy opened this issue Sep 8, 2011 · 0 comments

Comments

@kimchy
Copy link
Member

commented Sep 8, 2011

Allow to control allocation if indices on nodes based on include/exclude filters. The filters can be set both on the index level and on the cluster level. Lets start with an example of setting it on the cluster level:

Lets say we have 4 nodes, each has specific attribute called tag associated with it (the name of the attribute can be any name). Each node has a specific value associated with tag. Node 1 has a setting node.tag: value1, Node 2 a setting of node.tag: value2, and so on.

We can create an index that will only deploy on nodes that have tag set to value1 and value2 by setting index.routing.allocation.include.tag to value1,value2. For example:

curl -XPUT localhost:9200/test -d '{
    "index.routing.allocation.include.tag" : "value1,value2"
}'

On the other hand, we can create an index that will be deployed on all nodes except for nodes with a tag of value value3 by setting index.routing.allocation.exclude.tag to value3. For example:

curl -XPUT localhost:9200/test -d '{
    "index.routing.allocation.exclude.tag" : "value3"
}'

The include and exclude values can have generic simple matching wildcards, for example, value1*. A special attribute name called _ip can be used to match on node ip values.

Obviously a node can have several attributes associated with it, and both the attribute name and value are controlled in the setting. For example, here is a sample of several node configurations:

node.group1: group1_value1
node.group2: group2_value4

In the same manner, include and exclude can work against several attributes, for example:

curl -XPUT localhost:9200/test -d '{
    "index.routing.allocation.include.group1" : "xxx"
    "index.routing.allocation.include.group2" : "yyy",
    "index.routing.allocation.exclude.group3" : "zzz",
}'

The provided settings can also be updated in real time using the update settings API, allowing to "move" indices (shards) around in realtime.

Cluster wide filtering can also be defined, and be updated in real time using the cluster update settings API. This setting can come in handy for things like decommissioning nodes (even if the replica count is set to 0). Here is a sample of how to decommission a node based on _ip address:

curl -XPUT localhost:9200/_cluster/_settings -d '{
    "transient" : {
        "cluster.routing.allocation.exclude._ip" : "10.0.0.1"
    }
}'

@kimchy kimchy closed this in 5052282 Sep 8, 2011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.