No obvious way to delete persistent cluster admin settings

[andrewclegg]

There's no delete API for these, if you want to set them back to defaults on next startup.

karmi on irc suggested setting them to the empty string, e.g.:

curl -XPUT 'localhost:9200/_cluster/settings' -d '{"persistent":{"threadpool.search.type":""}}'

But then on next restart I got warnings like this:

[2013-09-11 20:08:41,548][WARN ][node.settings            ] [lo3uppaldbs009.palomino.pearson.com] failed to refresh settings for [org.elasticsearch.threadpool.ThreadPool$ApplySettings@49e3f731]
org.elasticsearch.ElasticSearchIllegalArgumentException: No type found [], for [search]
        at org.elasticsearch.threadpool.ThreadPool.rebuild(ThreadPool.java:452)
        at org.elasticsearch.threadpool.ThreadPool.updateSettings(ThreadPool.java:468)
        at org.elasticsearch.threadpool.ThreadPool$ApplySettings.onRefreshSettings(ThreadPool.java:809)
        at org.elasticsearch.node.settings.NodeSettingsService.clusterChanged(NodeSettingsService.java:84)
        at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:321)
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:95)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)

So now I have no way to tell what kind of thread pool I have.

Version: 0.90.3

[andrewclegg]

As a workaround, I had to stop the cluster, delete all my global state files, and restart, so it would reset to defaults (and anything defined in elasticsearch.yml). Thankfully I didn't have any other dynamically-added cluster settings.

[jennysivapalan]

We had a similar issue, we tried to update the threadpool settings dynamically. The first time it applied but did not send an acknowledgement. Subsequent tries caused an error:-
ProcessClusterEventTimeoutException: failed to process cluster event (reroute_after_cluster_update_settings) within 30s.
We fixed this by also shutting down the cluster and deleting global state files.

[jippi]

👍

[clintongormley]

Depends on #5019

[clintongormley]

Closed in favour of #6732