Need this utility for ES 2.3.1

[desagar]

Hi -

I have a situation where your cleanup utility would be very useful to me in ES 2.3.1, but I am unable to build it due to compile errors - it looks like some of the APIs the utility uses are no longer valid in 2.3.1 (e.g. SnapshotMetadata and others).

I am using a custom plugin to write the snapshots to an OpenStack Swift repository via http. The plugin has an issue with not releasing http connections, which is causing a snapshot thread to hang indefinitely waiting for a connection. When we did a rolling restart of the ES nodes, we ended up with an ABORTED snapshot. I cannot run any further snapshots in this environment because the snapshots module thinks that the snapshot is IN_PROGRESS.

We are working on fixing the plugin, but in the meantime, we have a production system which is already running the same faulty plugin, and could end up with this scenario in a matter of time. It would be very helpful if we could run your utility in the production system since we are not allowed to patch application code outside of our release cycle.

Any possibility of getting an updated version of this cleanup utility for ES 2.3.1?

[imotov]

This utility was created as a temporary solution for older versions of elasticsearch. Its functionality is built into all recent versions of elasticsearch including 2.3.1. It kicks in automatically on the master node restart. So, if restarting the master node doesn't help, you might have some other situation which this utilitity wouldn't be able to help with either. If this is the case, I would suggest opening an issue on the elasticsearch project with description of the situations and your cluster state dump (snapshot and nodes sections).

[inthecloud247]

Interesting! Is there an issue or documentation you've seen where it shows how this now "kicks in automatically on the master node restart" ?

[imotov]

@inthecloud247 elastic/elasticsearch/pull/11450

[desagar]

Thanks for the feedback @imotov. I plan to try and write a simple reproducer in the next day or two for this. Once I have it clearly demonstrable with the reproducer, I will file the issue against ES.