With snapshots being part of the new features of ES 1.0 it makes sense to have Curator be able to capture snapshots and save to a designated target (S3, for example). Bonus points if we get it to delete the old index after a successful snapshot/backup.
The text was updated successfully, but these errors were encountered:
I would say that the functionality that would best align with Curator would be to allow for removal of old snapshots within whatever repository is used (configured as an outside process) at a specified threshold. Since repository setup is a separate task, this functionality would work with all current and future repository providers. To accomplish this, you would need time based snapshot names, but combining this with the trimming of curator in one script would be awesome! Perhaps curator for local indices + snapshots for backup and/or archiving.
@imperialwicket I'm not sure how restore would necessarily fit in. Curator feels like a program that acts as a maintenance worker for your Elasticsearch needs. Daily maintenance, for example, doesn't feel like something you'd want to do "restore" process. In my mind, it certainly feels weird to propose doing a 'restore' via daily cronjob.
Maybe a separate tool or workflow to help doing restores, that is, if the restore API is insufficient?
Of course, I might be totally misunderstanding your request. Can you help me understand it more? :)
@jordansissel fair point; the fact that curator manages snapshots, closing, and deleting seems to qualify it as a candidate for restore capability to me. I completely understand if the concept for curator is 'routine' index maintenance and not general index maintenance, and we can leave restoration for a separate project.
However, I was under the impression that restore could fit into the 'custom tasks' mentioned in #44. That may circumvent this discussion entirely.
Curator is there to make repetitive practices easier, especially with regards to cron jobs. If it's really a daily occurrence for people, then it makes sense to add it. It does make some sense to have a mode to make it easier to "open" closed indices, or potentially to "restore" archived snapshots, but the latter is tricky to implement cleanly with command-line options, while the former is so simple you probably don't need curator to do it. I mean, how often are indices getting restored? In my experience it's 1 out of 1000 or less. When that arrives, you have to decide, "am I going to restore to a different index name?" among other things. Trying to provide all of the command-line flags for that is a yak shave I'd rather avoid, to be honest.
Regardless, snapshots to filesystem and S3 will be the first goals I tackle on that front.