Backing up and Restoring a Specific Index

[talsalmona]

Description:
Allow an index level granularity of backing up and restoring of indexes.

Use Case:
In a multi-tenant application, where there is an index per tenant, there is often a need to restore an index to a previous "snapshot".
It is even possible that a tenant owner / admin will be allowed to "go back in time" to a previous version of the index.

Feature Request:

1. Allow a per index backup and restore.
2. Right after the restore, the newest document in the index should be the newest document in the snapshot we restored from. Documents related to this index in the transaction log or the work folders should be removed.

A possibly more complicated feature request:
Have an API to define index periodic "save points". e.g. create a save point every 10 minutes.
Have another API to restore an index to a specific save point.
An additional API that would let us list the current available save points.

The process of backing up an index to NFS would be:

1. Define a periodic snapshot of every X minutes.
2. Snapshot the NFS every X minutes (with the S3 integration this can be automatic)

The process of restoring:

1. Restore a snapshot from NFS.
2. List the available snapshots (through API)
3. Restore to the latest snapshot.
   (with S3, this could be nicer: list the available snapshot then retrieve from S3 and restore in one API call)

• In the advanced feature request, the periodic mechanism can be left for some external scheduler.

See also: http://groups.google.com/a/elasticsearch.com/group/users/browse_thread/thread/fa111bbdcfe5872a/1ce33465200d66e1#1ce33465200d66e1

[medcl]

oh,time machine,cool feature

[spinscale]

Closing this as a duplicate of #3070 (even though it is the older one)
See #2458 for more info