API to simplify repair of red indices from a good snapshot copy

[Leaf-Lin]

Description

Today, if the data in hot/warm tier is lost and an index becomes red, it takes some manual process to recover:

1. Identify which snapshot contains the latest data.
2. Close/delete the existing red index.
3. Restore data from snapshot.

Since data from the cold/frozen tier can self-recover via searchable snapshot already, it would be great if this ability can be extended to hot/warm data via some configuration as autohealing.

This is an advanced proposal to automatically recover data related to elastic/kibana#137368

[elasticsearchmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[DaveCTurner]

I think we should not do this fully automatically: indices in the hot and warm tiers are still writeable so restoring from a recent snapshot represents some amount of risk of data loss. We can make an API to encapsulate the steps of this process into a single action but I think it's necessary to keep the user in the loop in any case. The linked Kibana issue also appears to be more about simplifying the process rather than making it fully automatic. I will adjust the title to clarify this.