Restore snapshot with data stream name conflict.

[martijnvg]

Restoring a data stream from a snapshot that has the same name as a data stream in the target cluster replaces the data stream in the target cluster with the data stream in the snapshot. Not all backing indices of the data stream in the target cluster may be replaced with the backing indices of the data stream in the snapshot. Because the data stream in the target cluster has backing indices that don't exist in the snapshot. This can result into backing indices (which are now regular indices) in the cluster that aren't part of any data stream after the restore has occurred.

If ilm was active on the backing indices of the replaced data stream prior to the restore then ilm will remain to be active on these lone backing indices. This can result in an error when ilm attempt to rollover these indices, the rollover action will fail because these indices are not part of any data stream.

The main question is what would be the desired behaviour here. If two data streams (one in the snapshot and one in the cluster the snapshot is restored into) have the same name, but the data stream in the target cluster has a backing index which doesn't exist in the snapshot. Maybe it is desired that this index should remain part of the data stream after the restore has occurred. Or maybe just removing it, since logically it is part of a data stream that has been replaced with a data stream from a snapshot.

This issue was discovered as part of an ILM issue with a backing index of a SLM data stream, which is a hidden data stream. Maybe hidden data streams / indices created by features such as SLM or ILM shouldn't be part of a snapshot by default?

[elasticmachine]

Pinging @elastic/es-core-features (Team:Core/Features)

[joegallo]

I'm removing the team-discuss label from some older Team:Data Management issues -- we've had plenty of time to discuss them, but we haven't, so the label isn't serving its purpose. Feel free to delete this comment and/or re-add the team-discuss label.

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)