Failure while restoring a source-only snapshot including a synthetic source index #112735
Description
Elasticsearch Version
8.15 and above
Installed Plugins
No response
Java Version
bundled
OS Version
all
Problem Description
Taking a snapshot using a source-only repository results in an error when trying to restore the snapshot from the source-only repository. Something like
junit.framework.AssertionFailedError: Unexpected exception type, expected IllegalArgumentException but got org.elasticsearch.client.ResponseException: method [POST], host [http://[::1]:49921], URI [_snapshot/lwclgyiaft/dlamhgdc/_restore?wait_for_completion=true], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"snapshot_restore_exception","reason":"[lwclgyiaft:dlamhgdc/opM_gmRIQQmp-baqUjm_BQ] index [fyeybahtkorp] wasn't fully snapshotted - cannot restore"}],"type":"snapshot_restore_exception","reason":"[lwclgyiaft:dlamhgdc/opM_gmRIQQmp-baqUjm_BQ] index [fyeybahtkorp] wasn't fully snapshotted - cannot restore"},"status":500}
At snapshot creation time a user is not notified about the issue. As a result, if a user tries to create a snapshot including indices using synthetic source, the user will discover the issue only when trying to restore it. This might result in data loss, given that source-only snapshots will include indices whose documents are missing the (stored) _source. At that point only metadata will be available with no opportunity for the user to actually use the source-only snapshot and restore the index.
The snapshot/repository creation API needs to be aware of synthetic source and "fail fast" when trying to create a source-only snapshot including indices adopting synthetic source.
Of course it should also fail with a 4xx error (not a 500 as above).
Steps to Reproduce
Create an index using synthetic source.
Register a "main" repository.
Register a source-only repository using the "main" repository as the delegate.
Create a snapshot.
Delete the original index.
Restore the source-only snapshot from the source-only repository.
Logs (if relevant)
No response