Fix audit log import to MySQL flag removal for old snapshots and skip rsync'ed indices

[juruen]

This PR solves two issues that could potentially cause data loss in the upcoming 2.17 version.

Fix audit log import flag

Problem

If an old snapshot (<2.17) is restored in a 2.17 instance where the audit import to MySQL is marked as complete in /data/user/common/audit-log-import/complete, the restored instance wrongly keeps the flag that indicates the import is complete.

This could cause a data loss because under certain circumstances a new backup would assume that data is migrated to MySQL when it's not.

Solution

The solution is simple: if a restored snapshot doesn't contain an audit-log-mysql directory, it means it's an old snapshot. In that case, we remove the flag on the instance where the restore is happening.

Skip rsync'ed audit log indices when GHE_BACKUP_ES_AUDIT_LOGS=no

Problem

Audit log indices can be backed up from:

• MySQL
• Elasticsearch through JSON dumps
• Elasticsearch through rsync

When GHE_BACKUP_ES_AUDIT_LOGS=no only MySQL should be used.

JSON dumps are correctly skipped, however all the indices including the audit log ones are rsynced in a subsequent step.

Solution

When GHE_BACKUP_ES_AUDIT_LOGS=no and the import is complete, we configure the rsync step to ignore all the audit log indices.

To do so, we build a file that contains the UUIDs of the audit log indices. Note this is what is used in disk and not the actual index name.

The UUIDs are fetched through a simple:

curl -s http://localhost:9201/_cat/indices/audit_log?h=uuid

The file containing this list is passed as a--exclude-from rsync parameter that contains one directory/pattern to exclude per line.

[anth1y]

thanks for the comments... lgtm