Allow retaining more than one snapshot

[deepthidevaki]

Only specific snapshots are valid to be included in the backup. Specifically a snapshot can be included in the backup if snapshot position < lastWrittenPosition < checkpoint position. Otherwise, if we recover from the backup, the state will be inconsistent.
Before taking a backup, BackupManager finds a snapshot that satisfies the above condition. If there is no suitable snapshot, backup will fail. Since snapshots are taken quite frequently, this can increase the chance of not taking a backup. To prevent the backup from failing, it would be better if we retain more than one snapshot. So if the latest snapshot cannot be included in the backup, it can choose a previous one.

To achieve this:

• Find a solution for Multiple snapshots are incompatible with Raft "catch-up" implementation #3573 This is a blocker.
• On startup, SnapshotStore should not delete all snapshots older that the latest one
• After taking a snapshot, delete older snapshot based on the allowed number of snapshots.
• Compaction based on oldest snapshot. (This is already done.)
• The maximum number of retained snapshots can be configured.

[megglos]

Planning:
worth investigating till 8.2 release, putting into backlog right now to be picked up in next planning

[megglos]

Decided to keep it in backlog for now as we have more urgent work and right now we don't consider this to be an issue.