Log Backup Task Randomly Paused due to Failed to get Initial Snapshot

[YuJuncen]

Bug Report

What version of TiKV are you using?

v6.5.x, but I guess this impacts all released versions (v6.5, v7.1, v7.5). (Maybe not for master because #16008 refines the retry strategy.)

What operating system and CPU are you using?

Irrelative.

Steps to reproduce

Start a log backup task in the cluster. Maybe trigger some merges or restarting TiKV.

What did you expect?

Log backup should run normally.

What did happened?

It encountered a fatal error:

retry time exceeds: and error and error failed to get initial snapshot: 
failed to get the snapshot (region_id = ...): 
Error during requesting raftstore: message: "read index not ready, reason can not read index due to merge, region ..." read_index_not_ready { reason: "can not read index due to merge" region_id: ... }: 
failed to get initial snapshot: failed to get the snapshot (region_id = ...): 
Error during requesting raftstore: message: "read index not ready, reason can not read index due to merge, region ..." read_index_not_ready { reason: "can not read index due to merge" region_id: ... }: 
failed to get initial snapshot: failed to get the snapshot (region_id = ...): 
Error during requesting raftstore: message: "read index not ready, reason can not read index due to merge, region ..." read_index_not_ready { reason: "can not read index due to merge" region_id: ... }

Or, sometimes:

retry time exceeds: and error and error failed to get initial snapshot: 
failed to get the snapshot (region_id = ...): 
Other Error: [components/backup-stream/src/event_loader.rs:320]: message 'CaptureChange' dropped for region ...: oneshot canceled: failed to get initial snapshot: 
failed to get the snapshot (region_id = ...): 
Other Error: [components/backup-stream/src/event_loader.rs:320]: message 'CaptureChange' dropped for region ...: oneshot canceled: failed to get initial snapshot: 
failed to get the snapshot (region_id = ...): 
Other Error: [components/backup-stream/src/event_loader.rs:320]: message 'CaptureChange' dropped for region ...: oneshot canceled