enhance: [2.3] retry to read when the s3 get the unexpect eof error

[SimFG]

issue: #30877
pr: #30861

[mergify]

@SimFG E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

[SimFG]

/run-cpu-e2e

[codecov]

Codecov Report

Attention: Patch coverage is 54.63918% with 44 lines in your changes are missing coverage. Please review.

Project coverage is 82.18%. Comparing base (ef84d40) to head (3a497ff).
Report is 2 commits behind head on 2.3.

Additional details and impacted files

@@           Coverage Diff            @@
##              2.3   #30976    +/-   ##
========================================
  Coverage   82.18%   82.18%            
========================================
  Files         843      843            
  Lines      121541   121687   +146     
========================================
+ Hits        99892   100012   +120     
- Misses      18421    18448    +27     
+ Partials     3228     3227     -1     

Files	Coverage Δ
pkg/util/merr/errors.go	91.07% <ø> (ø)
pkg/util/retry/options.go	100.00% <100.00%> (ø)
pkg/util/retry/retry.go	100.00% <100.00%> (ø)
pkg/util/merr/utils.go	88.13% <40.00%> (-0.39%)	⬇️
internal/storage/remote_chunk_manager.go	68.52% <53.48%> (-0.76%)	⬇️
internal/storage/minio_chunk_manager.go	74.61% <50.00%> (-0.23%)	⬇️

... and 25 files with indirect coverage changes

[xiaofan-luan]

/lgtm
/approve

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: SimFG, xiaofan-luan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• internal/storage/OWNERS [xiaofan-luan]
• pkg/util/OWNERS [xiaofan-luan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment