[fix][broker] Avoid consumers receiving acknowledged messages from compacted topic after reconnection

[coderzc]

Fixes #21074

Motivation

In #11287, we use MessageId.earliest instead of the earliest available message to read on the topic when we first read after reconnection, this will lead to consumers receiving acknowledged messages from the compacted topic after reconnection, so I think for a durable cursor (exclude __compaction cursor), we should read messages from the mark-delete position.

Modifications

• If a durable cursor (exclude __compaction cursor) has the mark-delete position, don't read messages from the earliest.
• To avoid skipping the remaining data in the compacted ledger after rewind, we only skip invalid md-position when recovering cursor and add rewind(boolean readCompacted) method in ManagedCursor and just rewind to the next message of the mark delete position instead of resetting the read position to the next valid position of the original topic.
• This change will also make subscriptionInitialPosition=Latest work normally since the mark-delete doesn't is invalid position (entryID==-1L).

Verifying this change

• Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

• Added integration tests for end-to-end deployment with large payloads (10MB)
• Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository:

[github-actions]

The pr had no activity for 30 days, mark with Stale label.

[heesung-sn]

@coderzc could you help to cherry-pick this to branch-3.0? I see conflict.