Skip to content

fix(log): preserve confirm offset metadata on append ack - #3531

Merged
superhx merged 1 commit into
1.7from
backport/confirm-offset-metadata-1.7
Aug 20, 2026
Merged

fix(log): preserve confirm offset metadata on append ack#3531
superhx merged 1 commit into
1.7from
backport/confirm-offset-metadata-1.7

Conversation

@superhx

@superhx superhx commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Why

ElasticLog.append constructed LogOffsetMetadata inside the asynchronous WAL acknowledgement callback. The callback used the append's captured endOffset, but read activeSegment.size when the callback ran.

This allowed two appends to produce inconsistent confirm-offset metadata:

  1. Append A updates LEO to (offset=1, position=100 KiB) and waits for its WAL acknowledgement.
  2. Append B appends to the same active segment, advancing it to (offset=2, position=200 KiB).
  3. Append A's acknowledgement callback runs and publishes (offset=1, position=200 KiB): the message offset belongs to Append A, while the physical position belongs to Append B.
  4. A consumer Fetch at offset 1 observes no available records and enters delayed-fetch purgatory with (offset=1, position=200 KiB) as its start metadata.
  5. Append B is acknowledged and advances HW to (offset=2, position=200 KiB).
  6. Although offset 1 is now consumable, DelayedFetch calculates a zero position difference and does not complete the request.

The affected Fetch remains delayed until fetch.max.wait.ms expires, another partition satisfies min.bytes, or a later append advances the position again. The issue does not expose records beyond HW or persist in the incremental Fetch session, but it can add up to one Fetch wait interval to record E2E latency.

RouterIn appends also run outside the normal KafkaApis.handle lifecycle. Actions added while appending records were therefore not drained immediately, which could delay purgatory checks for that path.

What

Capture logEndOffsetMetadata immediately after updating LEO and publish that immutable snapshot when the WAL acknowledgement completes. This keeps the message offset and physical position from the same append.

Also drain the delayed-action queue after a RouterIn append, matching the completion behavior of requests handled through KafkaApis.handle.

@superhx
superhx enabled auto-merge (squash) August 20, 2026 08:39
@superhx
superhx merged commit 00e3513 into 1.7 Aug 20, 2026
7 checks passed
@superhx
superhx deleted the backport/confirm-offset-metadata-1.7 branch August 20, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants