Skip to content

[server] Active log segment is not rolled after table.log.ttl expires#3830

Description

@fxbing

Search before asking

  • I searched in the issues and found nothing similar.

Fluss version

main (development)

Please describe the bug 馃悶

When remote log is enabled and no new records are appended after the active log segment exceeds table.log.ttl, the periodic local retention cleanup does not roll the active segment.

Steps to reproduce

  1. Enable remote log with remote.log.task-interval-duration > 0.
  2. Configure a short table.log.ttl.
  3. Append records to create a non-empty active segment.
  4. Stop writing and wait until the active segment exceeds the configured TTL.
  5. Trigger the periodic local log retention cleanup.

Actual behavior

The active segment remains active and is not uploaded to remote storage. It is therefore not eligible for the subsequent local segment cleanup.

The current implementation of LogTablet.deletableExpiredSegments() only scans inactive segments and excludes the active segment.

Expected behavior

The cleanup should roll a non-empty expired active segment when:

  • remote log is enabled;
  • the active segment is expired by table.log.ttl;
  • highWatermark >= logEndOffset.

The rolled segment should not be deleted in the same cleanup pass. After it is uploaded to remote storage, a later cleanup pass should remove it according to the existing remote boundary and retention rules.

This behavior is similar to KAFKA-15047.

Solution

  • Extend the existing contiguous TTL scan to inspect the active segment.
  • Roll the active segment only after preceding inactive segments satisfy the existing TTL and offset boundaries.
  • Keep the active segment out of the current deletion list.
  • Preserve the existing remoteLogEndOffset, minRetainOffset, high-watermark, and empty-active-segment behavior.
  • Add regression coverage in TieredLocalSegmentTTLTest for:
    • active segment roll after TTL expiration;
    • high watermark below LEO;
    • remote end offset boundaries;
    • remote log disabled;
    • empty active segment.

This change does not modify any public API, RPC protocol, or storage format.

Potential leader/follower segment boundary differences during timestamp roll are out of scope and will be handled separately.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions