Skip to content

[docs] Document local log TTL and active segment rolling #3938

Description

@wuchong

Motivation

Local log retention behavior is being extended by #3748 and #3830. The user-facing documentation should describe the resulting configuration model and clearly distinguish remote log retention, local log retention, and active-segment rolling. It should also provide safe rollout guidance for clusters upgrading from 0.9 to 1.0.

Documentation scope

  • Document table.log.local-ttl:
    • it controls retention of local log segments;
    • a local segment is eligible for TTL cleanup only after it has been copied to remote storage;
    • document its default value and validation constraints, including its relationship with table.log.ttl;
    • reconcile with the initial documentation changes already included in [server] Support independent local log TTL #3778.
  • Document log.retention.roll-active-segment.enabled:
    • it is a server configuration and is disabled by default;
    • when enabled, a non-empty active segment may be rolled after the table log TTL expires and the high watermark reaches the log end offset;
    • explain that rolling makes the segment eligible for remote upload and subsequent local cleanup.
  • Expand the TTL page to distinguish:
    • table.log.ttl for table/remote log retention;
    • table.log.local-ttl for copied local segments;
    • active-segment rolling controlled by log.retention.roll-active-segment.enabled.
  • Update or regenerate the relevant table and server configuration reference pages.
  • Update the tiered-storage documentation and add cross-links to the TTL page.
  • Update the 1.0 upgrade notes for clusters upgrading from 0.9:
    • keep log.retention.roll-active-segment.enabled disabled during the upgrade; this is also the default value;
    • after all cluster components have been upgraded to 1.0 and the upgrade is complete, enable the option through a dynamic cluster configuration update;
    • explain that enabling the option allows expired active local log segments to be rolled and subsequently cleaned up automatically.

Related issues and pull requests

Acceptance criteria

  • Both configuration keys are searchable in the published documentation.
  • Their defaults, constraints, scope, and interaction are documented.
  • The TTL page explains the local-segment lifecycle from active segment, to rolled/uploaded segment, to local cleanup.
  • The 1.0 upgrade notes document the staged 0.9-to-1.0 rollout: keep log.retention.roll-active-segment.enabled disabled during the upgrade, then enable it with a dynamic cluster configuration update after the upgrade is complete.
  • Existing documentation changes in [server] Support independent local log TTL #3778 are retained and reconciled instead of duplicated.
  • The TTL, configuration-reference, tiered-storage, and upgrade-note pages link to each other where appropriate.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions