Skip to content

fix(wal): ensure WAL offset monotonicity under S3 backpressure #3033

@Chandi977

Description

@Chandi977

Bug Summary

Under S3 slowdown/backpressure, multiple concurrent calls to DefaultWriter.uploadBulk0() can reuse the same baseOffset, causing duplicate offsets. This violates WAL monotonicity guarantees and breaks Kafka-compatible behavior.

Steps to Reproduce

  1. Pause MinIO (docker pause)
  2. Run two high-throughput producers
  3. Observe overlapping offsets in WAL logs

Expected

Offsets must increase strictly and never repeat.

Actual

Concurrent bulks reuse the same start offset under backpressure.

Maintainers: Please apply the labels bug, area/wal, severity/high.

I am preparing a PR with a fix + concurrency test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions