v2.15.0

KIP-932 Queues for Kafka – Now in Preview

• Added a preview implementation of the share consumer (Queues for Kafka,
  KIP-932).
  Members of a share group cooperatively consume from the same partitions with
  per-record acquire/acknowledge semantics and redelivery, providing queue-like
  consumption on top of Kafka.
• New rd_kafka_share_* public API, with a dedicated rd_kafka_share_t handle
  created via rd_kafka_share_consumer_new():
  • Subscription: rd_kafka_share_subscribe(), rd_kafka_share_unsubscribe(),
    rd_kafka_share_subscription().
  • Batch polling: rd_kafka_share_poll() returns an rd_kafka_messages_t
    batch (rd_kafka_messages_count() / rd_kafka_messages_get() /
    rd_kafka_messages_destroy()).
  • Acknowledgement: rd_kafka_share_acknowledge(),
    rd_kafka_share_acknowledge_type(), rd_kafka_share_acknowledge_offset()
    with ACCEPT / RELEASE / REJECT types, and
    rd_kafka_message_delivery_count().
  • Commit: rd_kafka_share_commit_sync(), rd_kafka_share_commit_async() and
    the acknowledgement-commit callback
    (rd_kafka_share_set_acknowledgement_commit_cb()).
  • Lifecycle: rd_kafka_share_consumer_close(),
    rd_kafka_share_consumer_close_queue(), rd_kafka_share_destroy().
• Two acknowledgement modes selected by share.acknowledgement.mode
  (default implicit; explicit requires the application to acknowledge every
  record before the next poll).
• New max.poll.records property (default 500) and adjusted defaults for
  several network properties for share consumers (receive.message.max.bytes,
  connections.max.idle.ms, reconnect.backoff.ms,
  reconnect.backoff.max.ms).
• See the Share consumers (Queues for Kafka) section of
  INTRODUCTION.md, the
  Share consumer section in rdkafka.h, and the
  examples/share_consumer* programs.

Note

The KIP-932
share consumer is currently in Preview and should not be used in
production environments. The public interfaces may change before General
Availability, and known limitations apply (see
INTRODUCTION.md). The
share consumer is single-threaded and not thread-safe by design. It requires
a broker with share groups enabled (generally available in Apache Kafka
4.2.0).

Enhancements

• Add aws_iam option to sasl.oauthbearer.metadata.authentication.type, with a defensive stub that fails when no token-refresh callback is registered.

Fixes

General fixes

• Issues: #5135.
  Fix compilation with CMake when CURL is disabled.
  The OAuthBearer OIDC code included <curl/curl.h> under #ifdef WITH_OAUTHBEARER_OIDC, but
  CMake always defines that macro (to 0 or 1), so CURL was required even when it was turned off.
  Happening since 2.11.0 (#5136).
• Issues: #5282.
  Fix rd_atomic32_set/rd_atomic64_set returning the new value instead of the previous one in CMake builds.
  CMake never defined HAVE_ATOMICS_{32,64}_ATOMIC, so the setters used a non-atomic fallback that
  returned the new value, which prevented the ALL_BROKERS_DOWN event from being raised under CMake.
  Happening since 2.11.1 (#5136).

Consumer fixes

• Issues: #5541.
  Improve error handling in the KIP-848 consumer group protocol:
  • Defer the leave heartbeat until the assignment is revoked, so a member that
    exceeds max.poll.interval.ms rejoins cleanly instead of being rejected
    with a fatal INVALID_REQUEST.
  • Treat GROUP_ID_NOT_FOUND in the ConsumerGroupHeartbeat response as fatal,
    except while the member is already leaving.
  • Surface unexpected permanent broker-level heartbeat errors instead of
    retrying them in a loop; internal transport/timeout codes keep their
    existing retry/reconnect handling.
    Happening since 2.12.0 (#5488)

Checksums

Release asset checksums:

• v2.15.0.zip SHA256 d0655c770f7edc5699855f65b391d1e20dc24fc03ecf2dd680665934dd4d1cc1
• v2.15.0.tar.gz SHA256 259015220cdca708afe838b5aa79ebf1a5fb710fb4179cf918d390aed85d5dbc