Skip to content

Releases: confluentinc/librdkafka

v2.4.0

07 May 12:47
a4bf487
Compare
Choose a tag to compare

librdkafka v2.4.0 is a feature release:

  • KIP-848: The Next Generation of the Consumer Rebalance Protocol.
    Early Access: This should be used only for evaluation and must not be used in production. Features and contract of this KIP might change in future (#4610).
  • KIP-467: Augment ProduceResponse error messaging for specific culprit records (#4583).
  • KIP-516
    Continue partial implementation by adding a metadata cache by topic id
    and updating the topic id corresponding to the partition name (#4676)
  • Upgrade OpenSSL to v3.0.12 (while building from source) with various security fixes,
    check the release notes.
  • Integration tests can be started in KRaft mode and run against any
    GitHub Kafka branch other than the released versions.
  • Fix pipeline inclusion of static binaries (#4666)
  • Fix to main loop timeout calculation leading to a tight loop for a
    max period of 1 ms (#4671).
  • Fixed a bug causing duplicate message consumption from a stale
    fetch start offset in some particular cases (#4636)
  • Fix to metadata cache expiration on full metadata refresh (#4677).
  • Fix for a wrong error returned on full metadata refresh before joining
    a consumer group (#4678).
  • Fix to metadata refresh interruption (#4679).
  • Fix for an undesired partition migration with stale leader epoch (#4680).
  • Fix hang in cooperative consumer mode if an assignment is processed
    while closing the consumer (#4528).

Upgrade considerations

  • With KIP 467, INVALID_MSG (Java: CorruptRecordExpection) will
    be retried automatically. INVALID_RECORD (Java: InvalidRecordException) instead
    is not retriable and will be set only to the records that caused the
    error. Rest of records in the batch will fail with the new error code
    _INVALID_DIFFERENT_RECORD (Java: KafkaException) and can be retried manually,
    depending on the application logic (#4583).

Early Access

KIP-848: The Next Generation of the Consumer Rebalance Protocol

  • With this new protocol the role of the Group Leader (a member) is removed and
    the assignment is calculated by the Group Coordinator (a broker) and sent
    to each member through heartbeats.

    The feature is still not production-ready.
    It's possible to try it in a non-production enviroment.

    A guide is available
    with considerations and steps to follow to test it (#4610).

Fixes

General fixes

  • Issues: confluentinc/confluent-kafka-go#981.
    In librdkafka release pipeline a static build containing libsasl2
    could be chosen instead of the alternative one without it.
    That caused the libsasl2 dependency to be required in confluent-kafka-go
    v2.1.0-linux-musl-arm64 and v2.3.0-linux-musl-arm64.
    Solved by correctly excluding the binary configured with that library,
    when targeting a static build.
    Happening since v2.0.2, with specified platforms,
    when using static binaries (#4666).
  • Issues: #4684.
    When the main thread loop was awakened less than 1 ms
    before the expiration of a timeout, it was serving with a zero timeout,
    leading to increased CPU usage until the timeout was reached.
    Happening since 1.x.
  • Issues: #4685.
    Metadata cache was cleared on full metadata refresh, leading to unnecessary
    refreshes and occasional UNKNOWN_TOPIC_OR_PART errors. Solved by updating
    cache for existing or hinted entries instead of clearing them.
    Happening since 2.1.0 (#4677).
  • Issues: #4589.
    A metadata call before member joins consumer group,
    could lead to an UNKNOWN_TOPIC_OR_PART error. Solved by updating
    the consumer group following a metadata refresh only in safe states.
    Happening since 2.1.0 (#4678).
  • Issues: #4577.
    Metadata refreshes without partition leader change could lead to a loop of
    metadata calls at fixed intervals. Solved by stopping metadata refresh when
    all existing metadata is non-stale. Happening since 2.3.0 (#4679).
  • Issues: #4687.
    A partition migration could happen, using stale metadata, when the partition
    was undergoing a validation and being retried because of an error.
    Solved by doing a partition migration only with a non-stale leader epoch.
    Happening since 2.1.0 (#4680).

Consumer fixes

  • Issues: #4686.
    In case of subscription change with a consumer using the cooperative assignor
    it could resume fetching from a previous position.
    That could also happen if resuming a partition that wasn't paused.
    Fixed by ensuring that a resume operation is completely a no-op when
    the partition isn't paused.
    Happening since 1.x (#4636).
  • Issues: #4527.
    While using the cooperative assignor, given an assignment is received while closing the consumer
    it's possible that it gets stuck in state WAIT_ASSIGN_CALL, while the method is converted to
    a full unassign. Solved by changing state from WAIT_ASSIGN_CALL to WAIT_UNASSIGN_CALL
    while doing this conversion.
    Happening since 1.x (#4528).

Checksums

Release asset checksums:

  • v2.4.0.zip SHA256 24b30d394fc6ce5535eaa3c356ed9ed9ae4a6c9b4fc9159c322a776786d5dd15
  • v2.4.0.tar.gz SHA256 d645e47d961db47f1ead29652606a502bdd2a880c85c1e060e94eea040f1a19a

v2.3.0

25 Oct 15:18
95a542c
Compare
Choose a tag to compare

librdkafka v2.3.0 is a feature release:

  • KIP-516
    Partial support of topic identifiers. Topic identifiers in metadata response
    available through the new rd_kafka_DescribeTopics function (#4300, #4451).
  • KIP-117 Add support for AdminAPI DescribeCluster() and DescribeTopics()
    (#4240, @jainruchir).
  • KIP-430:
    Return authorized operations in Describe Responses.
    (#4240, @jainruchir).
  • KIP-580: Added Exponential Backoff mechanism for
    retriable requests with retry.backoff.ms as minimum backoff and retry.backoff.max.ms as the
    maximum backoff, with 20% jitter (#4422).
  • KIP-396: completed the implementation with
    the addition of ListOffsets (#4225).
  • Fixed ListConsumerGroupOffsets not fetching offsets for all the topics in a group with Apache Kafka version below 2.4.0.
  • Add missing destroy that leads to leaking partition structure memory when there
    are partition leader changes and a stale leader epoch is received (#4429).
  • Fix a segmentation fault when closing a consumer using the
    cooperative-sticky assignor before the first assignment (#4381).
  • Fix for insufficient buffer allocation when allocating rack information (@wolfchimneyrock, #4449).
  • Fix for infinite loop of OffsetForLeaderEpoch requests on quick leader changes. (#4433).
  • Fix to add leader epoch to control messages, to make sure they're stored
    for committing even without a subsequent fetch message (#4434).
  • Fix for stored offsets not being committed if they lacked the leader epoch (#4442).
  • Upgrade OpenSSL to v3.0.11 (while building from source) with various security fixes,
    check the release notes
    (#4454, started by @migarc1).
  • Fix to ensure permanent errors during offset validation continue being retried and
    don't cause an offset reset (#4447).
  • Fix to ensure max.poll.interval.ms is reset when rd_kafka_poll is called with
    consume_cb (#4431).
  • Fix for idempotent producer fatal errors, triggered after a possibly persisted message state (#4438).
  • Fix rd_kafka_query_watermark_offsets continuing beyond timeout expiry (#4460).
  • Fix rd_kafka_query_watermark_offsets not refreshing the partition leader
    after a leader change and subsequent NOT_LEADER_OR_FOLLOWER error (#4225).

Upgrade considerations

  • retry.backoff.ms:
    If it is set greater than retry.backoff.max.ms which has the default value of 1000 ms then it is assumes the value of retry.backoff.max.ms.
    To change this behaviour make sure that retry.backoff.ms is always less than retry.backoff.max.ms.
    If equal then the backoff will be linear instead of exponential.

  • topic.metadata.refresh.fast.interval.ms:
    If it is set greater than retry.backoff.max.ms which has the default value of 1000 ms then it is assumes the value of retry.backoff.max.ms.
    To change this behaviour make sure that topic.metadata.refresh.fast.interval.ms is always less than retry.backoff.max.ms.
    If equal then the backoff will be linear instead of exponential.

Fixes

General fixes

  • An assertion failed with insufficient buffer size when allocating
    rack information on 32bit architectures.
    Solved by aligning all allocations to the maximum allowed word size (#4449).
  • The timeout for rd_kafka_query_watermark_offsets was not enforced after
    making the necessary ListOffsets requests, and thus, it never timed out in
    case of broker/network issues. Fixed by setting an absolute timeout (#4460).

Idempotent producer fixes

  • After a possibly persisted error, such as a disconnection or a timeout, next expected sequence
    used to increase, leading to a fatal error if the message wasn't persisted and
    the second one in queue failed with an OUT_OF_ORDER_SEQUENCE_NUMBER.
    The error could contain the message "sequence desynchronization" with
    just one possibly persisted error or "rewound sequence number" in case of
    multiple errored messages.
    Solved by treating the possible persisted message as not persisted,
    and expecting a DUPLICATE_SEQUENCE_NUMBER error in case it was or
    NO_ERROR in case it wasn't, in both cases the message will be considered
    delivered (#4438).

Consumer fixes

  • Stored offsets were excluded from the commit if the leader epoch was
    less than committed epoch, as it's possible if leader epoch is the default -1.
    This didn't happen in Python, Go and .NET bindings when stored position was
    taken from the message.
    Solved by checking only that the stored offset is greater
    than committed one, if either stored or committed leader epoch is -1 (#4442).
  • If an OffsetForLeaderEpoch request was being retried, and the leader changed
    while the retry was in-flight, an infinite loop of requests was triggered,
    because we weren't updating the leader epoch correctly.
    Fixed by updating the leader epoch before sending the request (#4433).
  • During offset validation a permanent error like host resolution failure
    would cause an offset reset.
    This isn't what's expected or what the Java implementation does.
    Solved by retrying even in case of permanent errors (#4447).
  • If using rd_kafka_poll_set_consumer, along with a consume callback, and then
    calling rd_kafka_poll to service the callbacks, would not reset
    max.poll.interval.ms. This was because we were only checking rk_rep for
    consumer messages, while the method to service the queue internally also
    services the queue forwarded to from rk_rep, which is rkcg_q.
    Solved by moving the max.poll.interval.ms check into rd_kafka_q_serve (#4431).
  • After a leader change a rd_kafka_query_watermark_offsets call would continue
    trying to call ListOffsets on the old leader, if the topic wasn't included in
    the subscription set, so it started querying the new leader only after
    topic.metadata.refresh.interval.ms (#4225).

Checksums

Release asset checksums:

  • v2.3.0.zip SHA256 15e77455811b3e5d869d6f97ce765b634c7583da188792e2930a2098728e932b
  • v2.3.0.tar.gz SHA256 2d49c35c77eeb3d42fa61c43757fcbb6a206daa560247154e60642bcdcc14d12

v2.2.0

12 Jul 09:58
e75de5b
Compare
Choose a tag to compare

librdkafka v2.2.0 is a feature release:

  • Fix a segmentation fault when subscribing to non-existent topics and
    using the consume batch functions (#4273).
  • Store offset commit metadata in rd_kafka_offsets_store (@mathispesch, #4084).
  • Fix a bug that happens when skipping tags, causing buffer underflow in
    MetadataResponse (#4278).
  • Fix a bug where topic leader is not refreshed in the same metadata call even if the leader is
    present.
  • KIP-881:
    Add support for rack-aware partition assignment for consumers
    (#4184, #4291, #4252).
  • Fix several bugs with sticky assignor in case of partition ownership
    changing between members of the consumer group (#4252).
  • KIP-368:
    Allow SASL Connections to Periodically Re-Authenticate
    (#4301, started by @vctoriawu).
  • Avoid treating an OpenSSL error as a permanent error and treat unclean SSL
    closes as normal ones (#4294).
  • Added fetch.queue.backoff.ms to the consumer to control how long
    the consumer backs off next fetch attempt. (@bitemyapp, @edenhill, #2879)
  • KIP-235:
    Add DNS alias support for secured connection (#4292).
  • KIP-339:
    IncrementalAlterConfigs API (started by @PrasanthV454, #4110).
  • KIP-554: Add Broker-side SCRAM Config API (#4241).

Enhancements

  • Added fetch.queue.backoff.ms to the consumer to control how long
    the consumer backs off next fetch attempt. When the pre-fetch queue
    has exceeded its queuing thresholds: queued.min.messages and
    queued.max.messages.kbytes it backs off for 1 seconds.
    If those parameters have to be set too high to hold 1 s of data,
    this new parameter allows to back off the fetch earlier, reducing memory
    requirements.

Fixes

General fixes

  • Fix a bug that happens when skipping tags, causing buffer underflow in
    MetadataResponse. This is triggered since RPC version 9 (v2.1.0),
    when using Confluent Platform, only when racks are set,
    observers are activated and there is more than one partition.
    Fixed by skipping the correct amount of bytes when tags are received.
  • Avoid treating an OpenSSL error as a permanent error and treat unclean SSL
    closes as normal ones. When SSL connections are closed without close_notify,
    in OpenSSL 3.x a new type of error is set and it was interpreted as permanent
    in librdkafka. It can cause a different issue depending on the RPC.
    If received when waiting for OffsetForLeaderEpoch response, it triggers
    an offset reset following the configured policy.
    Solved by treating SSL errors as transport errors and
    by setting an OpenSSL flag that allows to treat unclean SSL closes as normal
    ones. These types of errors can happen it the other side doesn't support close_notify or if there's a TCP connection reset.

Consumer fixes

  • In case of multiple owners of a partition with different generations, the
    sticky assignor would pick the earliest (lowest generation) member as the
    current owner, which would lead to stickiness violations. Fixed by
    choosing the latest (highest generation) member.
  • In case where the same partition is owned by two members with the same
    generation, it indicates an issue. The sticky assignor had some code to
    handle this, but it was non-functional, and did not have parity with the
    Java assignor. Fixed by invalidating any such partition from the current
    assignment completely.

Checksums

Release asset checksums:

  • v2.2.0.zip SHA256 e9a99476dd326089ce986afd3a5b069ef8b93dbb845bc5157b3d94894de53567
  • v2.2.0.tar.gz SHA256 af9a820cbecbc64115629471df7c7cecd40403b6c34bfdbb9223152677a47226

v2.1.1

02 May 22:11
c282ba2
Compare
Choose a tag to compare

librdkafka v2.1.1 is a maintenance release:

  • Avoid duplicate messages when a fetch response is received
    in the middle of an offset validation request (#4261).
  • Fix segmentation fault when subscribing to a non-existent topic and
    calling rd_kafka_message_leader_epoch() on the polled rkmessage (#4245).
  • Fix a segmentation fault when fetching from follower and the partition lease
    expires while waiting for the result of a list offsets operation (#4254).
  • Fix documentation for the admin request timeout, incorrectly stating -1 for infinite
    timeout. That timeout can't be infinite.
  • Fix CMake pkg-config cURL require and use
    pkg-config Requires.private field (@FantasqueX, @stertingen, #4180).
  • Fixes certain cases where polling would not keep the consumer
    in the group or make it rejoin it (#4256).
  • Fix to the C++ set_leader_epoch method of TopicPartitionImpl,
    that wasn't storing the passed value (@pavel-pimenov, #4267).

Fixes

Consumer fixes

  • Duplicate messages can be emitted when a fetch response is received
    in the middle of an offset validation request. Solved by avoiding
    a restart from last application offset when offset validation succeeds.
  • When fetching from follower, if the partition lease expires after 5 minutes,
    and a list offsets operation was requested to retrieve the earliest
    or latest offset, it resulted in segmentation fault. This was fixed by
    allowing threads different from the main one to call
    the rd_kafka_toppar_set_fetch_state function, given they hold
    the lock on the rktp.
  • In v2.1.0, a bug was fixed which caused polling any queue to reset the
    max.poll.interval.ms. Only certain functions were made to reset the timer,
    but it is possible for the user to obtain the queue with messages from
    the broker, skipping these functions. This was fixed by encoding information
    in a queue itself, that, whether polling, resets the timer.

Checksums

Release asset checksums:

  • v2.1.1.zip SHA256 3b8a59f71e22a8070e0ae7a6b7ad7e90d39da8fddc41ce6c5d596ee7f5a4be4b
  • v2.1.1.tar.gz SHA256 7be1fc37ab10ebdc037d5c5a9b35b48931edafffae054b488faaff99e60e0108

v2.1.0

06 Apr 09:07
2c6dd0a
Compare
Choose a tag to compare

librdkafka v2.1.0 is a feature release:

  • KIP-320
    Allow fetchers to detect and handle log truncation (#4122).
  • Fix a reference count issue blocking the consumer from closing (#4187).
  • Fix a protocol issue with ListGroups API, where an extra
    field was appended for API Versions greater than or equal to 3 (#4207).
  • Fix an issue with max.poll.interval.ms, where polling any queue would cause
    the timeout to be reset (#4176).
  • Fix seek partition timeout, was one thousand times lower than the passed
    value (#4230).
  • Fix multiple inconsistent behaviour in batch APIs during pause or resume operations (#4208).
    See Consumer fixes section below for more information.
  • Update lz4.c from upstream. Fixes CVE-2021-3520
    (by @filimonov, #4232).
  • Upgrade OpenSSL to v3.0.8 with various security fixes,
    check the release notes (#4215).

Enhancements

  • Added rd_kafka_topic_partition_get_leader_epoch() (and set..()).
  • Added partition leader epoch APIs:
    • rd_kafka_topic_partition_get_leader_epoch() (and set..())
    • rd_kafka_message_leader_epoch()
    • rd_kafka_*assign() and rd_kafka_seek_partitions() now supports
      partitions with a leader epoch set.
    • rd_kafka_offsets_for_times() will return per-partition leader-epochs.
    • leader_epoch, stored_leader_epoch, and committed_leader_epoch
      added to per-partition statistics.

Fixes

OpenSSL fixes

  • Fixed OpenSSL static build not able to use external modules like FIPS
    provider module.

Consumer fixes

  • A reference count issue was blocking the consumer from closing.
    The problem would happen when a partition is lost, because forcibly
    unassigned from the consumer or if the corresponding topic is deleted.
  • When using rd_kafka_seek_partitions, the remaining timeout was
    converted from microseconds to milliseconds but the expected unit
    for that parameter is microseconds.
  • Fixed known issues related to Batch Consume APIs mentioned in v2.0.0
    release notes.
  • Fixed rd_kafka_consume_batch() and rd_kafka_consume_batch_queue()
    intermittently updating app_offset and store_offset incorrectly when
    pause or resume was being used for a partition.
  • Fixed rd_kafka_consume_batch() and rd_kafka_consume_batch_queue()
    intermittently skipping offsets when pause or resume was being
    used for a partition.

Known Issues

Consume Batch API

  • When rd_kafka_consume_batch() and rd_kafka_consume_batch_queue() APIs are used with
    any of the seek, pause, resume or rebalancing operation, on_consume
    interceptors might be called incorrectly (maybe multiple times) for not consumed messages.

Consume API

  • Duplicate messages can be emitted when a fetch response is received
    in the middle of an offset validation request.
  • Segmentation fault when subscribing to a non-existent topic and
    calling rd_kafka_message_leader_epoch() on the polled rkmessage.

Checksums

Release asset checksums:

  • v2.1.0.zip SHA256 2fe898f9f5e2b287d26c5f929c600e2772403a594a691e0560a2a1f2706edf57
  • v2.1.0.tar.gz SHA256 d8e76c4b1cde99e283a19868feaaff5778aa5c6f35790036c5ef44bc5b5187aa

v2.0.2

20 Jan 22:34
292d2a6
Compare
Choose a tag to compare

librdkafka v2.0.2 is a bugfix release:

  • Fix OpenSSL version in Win32 nuget package (#4152).

Checksums

Release asset checksums:

  • v2.0.2.zip SHA256 87010c722111539dc3c258a6be0c03b2d6d4a607168b65992eb0076c647e4e9d
  • v2.0.2.tar.gz SHA256 f321bcb1e015a34114c83cf1aa7b99ee260236aab096b85c003170c90a47ca9d

v2.0.1

19 Jan 11:51
96a70a7
Compare
Choose a tag to compare

librdkafka v2.0.1 is a bugfix release:

  • Fixed nuget package for Linux ARM64 release (#4150).

Checksums

Release asset checksums:

  • v2.0.1.zip SHA256 7121df3fad1f72ea1c42dcc4e5367337207a75966216c63e58222c6433c528e0
  • v2.0.1.tar.gz SHA256 3670f8d522e77f79f9d09a22387297ab58d1156b22de12ef96e58b7d57fca139

v2.0.0

18 Jan 14:02
6a66f95
Compare
Choose a tag to compare

librdkafka v2.0.0 is a feature release:

  • KIP-88
    OffsetFetch Protocol Update (#3995).
  • KIP-222
    Add Consumer Group operations to Admin API (started by @lesterfan, #3995).
  • KIP-518
    Allow listing consumer groups per state (#3995).
  • KIP-396
    Partially implemented: support for AlterConsumerGroupOffsets
    (started by @lesterfan, #3995).
  • OpenSSL 3.0.x support - the maximum bundled OpenSSL version is now 3.0.7 (previously 1.1.1q).
  • Fixes to the transactional and idempotent producer.

Upgrade considerations

OpenSSL 3.0.x

OpenSSL default ciphers

The introduction of OpenSSL 3.0.x in the self-contained librdkafka bundles
changes the default set of available ciphers, in particular all obsolete
or insecure ciphers and algorithms as listed in the
OpenSSL legacy
manual page are now disabled by default.

WARNING: These ciphers are disabled for security reasons and it is
highly recommended NOT to use them.

Should you need to use any of these old ciphers you'll need to explicitly
enable the legacy provider by configuring ssl.providers=default,legacy
on the librdkafka client.

OpenSSL engines and providers

OpenSSL 3.0.x deprecates the use of engines, which is being replaced by
providers. As such librdkafka will emit a deprecation warning if
ssl.engine.location is configured.

OpenSSL providers may be configured with the new ssl.providers
configuration property.

Broker TLS certificate hostname verification

The default value for ssl.endpoint.identification.algorithm has been
changed from none (no hostname verification) to https, which enables
broker hostname verification (to counter man-in-the-middle
impersonation attacks) by default.

To restore the previous behaviour, set ssl.endpoint.identification.algorithm to none.

Known Issues

Poor Consumer batch API messaging guarantees

The Consumer Batch APIs rd_kafka_consume_batch() and rd_kafka_consume_batch_queue()
are not thread safe if rkmessages_size is greater than 1 and any of the seek,
pause, resume or rebalancing operation is performed in parallel with any of
the above APIs. Some of the messages might be lost, or erroneously returned to the
application, in the above scenario.

It is strongly recommended to use the Consumer Batch APIs and the mentioned
operations in sequential order in order to get consistent result.

For rebalancing operation to work in sequencial manner, please set rebalance_cb
configuration property (refer examples/rdkafka_complex_consumer_example.c for the help with the usage) for the consumer.

Enhancements

  • Self-contained static libraries can now be built on Linux arm64 (#4005).
  • Updated to zlib 1.2.13, zstd 1.5.2, and curl 7.86.0 in self-contained
    librdkafka bundles.
  • Added on_broker_state_change() interceptor
  • The C++ API no longer returns strings by const value, which enables better move optimization in callers.
  • Added rd_kafka_sasl_set_credentials() API to update SASL credentials.
  • Setting allow.auto.create.topics will no longer give a warning if used by a producer, since that is an expected use case.
    Improvement in documentation for this property.
  • Added a resolve_cb configuration setting that permits using custom DNS resolution logic.
  • Added rd_kafka_mock_broker_error_stack_cnt().
  • The librdkafka.redist NuGet package has been updated to have fewer external
    dependencies for its bundled librdkafka builds, as everything but cyrus-sasl
    is now built-in. There are bundled builds with and without linking to
    cyrus-sasl for maximum compatibility.
  • Admin API DescribeGroups() now provides the group instance id
    for static members KIP-345 (#3995).

Fixes

General fixes

  • Windows: couldn't read a PKCS#12 keystore correctly because binary mode
    wasn't explicitly set and Windows defaults to text mode.
  • Fixed memory leak when loading SSL certificates (@Mekk, #3930)
  • Load all CA certificates from ssl.ca.pem, not just the first one.
  • Each HTTP request made when using OAUTHBEARER OIDC would leak a small
    amount of memory.

Transactional producer fixes

  • When a PID epoch bump is requested and the producer is waiting
    to reconnect to the transaction coordinator, a failure in a find coordinator
    request could cause an assert to fail. This is fixed by retrying when the
    coordinator is known (#4020).
  • Transactional APIs (except send_offsets_for_transaction()) that
    timeout due to low timeout_ms may now be resumed by calling the same API
    again, as the operation continues in the background.
  • For fatal idempotent producer errors that may be recovered by bumping the
    epoch the current transaction must first be aborted prior to the epoch bump.
    This is now handled correctly, which fixes issues seen with fenced
    transactional producers on fatal idempotency errors.
  • Timeouts for EndTxn requests (transaction commits and aborts) are now
    automatically retried and the error raised to the application is also
    a retriable error.
  • TxnOffsetCommitRequests were retried immediately upon temporary errors in
    send_offsets_to_transactions(), causing excessive network requests.
    These retries are now delayed 500ms.
  • If init_transactions() is called with an infinite timeout (-1),
    the timeout will be limited to 2 * transaction.timeout.ms.
    The application may retry and resume the call if a retriable error is
    returned.

Consumer fixes

  • Back-off and retry JoinGroup request if coordinator load is in progress.
  • Fix rd_kafka_consume_batch() and rd_kafka_consume_batch_queue() skipping
    other partitions' offsets intermittently when seek, pause, resume
    or rebalancing is used for a partition.
  • Fix rd_kafka_consume_batch() and rd_kafka_consume_batch_queue()
    intermittently returing incorrect partitions' messages if rebalancing
    happens during these operations.

Checksums

Release asset checksums:

  • v2.0.0.zip SHA256 9d8a8be30ed09daf6c560f402e91db22fcaea11cac18a0d3c0afdbf884df1d4e
  • v2.0.0.tar.gz SHA256 f75de3545b3c6cc027306e2df0371aefe1bb8f86d4ec612ed4ebf7bfb2f817cd

v1.9.2

01 Aug 21:19
Compare
Choose a tag to compare

librdkafka v1.9.2 is a maintenance release:

  • The SASL OAUTHBEAR OIDC POST field was sometimes truncated by one byte (#3192).
  • The bundled version of OpenSSL has been upgraded to version 1.1.1q for non-Windows builds. Windows builds remain on OpenSSL 1.1.1n for the time being.
  • The bundled version of Curl has been upgraded to version 7.84.0.

Checksums

Release asset checksums:

  • v1.9.2.zip SHA256 4ecb0a3103022a7cab308e9fecd88237150901fa29980c99344218a84f497b86
  • v1.9.2.tar.gz SHA256 3fba157a9f80a0889c982acdd44608be8a46142270a389008b22d921be1198ad

v1.9.1

06 Jul 22:27
Compare
Choose a tag to compare

librdkafka v1.9.1

librdkafka v1.9.1 is a maintenance release:

  • The librdkafka.redist NuGet package now contains OSX M1/arm64 builds.
  • Self-contained static libraries can now be built on OSX M1 too, thanks to
    disabling curl's configure runtime check.

Checksums

Release asset checksums:

  • v1.9.1.zip SHA256 d3fc2e0bc00c3df2c37c5389c206912842cca3f97dd91a7a97bc0f4fc69f94ce
  • v1.9.1.tar.gz SHA256 3a54cf375218977b7af4716ed9738378e37fe400a6c5ddb9d622354ca31fdc79