Skip to content

v0.11.5
Compare
Choose a tag to compare
@edenhill edenhill released this 19 Jul 09:11
· 1735 commits to master since this release
v0.11.5
44242a4

v0.11.5 is a feature release that adds support for the Kafka Admin API (KIP-4).

Admin API

This release adds support for the Admin API, enabling applications and users to perform administrative Kafka tasks programmatically:

  • Create topics - specifying partition count, replication factor and topic configuration.
  • Delete topics - delete topics in cluster.
  • Create partitions - extend a topic with additional partitions.
  • Alter configuration - set, modify or delete configuration for any Kafka resource (topic, broker, ..).
  • Describe configuration - view configuration for any Kafka resource.

The API closely follows the Java Admin API:

https://github.com/edenhill/librdkafka/blob/master/src/rdkafka.h#L4495

New and updated configuration

  • Added compresion.level configuration option, which allows fine-tuning of gzip and LZ4 comression level (@erkoln)
  • Implement ssl.curves.list and ssl.sigalgs.list configuration settings (@jvgutierrez)
  • Changed queue.buffering.backpressure.threshold default (#1848)

Enhancements

  • Callback based event notifications (@fede1024)
  • Event callbacks may now optionally be triggered from a dedicated librdkafka background thread, see rd_kafka_conf_set_background_event_cb.
  • Log the value that couldn't be found for flag configuration options (@ankon)
  • Add support for rd_kafka_conf_set_events(conf, ..EVENT_ERROR) to allow generic errors to be retrieved as events.
  • Avoid allocating BIOs and copying for base64 processing (@agl)
  • Don't log connection close for idle connections (regardless of log.connection.close)
  • Improve latency by using high-precision QPC clock on Windows
  • Added make uninstall
  • Minor documentation updates from replies to #1794 (@briot)
  • Added rd_kafka_controllerid() to return the current controller.
  • INTRODUCTION.md: add chapter on latency measurement.
  • Add relative hyperlinks to table of contents in INTRODUCTION.md (#1791, @stanislavkozlovski)
  • Improved statistics:
    • Added Hdr Histograms for all windowed stats (rtt, int_latency, throttle) #1798
    • Added top-level totals for broker receive and transmit metrics.
    • Added batchcnt, batchsize histograms to consumer.
    • Added outbuf_latency histograms.
    • STATISTICS.md moved from wiki to source tree.

Fixes

  • Fixed murmur2 partitioner to make it compatible with java version (#1816, @lins05)
  • Fix pause/resume: next_offset was not properly initialized
  • Fix a segment fault error in rdkafka_buf with zero-length string (@sunny1988)
  • Set error string length in rkmessage.len on error (#1851)
  • Don't let metadata ERR_UNKNOWN set topic state to non-existent.
  • The app_offset metric is now reset to INVALID when the fetcher is stopped.
  • consumer_lag is now calculated as consumer_lag = hi_wmark_offset - MAX(app_offset, committed_offset), which makes it correct after a reassignment but before new messages have been consumed (#1878)
  • socket.nagle.disable=true was never applied on non-Windows platforms (#1838)
  • Update interface compile definitions for Windows using CMake (#1800, @raulbocanegra)
  • Fix queue hang when queue_destroy() is called on rdkafka-owned queue (#1792)
  • Fix hang on unclean termination when there are outstanding requests.
  • Metadata: fix crash when topic is in transitionary state
  • Metadata: sort topic partition list
  • rdkafka_example emitted bogus produce errors
  • Increase BROKERS_MAX to 10K and PARTITIONS_MAX to 100K
  • Proper log message on SSL connection close
  • Missing return on error causes use-after-free in SASL code (@sidhpurwala-huzaifa)
  • Fix configure --pkg-config-path=... (#1797, @xbolshe)
  • Fix -fsanitize=undefined warning for overflowed OP switches (#1789)
Assets 2