Skip to content

Releases: IBM/sarama

Version 1.17.0 (2018-05-30)

30 May 15:18
@bai bai
35324cf
Compare
Choose a tag to compare

New Features:

  • Add support for gzip compression levels (#1044).
  • Add support for Metadata request/response pairs versions v1 to v5 (#1047, #1069).
  • Add versioning to JoinGroup request/response pairs (#1098).
  • Add support for CreatePartitions, DeleteGroups, DeleteRecords request/response pairs (#1065, #1096, #1027).
  • Add Controller() method to Client interface (#1063).

Improvements:

  • ConsumerMetadataReq/Resp has been migrated to FindCoordinatorReq/Resp (#1010).
  • Expose missing protocol parts: msgSet and recordBatch (#1049).
  • Add support for v1 DeleteTopics Request (#1052).
  • Add support for Go 1.10 (#1064).
  • Claim support for Kafka 1.1.0 (#1073).

Bug Fixes:

  • Fix FindCoordinatorResponse.encode to allow nil Coordinator (#1050, #1051).
  • Clear all metadata when we have the latest topic info (#1033).
  • Make PartitionConsumer.Close idempotent (#1092).

Version 1.16.0 (2018-02-12)

12 Feb 19:34
Compare
Choose a tag to compare

New Features:

  • Add support for the Create/Delete Topics request/response pairs (#1007, #1008).
  • Add support for the Describe/Create/Delete ACL request/response pairs (#1009).
  • Add support for the five transaction-related request/response pairs (#1016).

Improvements:

  • Permit setting version on mock producer responses (#999).
  • Add NewMockBrokerListener helper for testing TLS connections (#1019).
  • Changed the default value for Consumer.Fetch.Default from 32KiB to 1MiB which results in much higher throughput in most cases (#1024).
  • Reuse the time.Ticker across fetch requests in the PartitionConsumer to reduce CPU and memory usage when processing many partitions (#1028).
  • Assign relative offsets to messages in the producer to save the brokers a recompression pass (#1002, #1015).

Bug Fixes:

  • Fix producing uncompressed batches with the new protocol format (#1032).
  • Fix consuming compacted topics with the new protocol format (#1005).
  • Fix consuming topics with a mix of protocol formats (#1021).
  • Fix consuming when the broker includes multiple batches in a single response (#1022).
  • Fix detection of PartialTrailingMessage when the partial message was truncated before the magic value indicating its version (#1030).
  • Fix expectation-checking in the mock of SyncProducer.SendMessages (#1035).

Version 1.15.0 (2017-12-08)

08 Dec 15:14
Compare
Choose a tag to compare

New Features:

  • Claim official support for Kafka 1.0, though it did already work (#984).
  • Helper methods for Kafka version numbers to/from strings (#989).
  • Implement CreatePartitions request/response (#985).

Improvements:

  • Add error codes 45-60 (#986).

Bug Fixes:

  • Fix slow consuming for certain Kafka 0.11/1.0 configurations (#982).
  • Correctly determine when a FetchResponse contains the new message format (#990).
  • Fix producing with multiple headers (#996).
  • Fix handling of truncated record batches (#998).
  • Fix leaking metrics when closing brokers (#991).

Version 1.14.0 (2017-11-13)

13 Nov 14:13
Compare
Choose a tag to compare

New Features:

  • Add support for the new Kafka 0.11 record-batch format, including the wire protocol and the necessary behavioural changes in the producer and consumer. Transactions and idempotency are not yet supported, but producing and consuming should work with all the existing bells and whistles (batching, compression, etc) as well as the new custom headers. Thanks to Vlad Hanciuta of Arista Networks for this work. Part of (#901).

Bug Fixes:

  • Fix encoding of ProduceResponse versions in test (#970).
  • Return partial replicas list when we have it (#975).

Version 1.13.0 (2017-10-04)

04 Oct 12:58
Compare
Choose a tag to compare

New Features:

  • Support for FetchRequest version 3 (#905).
  • Permit setting version on mock FetchResponses (#939).
  • Add a configuration option to support storing only minimal metadata for extremely large clusters (#937).
  • Add PartitionOffsetManager.ResetOffset for backtracking tracked offsets (#932).

Improvements:

  • Provide the block-level timestamp when consuming compressed messages (#885).
  • Client.Replicas and Client.InSyncReplicas now respect the order returned by the broker, which can be meaningful (#930).
  • Use a Ticker to reduce consumer timer overhead at the cost of higher variance in the actual timeout (#933).

Bug Fixes:

  • Gracefully handle messages with negative timestamps (#907).
  • Raise a proper error when encountering an unknown message version (#940).

Version 1.12.0 (2017-05-08)

08 May 18:40
Compare
Choose a tag to compare

New Features:

  • Added support for the ApiVersions request and response pair, and Kafka version 0.10.2 (#867). Note that you still need to specify the Kafka version in the Sarama configuration for the time being.
  • Added a Brokers method to the Client which returns the complete set of active brokers (#813).
  • Added an InSyncReplicas method to the Client which returns the set of all in-sync broker IDs for the given partition, now that the Kafka versions for which this was misleading are no longer in our supported set (#872).
  • Added a NewCustomHashPartitioner method which allows constructing a hash partitioner with a custom hash method in case the default (FNV-1a) is not suitable (#837, #841).

Improvements:

  • Recognize more Kafka error codes (#859).

Bug Fixes:

  • Fix an issue where decoding a malformed FetchRequest would not return the correct error (#818).
  • Respect ordering of group protocols in JoinGroupRequests. This fix is transparent if you're using the AddGroupProtocol or AddGroupProtocolMetadata helpers; otherwise you will need to switch from the GroupProtocols field (now deprecated) to use OrderedGroupProtocols (#812).
  • Fix an alignment-related issue with atomics on 32-bit architectures (#859).

Version 1.11.0 (2016-12-20)

20 Dec 18:07
Compare
Choose a tag to compare

New Features:

  • Metrics! Thanks to Sébastien Launay for all his work on this feature
    (#701,
    #746,
    #766).
  • Add support for LZ4 compression
    (#786).
  • Add support for ListOffsetRequest v1 and Kafka 0.10.1
    (#775).
  • Added a HighWaterMarks method to the Consumer which aggregates the
    HighWaterMarkOffset values of its child topic/partitions
    (#769).

Bug Fixes:

  • Fixed producing when using timestamps, compression and Kafka 0.10
    (#759).
  • Added missing decoder methods to DescribeGroups response
    (#756).
  • Fix producer shutdown when Return.Errors is disabled
    (#787).
  • Don't mutate configuration in SyncProducer
    (#790).
  • Fix crash on SASL initialization failure
    (#795).

Version 1.10.1 (2016-08-30)

30 Aug 13:28
Compare
Choose a tag to compare

Bug Fixes:

  • Fix the documentation for HashPartitioner which was incorrect
    (#717).
  • Permit client creation even when it is limited by ACLs
    (#722).
  • Several fixes to the consumer timer optimization code, regressions introduced
    in v1.10.0. Go's timers are finicky
    (#730,
    #733,
    #734).
  • Handle consuming compressed relative offsets with Kafka 0.10
    (#735).

Version 1.10.0 (2016-08-02)

02 Aug 12:50
Compare
Choose a tag to compare

Important: As of Sarama 1.10 it is necessary to tell Sarama the version of
Kafka you are running against (via the config.Version value) in order to use
features that may not be compatible with old Kafka versions. If you don't
specify this value it will default to 0.8.2 (the minimum supported), and trying
to use more recent features (like the offset manager) will fail with an error.

Also: The offset-manager's behaviour has been changed to match the upstream
java consumer (see #705 and #713). If you use the offset-manager, please ensure
that you are committing one greater than the last consumed message offset or else
you may end up consuming duplicate messages.

New Features:

  • Support for Kafka 0.10
    (#672,
    #678,
    #681, and others).
  • Support for configuring the target Kafka version
    (#676).
  • Batch producing support in the SyncProducer
    (#677).
  • Extend producer mock to allow setting expectations on message contents
    (#667).

Improvements:

  • Support nil compressed messages for deleting in compacted topics
    (#634).
  • Pre-allocate decoding errors, greatly reducing heap usage and GC time against
    misbehaving brokers (#690).
  • Re-use consumer expiry timers, removing one allocation per consumed message
    (#707).

Bug Fixes:

  • Actually default the client ID to "sarama" like we say we do
    (#664).
  • Fix a rare issue where Client.Leader could return the wrong error
    (#685).
  • Fix a possible tight loop in the consumer
    (#693).
  • Match upstream's offset-tracking behaviour
    (#705).
  • Report UnknownTopicOrPartition errors from the offset manager
    (#706).
  • Fix possible negative partition value from the HashPartitioner
    (#709).

Version 1.9.0 (2016-05-16)

16 May 18:25
Compare
Choose a tag to compare

New Features:

  • Add support for custom offset manager retention durations
    (#602).
  • Publish low-level mocks to enable testing of third-party producer/consumer
    implementations (#570).
  • Declare support for Golang 1.6
    (#611).
  • Support for SASL plain-text auth
    (#648).

Improvements:

  • Simplified broker locking scheme slightly
    (#604).
  • Documentation cleanup
    (#605,
    #621,
    #654).

Bug Fixes:

  • Fix race condition shutting down the OffsetManager
    (#658).