Releases: zendesk/sarama
Releases · zendesk/sarama
v1.26.2-zd5: Remove idempotent production fix feature flag
Add a functional test to cover idempotent production
v1.26.2-zd4: Add a feature toggle for the idempotent publish fix
Add a feature toggle for the broker fix.
v1.26.2-zd3: Fix "broker received out of order sequence" when brokers die
When the following three conditions are satisfied, the producer code can skip message sequence numbers and cause the broker to complain that the sequences are out of order: * config.Producer.Idempotent is set * The producer loses, and then regains, its connection to a broker * The client code continues to attempt to produce messages whilst the broker is unavailable. For every message the client attempted to send while the broker is unavailable, the transaction manager sequence number will be incremented, however these messages will eventually fail and return an error to the caller. When the broker re-appears, and another message is published, it's sequence number is higher than the last one the broker remembered - the values that were attempted while it was down were never seen. Thus, from the broker's perspective, it's seeing out-of-order sequence numbers. The fix to this has a few parts: * Don't obtain a sequence number from the transaction manager until we're sure we want to try publishing the message * Affix the producer ID and epoch to the message once the sequence is generated * Increment the transaction manager epoch (and reset all sequence numbers to zero) when we permenantly fail to publish a message. That represents a sequence that the broker will never see, so the only safe thing to do is to roll over the epoch number. * Ensure we don't publish message sets that contain messages from multiple transaction manager epochs.
v1.26.2-zd2: Fix session ID thrashing (take 2)
Fix brokers continually allocating new Session IDs KIP-227 says that if a client provides a session ID of 0 and a session epoch of 0 in a FetchRequest, the broker should allocate a new sessionID. Then, the consumer can use this ID to avoid repeating the list of partitions to consume on every call. Sarama is currently sending 0 for both values, but totally ignoring the session ID generated by the broker. This means that the broker is allocating a new session ID on every request, filling its session ID cache and leading to FETCH_SESSION_ID_NOT_FOUND logs being written by the broker because other _actual_ session IDs are evicted. Instead, we should set the epoch to -1, which instructs the broker not to allocate a new session ID.
v1.26.2-zd1: Fix session ID thrashing
Fix brokers continually allocating new Session IDs KIP-227 says that if a client provides a session ID of 0 and a session epoch of 0 in a FetchRequest, the broker should allocate a new sessionID. Then, the consumer can use this ID to avoid repeating the list of partitions to consume on every call. Sarama is currently sending 0 for both values, but totally ignoring the session ID generated by the broker. This means that the broker is allocating a new session ID on every request, filling its session ID cache and leading to FETCH_SESSION_ID_NOT_FOUND logs being written by the broker because other _actual_ session IDs are evicted. Instead, we should set the epoch to -1, which instructs the broker not to allocate a new session ID.