Skip to content

Commit

Permalink
Handle partial messages at the end of a message set
Browse files Browse the repository at this point in the history
As an optimization, Kafka brokers are allowed to include partial
messages at the end of a message set -- these should be ignored by
clients.
  • Loading branch information
dasch committed Mar 8, 2016
1 parent caa5976 commit c933344
Showing 1 changed file with 11 additions and 7 deletions.
18 changes: 11 additions & 7 deletions lib/kafka/protocol/message_set.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,17 @@ def self.decode(decoder)
fetched_messages = []

until decoder.eof?
message = Message.decode(decoder)

if message.compressed?
wrapped_message_set = message.decompress
fetched_messages.concat(wrapped_message_set.messages)
else
fetched_messages << message
begin
message = Message.decode(decoder)

if message.compressed?
wrapped_message_set = message.decompress
fetched_messages.concat(wrapped_message_set.messages)
else
fetched_messages << message
end
rescue EOFError
# We tried to decode a partial message; just skip it.
end
end

Expand Down

0 comments on commit c933344

Please sign in to comment.