Rename producer/multi-producer methods to push
Send is problematic in ruby because it is defined on all objects as a way to dynamically send method calls (messages). You do not want to have any methods in your interface that define send. This would resolve #18
*kafka-subscribe* and *kafka-publish* are command line programs to help in the task of debugging a kafka installation.
…ession (Snappy and GZip) to the Producer and MultiProducer. To enable snappy compression, include the following in your Gemfile (This branch is necessary until a pending pull request is accepted): gem "snappy", "0.0.4", :git => "git://github.com/watersofoblivion/snappy.git", :branch => "snappy-streaming"
Don't specify rubygems_version - set when package created. Specify files in lib/spec with globs.
Like Producer, but doesn't have a fixed topic and partition. 'send' sends a single message or a set of messages to the same topic/partition using the 'produce' request type. 'multi_send' sends groups of messages across partitions/topics using the 'multiproduce' request type.
Provides methods for encoding messages, produce requests, and multiproduce requests.
A ProducerRequest instance is a very simple wrapper around a topic, partition, and set of messages.
Add consumer support for transparent gzip compression
Fix a warning about top-level constant reference while running the specs
The Scala Kafka producer implementation has support for gzip-compressing individual messages or sets of consecutive messages, and the compression is transparent to the Scala Kafka consumer implementation. It's very convenient (just a matter of setting compression.codec=1 in the producer config). The compression codec of a message is indicated in the bottom two bits of the 'attributes' byte of messages with magic == 1. This means it's possible to add transparent compression support to consumers without any configuration. This commit adds compression support to the Ruby consumer. Because a compressed message may actually contain more than one message inside it (this makes compression more effective by grouping lots of small messages into one big message), I had to move some of the parsing logic from Kafka::Consumer to Kafka::Message.
Support messages with magic version 1
One #unpack call, with a constant format specifier, yields about 50% speedup in message decoding.