Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publisher topic generates successfully but message doesn't post. #80

Open
dsalazar-carecloud opened this issue Jan 21, 2015 · 9 comments

Comments

@dsalazar-carecloud
Copy link

Consumer works like a charm but the producer is another story. When attempting to post to a topic the library will generate the topic, if not already there, but the message to be posted never posts. Infact I get the following error.

/Users/foo/.rbenv/versions/2.1.0/gemsets/kafka/gems/poseidon-0.0.5/lib/poseidon/sync_producer.rb:66:in `send_messages': Failed to send all messages: [#<Poseidon::Message:0x007fbb7a9f1ef0 @struct=#<struct Poseidon::Protocol::MessageWithOffsetStruct offset=0, message=#<struct Poseidon::Protocol::MessageStruct magic_type=0, attributes=0, key=nil, value="hello">>, @topic="test">] remaining (RuntimeError)
    from /Users/foo/.rbenv/versions/2.1.0/gemsets/kafka/gems/poseidon-0.0.5/lib/poseidon/producer.rb:163:in `send_messages'
    from producer.rb:6:in `<main>'

My implementation is very simple.

require 'poseidon'
producer = Poseidon::Producer.new(['foo:9092'], 'test_producer')
message_buffer = []
message_buffer << Poseidon::MessageToSend.new('test_topic', 'bar')
producer.send_messages(message_buffer)

Kafka install: kafka_2.9.2-0.8.1.1

Please advise...

@destagia
Copy link

I have the same problem...

@destagia
Copy link

I tried to send message with other languages, Scala, Node.js and Java.
but all of them failed to send message to Kafka on external server.

So I think our problem occurs due to server settings.

@divyesh2506
Copy link

I too have the same problem :P . Please let me know if you find the cause or the solution.

@destagia
Copy link

destagia commented Oct 6, 2015

I solved this. and found poseidon has no problems... ^^;
If you use Kafka with docker-kafka, I can tell you the solution.

@divyesh2506
Copy link

Hey @tagia0212 ,the issue resolved by itself ,once I restarted the kafka server. But, I couldn't find the reason behind the error. Though I do not use docker, but some insight on the issue might help.

@daluu
Copy link

daluu commented Oct 6, 2015

@tagia0212, could you elaborate on the solution for docker-kafka? Just curious to know.

@destagia
Copy link

destagia commented Oct 7, 2015

@daluu
OK, but I know the solution for kafka-docker, not docker-kafka. sorry.

I cloned the repository onto the server, and run docker up -d. When I failed to send topic from Poseidon to Kafka, I have not set config.
So, I rewrite docker-compose.yml.
environment, KAFKA_ADVERTISED_HOST_NAME must be set global IP Address of the server!

After I rerun the docker container, I could send the topics to Kafka.

@rhardih
Copy link

rhardih commented Oct 9, 2015

Not a fix, but at least it would then be possible to capture an explicit error instead of a RuntimeError: #101

@mensfeld
Copy link

this would be useful to merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants