Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.0 #106

Merged
merged 84 commits into from
Apr 23, 2020
Merged

2.0 #106

merged 84 commits into from
Apr 23, 2020

Conversation

mensfeld
Copy link
Member

@mensfeld mensfeld commented Aug 9, 2019

WaterDrop 2.0 based on https://github.com/edenhill/librdkafka instead of ruby-kafka. Provides instantiable API as well as many additional features. Please see README for more details.

poc.rb Outdated
producer.flush_async

producer.buffer_many(Array.new(10) { msg })
producer.flush_sync

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really love this idea with buffer! 💯

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API will slightly change tomorrow but that's not a big change (will ping you).

poc.rb Outdated
@@ -0,0 +1,38 @@
require 'waterdrop'

producer = WaterDrop::Producer.new

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT about adding ability to also put config into constructor of producer? and use #setup too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean this?

producer = WaterDrop::Producer.new do |config|
# trolololo
end

We can do that BUT we need to indicate, that this needs to run after fork.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, something like this. or producer = WaterDrop::Producer.new({ ... }) if block harder in implementation 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

already in the 2.0 branch

producer = WaterDrop::Producer.new do |config|
  config.kafka = {
    'bootstrap.servers' => 'localhost:9092',
    'request.required.acks' => [-1, 1].sample
  }
end

@mensfeld
Copy link
Member Author

@davydovanton could you go through the new README.md and assess whether it is understandable? It lacks the buffering section and instrumentation but I would apprecaite if someone look at other places.

README.md Outdated Show resolved Hide resolved
@mensfeld
Copy link
Member Author

Ok. Time to merge this and move on!

@mensfeld
Copy link
Member Author

@nijikon feel free to remove this branch once we've managed to upgrade our internal references.

@mensfeld mensfeld changed the title 2.0 [DO NOT MERGE] 2.0 Apr 23, 2020
@mensfeld mensfeld deleted the 2.0 branch June 6, 2021 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

6 participants