Consumer for Kafka using brod and elixir (production ready)
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config move to brod Jun 20, 2017
lib move to brod Jun 20, 2017
test move to brod Jun 20, 2017
.gitignore init commit Aug 11, 2016
LICENSE Initial commit Aug 11, 2016
README.md move to brod Jun 20, 2017
mix.exs add description and package to project Jun 20, 2017
mix.lock move to brod Jun 20, 2017

README.md

Kafka consumer

Scalable consumer for Kafka using brod.

Installation

If available in Hex, the package can be installed as:

  • Add kafka_consumer to your list of dependencies in mix.exs:
def deps do
  [{:kafka_consumer, "~> 2.0"}]
end
  • Ensure kafka_consumer is started before your application:
def application do
  [applications: [:kafka_consumer]]
end

Usage

  • Set config or pass default attributes
config :brod,
  clients: [
    kafka_client: [
      endpoints: [{'localhost', 9092}],
      auto_start_producers: true
    ]
  ]
  • Write your own Consumer. Functions handle_async/1 and handle_sync/2 is overridable
defmodule KafkaConsumer.ExampleConsumer do
  use KafkaConsumer.Consumer

  def handle_async(_message) do
    :ok
  end
end
  • Set event handlers in config.
config :kafka_consumer,
  consumers: [
    [client: :kafka_client,
     group_id: "messaging",
     topics: ["message-events", "system-events"],
     callback: KafkaConsumer.ExampleConsumer,
     callback_args: []]
  ]
  • Define KafkaConsumer.Supervisor as child supervisor of your app.
children = [
  supervisor(KafkaConsumer.Supervisor, []),
]
opts = [strategy: :one_for_one, name: YourApplication.Supervisor]
Supervisor.start_link(children, opts)
  • Start your app.