Pedestal interceptors and chain provider for Kafka applications.
A Pedestal chain provider connects a network library or container to an application via interceptors. The chain provider creates the execution context and invokes a stack of interceptors.
This library provides interceptors that can parse and process Kafka messages. Other interceptors are provided to generate new Kafka messages and topics.
One service map handles one topic. If you need to handle multiple topics, just make several service maps and start each one.
Define a Configuration
A "server" starts with a configuration that tells the Kafka client library how to connect. The configuration is a map with specific key names.
Configurations are fully specified in
com.cognitect.kafka.producer. Both of these pull in specs from
All the settings come directly from Kafka's client libraries. See Kafka Documentation for details.
Attach interceptors to your service map by assoc-ing them to the key
:enter function will be called with a context map
that includes the key
:message. The first interceptor's message
value will be a Kafka
After that it depends on how your interceptors transform the message.
Create a server and start it
Pass the configuration to
transforms the configuration map and returns a "service map".
Start the service map by passing it to
com.cognitect.kafka/start. That returns the "server". You'll want to
keep a reference to that server so you can stop it later.
This library provides some interceptors that help process messages.
The whole point of interceptors is to break a large computation or data transformation into manageable, isolated chunks. Think about how you can reduce semantic dependencies by communicating through the context.
The easiest way to set up is with docker-compose and the env directory.
$ cd env/dev $ bin/up
Several examples in
comment sections use the Kafka broker and
Zookeeper nodes created by the docker-compose file. You can, of
course, replace hostnames and ports to connect to any other Kafka and
Copyright © 2016-2017 Cognitect, Inc.
Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.