Skip to content
Kafka-based storage for Zipkin.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.mvn/wrapper
autoconfigure
docs
examples
src/etc
storage
.gitignore
.travis.yml
CONTRIBUTING.md
DESIGN.md
Dockerfile
LICENSE
Makefile
README.md
docker-compose.yml
mvnw
mvnw.cmd
pom.xml

README.md

Zipkin Storage: Kafka

Build Status

Kafka-based storage for Zipkin.

This is in experimentation phase at the moment.

                    +-----------------------------zipkin------------------------------------------
                    |                                                         +-->( service:span )
                    |                                                         +-->( span-index   )
( collected-spans )-|->[ span-consumer ]                      [ span-store ]--+-->( traces       )
                    |         |                                    ^    |     +-->( dependencies )
                    +---------|------------------------------------|----|--------------------------
                              |                                    |    |
------------------------------|------------------------------------|----|--------------------
                              |                                    |    |
kafka                         +-->(  raw-spans  )--// potential //-+    +-->( traces )
                              |                    // enriching // |    |
                              +-->( light-spans )--// sampling  //-+    +-->( dependencies )

--------------------------------------------------------------------------------------------

Configuration

Configuration Description Default
KAFKA_STORE_SPAN_CONSUMER_ENABLED Process spans collected by Zipkin server true
KAFKA_STORE_SPAN_STORE_ENABLED Aggregate and store Zipkin data true
KAFKA_STORE_BOOTSTRAP_SERVERS Kafka bootstrap servers, format: host:port localhost:9092
KAFKA_STORE_ENSURE_TOPICS Ensure topics are created if don't exist true
KAFKA_STORE_DIRECTORY Root path where Zipkin stores tracing data /tmp/zipkin
KAFKA_STORE_COMPRESSION_TYPE Compression type used to store data in Kafka topics NONE
KAFKA_STORE_RETENTION_SCAN_FREQUENCY Frequency to scan old records, in milliseconds. 86400000 (1 day)
KAFKA_STORE_RETENTION_MAX_AGE Max age of a trace, to recognize old one for retention policies. 604800000 (7 day)
KAFKA_STORE_SPANS_TOPIC Topic where incoming spans are stored. zipkin-spans
KAFKA_STORE_SPANS_TOPIC_PARTITIONS Span topic number of partitions. 1
KAFKA_STORE_SPANS_TOPIC_REPLICATION_FACTOR Span topic replication factor. 1
KAFKA_STORE_TRACES_TOPIC Topic where aggregated traces are stored. zipkin-traces
KAFKA_STORE_TRACES_TOPIC_PARTITIONS Traces topic number of partitions. 1
KAFKA_STORE_TRACES_TOPIC_REPLICATION_FACTOR Traces topic replication factor. 1
KAFKA_STORE_TRACE_SPANS_TOPIC Topic where aggregated service names are stored. zipkin-services
KAFKA_STORE_TRACE_SPANS_TOPIC_PARTITIONS Services topic number of partitions. 1
KAFKA_STORE_TRACE_SPANS_TOPIC_REPLICATION_FACTOR Services topic replication factor. 1
KAFKA_STORE_DEPENDENCIES_TOPIC Topic where aggregated service dependencies names are stored. zipkin-dependencies
KAFKA_STORE_DEPENDENCIES_TOPIC_PARTITIONS Services topic number of partitions. 1
KAFKA_STORE_DEPENDENCIES_TOPIC_REPLICATION_FACTOR Services topic replication factor. 1

Use partitions and replication factor when Topics are created by Zipkin. If topics are created manually those options are not used.

Get started

To build the project you will need Java 8+.

make build
make test

Run locally

To run locally, first you need to get Zipkin binaries:

make get-zipkin

By default Zipkin will be waiting for a Kafka broker to be running on localhost:29092. If you don't have one, this service is available via Docker Compose:

make docker-kafka-up

Then run Zipkin locally:

make run

Run with Docker

Run:

make run-docker

And Docker image will be built and Docker compose will start.

Testing

To validate storage:

make zipkin-test

This will start a browser and check a traces has been registered.

Examples

There are two examples, running zipkin with kafka as storage:

  • Single-node: examples/single-node
  • Multi-mode: examples/multi-mode

Acknowledged

This project is inspired in Adrian Cole's https://github.com/adriancole/zipkin-voltdb

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.