Vostok Hercules is an open-source distributed system based on Apache Kafka and used for reliable delivery of telemetry data from microservices to storage and analytics engines.
Vostok Hercules is developed by Kontur as a part of Vostok project — an ecosystem for easy and fast development of microservices primarily using .NET Core / C#.
Vostok Hercules provides abstractions for application code to encapsulate specific data format in telemetry engines depending on a telemetry type. Vostok Hercules provides out of the box 4 telemetry types:
- logs,
- metrics,
- distributed traces,
- annotations.
Telemetry is an event which has a timestamp and tags. Set of tags depends on a telemetry type.
Vostok Hercules delivers logs to ElasticSearch.
Also, logs with level ERROR or FATAL are delivered to Sentry.
See supported log event schema for details.
Vostok Hercules delivers metrics to Graphite, Moira or any other tool which supports Graphite format. Metrics can be in the plaintext format, when a metric name consists of several components joined by a dot or in the modern tagged format. See supported metric event schema for details.
Vostok Hercules stores distributed traces into ClickHouse. To analyze traces use Contrails web interface. Contrails is a tool for tree span analysis. See supported trace span schema for details.
Annotation marks point on graphs in Grafana and Kibana. Vostok Hercules delivers annotations to ElasticSearch. See supported annotation event schema for details.
Vostok Hercules supports an extendable processing of telemetry events and custom telemetry types as well. Hercules can be extended by one of the following ways:
- Use one of provided Sinks (e.g. see Elastic Sink),
- Write an integration via universal HTTP Stream API or Timeline API in any language you prefer,
- Write a new Sink in Java.
Minimal data unit of Vostok Hercules is an Event.
Events do not have predefined schema except well-known tags in some cases.
To support schemaless processing Vostok Hercules stores and transfers events
in the binary format with a self-described schema (see Hercules Protocol).
A Stream is a collection of events of the same type.
Each stream has a backing topic in Apache Kafka.
An application uses hercules client to write events to Gate into streams. Other applications can read these events from Stream API or Timeline API using the same client.
Various sink daemons can deliver events to various backends at the same time. List of supported sink daemons:
- Elastic Sink delivers events to ElasticSearch,
- Graphite Sink delivers metrics to Graphite and Moira,
- Sentry Sink delivers errors to Sentry.
Also, there are several auxiliary sink daemons:
- Stream Sink combines different streams into single one,
- Timeline Sink delivers events to Cassandra
to build time series or
Timelines, see Timeline API for details, - Tracing Sink delivers tracing spans to ClickHouse, see Tracing API for details.
To analyze distributed traces use Contrails. Contrails read tracing spans from Tracing API.
There are several applications to manage streams and timelines:
- Management API has HTTP JSON API,
- Stream Manager,
- Timeline Manager.
Vostok Hercules uses internally this list of tools:
- Apache Kafka for storing streams,
- Apache ZooKeeper for meta-data (configuration for streams, timelines and others),
- Apache Cassandra for storing timelines,
- ClickHouse for storing tracing spans.
Spaceport is an all-in-one development infrastructure.
Also, each component of Vostok Hercules can be run separately using Docker or as simple java application.
The official Docker images are on Docker Hub.
You need to have Java 8 or higher and Maven 3.3.9 or higher.
Build project with tests:
mvn package
Build project without tests:
mvn -DskipTests=true package
You need make and Docker 19.03 or higher.
Build Docker images:
make build_images