Skip to content
Punt is a tiny and lightweight daemon which helps ship logs to Elasticsearch.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cmd Rename Organization (#2) Sep 19, 2017
lib Collect go runtime stats Feb 20, 2018
packaging Rename Organization (#2) Sep 19, 2017
util
.gitignore Prepare some things for release, cleanup Sep 11, 2017
.travis.yml Rename Organization (#2) Sep 19, 2017
LICENSE
README.md Update readme Jan 10, 2018
config.json Add support for automatically cleaning indexes (#3) Dec 15, 2017

README.md

Punt

Punt is a lightweight and simple daemon that parses, transforms, mutates, and ships logs into Elasticsearch. Punt was built as a fast and reliable alternative to Logstash, which means it's focus is to fit directly into existing ELK setups. Punt was built at Discord to manage the over 4 billion log lines we process per day.

Features

  • Designed to be fast and reliable
  • Simple JSON-based configuration file
  • Supports rsyslog over UDP/TCP, including multiple framing formats and SSL
  • Multiple ingest servers and egress ES clusters
  • Management and configuration of ES index templates and type mappings
  • Ability to parse and transform structured (JSON) logs
  • Threshold/Matching based alerting
  • Streaming API for real time log tailing
  • Support for attaching maxmind GeoIP2 geolocation data
  • Automatic deletion of old indexes based on retention rules
  • Exporting of metrics in the Datadog statsd format

Why Not Logstash?

When Discord originally started logging, we used a standard ELK stack setup. Initially this worked well for a low-volume of logs, however as our log volume grew (~750m log lines a day) Logstash quickly began to fall behind. As we spent more and more time tweaking and scaling Logstash/JVM/JRuby, we quickly realised it was not a long-term solution. Punt spawned out of a frustrating weekend dealing with constant Logstash lockups and JVM struggles.

Where Logstash aims to be immensely configurable and pluggable via its DSL, Punt aims to be an extremely performant solution, without compromising or reducing the core features required to handle and store structured log data.

Installation

Go

To install Punt using the Go toolchain, simply

go get github.com/discordapp/punt/cmd/puntd

Package (Debian/Ubuntu)

Punt was designed to be installed as a package on debian systems, and thus includes a simple dpkg build script based on fpm. To build a package simply:

cd packaging/
VERSION=0.0.1 ./build.sh

The package includes a simple upstart script.

You can’t perform that action at this time.