send prometheus metrics to elastic
Clone or download
Permalink
Failed to load latest commit information.
_meta Make name field hierarchy configurable (#14) Sep 29, 2018
beater Strip underscore from labels.name and check for NaN of value Jun 13, 2018
cmd Fix build issues: Upgrade libbeat and refactor prom imports for lates… Jun 12, 2018
config Make name field hierarchy configurable (#14) Sep 29, 2018
data Fix build issues: Upgrade libbeat and refactor prom imports for lates… Jun 12, 2018
docs Fix build issues: Upgrade libbeat and refactor prom imports for lates… Jun 12, 2018
include Add vendoring & Upgrade libbeat to 6.4.1 (#10) Sep 28, 2018
prometheus Make name field hierarchy configurable (#14) Sep 29, 2018
tests/system Initial commit Jul 5, 2017
vendor make release functionality (#13) Sep 29, 2018
.gitignore Fix build issues: Upgrade libbeat and refactor prom imports for lates… Jun 12, 2018
.travis.yml Initial commit Jul 5, 2017
CONTRIBUTING.md Initial commit Jul 5, 2017
Dockerfile Move Dockerfile to root; own builder image; run as prometheusbeat user Sep 28, 2018
Gopkg.lock make release functionality (#13) Sep 29, 2018
Gopkg.toml make release functionality (#13) Sep 29, 2018
LICENSE.txt make release functionality (#13) Sep 29, 2018
Makefile Initial commit Jul 5, 2017
NOTICE.txt make release functionality (#13) Sep 29, 2018
README.md make release functionality (#13) Sep 29, 2018
magefile.go make release functionality (#13) Sep 29, 2018
main.go Fix build issues: Upgrade libbeat and refactor prom imports for lates… Jun 12, 2018
main_test.go Add vendoring & Upgrade libbeat to 6.4.1 (#10) Sep 28, 2018
prometheusbeat.reference.yml Add vendoring & Upgrade libbeat to 6.4.1 (#10) Sep 28, 2018
prometheusbeat.yml Make name field hierarchy configurable (#14) Sep 29, 2018

README.md

Prometheusbeat

Prometheusbeat is an elastic beat that can receive Prometheus metrics via the remote write feature.

Example Prometheusbeat configuration:

prometheusbeat:
  listen: ":8080"
  context: "/prometheus"

[...]

Example Prometheus configuration:

[...]

remote_write:
  url: "http://localhost:8080/prometheus"

[...]

Example Prometheusbeat event:

{
  "@timestamp": "2018-09-28T11:44:07.006Z",
  "@metadata": {
    "beat": "prometheusbeat",
    "type": "doc",
    "version": "6.4.1"
  },
  "labels": {
    "le": "10",
    "name": "prometheus_tsdb_tombstone_cleanup_seconds_bucket",
    "instance": "localhost:9090",
    "job": "prometheus"
  },
  "value": 0,
  "host": {
    "name": "example.com"
  },
  "beat": {
    "version": "6.4.1",
    "name": "prometheusbeat",
    "hostname": "example.com"
  }
}

Ensure that this folder is at the following location: ${GOPATH}/github.com/infonova/prometheusbeat

Getting Started with Prometheusbeat

Requirements

Init Project

To get running with Prometheusbeat and also install the dependencies, run the following command:

make setup

It will create a clean git history for each major step. Note that you can always rewrite the history if you wish before pushing your changes.

To push Prometheusbeat in the git repository, run the following commands:

git remote set-url origin https://github.com/infonova/prometheusbeat
git push origin master

For further development, check out the beat developer guide.

Build

To build the binary for Prometheusbeat run the command below. This will generate a binary in the same directory with the name prometheusbeat.

make

Run

To run Prometheusbeat with debugging output enabled, run:

./prometheusbeat -c prometheusbeat.yml -e -d "*"

Test

To test Prometheusbeat, run the following command:

make testsuite

alternatively:

make unit-tests
make system-tests
make integration-tests
make coverage-report

The test coverage is reported in the folder ./build/coverage/

Update

Each beat has a template for the mapping in elasticsearch and a documentation for the fields which is automatically generated based on etc/fields.yml. To generate etc/prometheusbeat.template.json and etc/prometheusbeat.asciidoc

make update

Cleanup

To clean Prometheusbeat source code, run the following commands:

make fmt
make simplify

To clean up the build directory and generated artifacts, run:

make clean

Clone

To clone Prometheusbeat from the git repository, run the following commands:

mkdir -p ${GOPATH}/github.com/infonova/prometheusbeat
cd ${GOPATH}/github.com/infonova/prometheusbeat
git clone https://github.com/infonova/prometheusbeat

For further development, check out the beat developer guide.

Releasing

The beat framework provides tools to crosscompile and package your beat for different platforms. This requires docker and vendoring as described above. Note that as long as dep#1306 is not fixed you have to perform make copy-vendor before creating a release.

To build packages of your beat, run the following command:

make release

This will fetch and create all images required for the build process. The whole process to finish can take several minutes.

Building Docker Image

Prometheusbeat Docker image can be customized and build to run on any environments

docker build . -t infonova/prometheusbeat:latest

Running Docker Image

Prometheusbeat Docker image can be run using below command

docker run -d  --name prometheusbeat -p <<dockerhost-port>>:8080 -v <<host-config-path>>:/prometheusbeat.yml infonova/prometheusbeat:latest