Skip to content
M3 monorepo - Distributed TSDB, Aggregator and Query Engine, Prometheus Sidecar, Graphite Compatible, Metrics Platform
Go Shell JavaScript Makefile TLA HTML Other
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.buildkite [ci] skip flaky test + fossa temporarily (#1895) Aug 21, 2019
.ci @ 323f860 [ci] update ci-scripts (#1900) Aug 21, 2019
.github [docs] Add draft PR instructions to PR template doc (#1659) May 22, 2019
docker [docker] build setcap from alpine 3.9 (#1850) Aug 2, 2019
docs [docs] Add shard/block size recs (#1890) Aug 21, 2019
examples/dbnode/proto_client Change to Github links (#1870) Aug 9, 2019
integrations [coordinator] Use table based approach for aggregation tile buffer pa… Jun 9, 2019
kube Fix minor typo in sysctl-setter daemonset (#1836) Jul 28, 2019
m3db.io Change to Github links (#1870) Aug 9, 2019
scripts [dbnode] Use already encoded tags when writing time series to commit … Aug 21, 2019
specs Add TLA+ specification for future M3DB version-based flushing (#1223) Jan 16, 2019
src [dbnode] Use already encoded tags when writing time series to commit … Aug 21, 2019
.codecov.yml Optimize serialize.TagDecoder internal pooling usage (#1114) Oct 19, 2018
.dockerignore Enforce mockgen is installed at retool installed version (#1257) Dec 20, 2018
.excludecoverage Merge m3aggregator into monorepo Oct 9, 2018
.excludefmt Update m3x and use code generated maps instead of ident.Hash based ma… Apr 17, 2018
.excludelint Update m3x and use code generated maps instead of ident.Hash based ma… Apr 17, 2018
.excludemetalint Ignore autogenerated grammar peg file from metalinting (#1284) Jan 4, 2019
.fossa.yml Add FOSSA license scan to CI (#1131) Nov 19, 2018
.gitattributes [k8s] Initial Kube manifests + docs (#733) May 30, 2018
.gitignore Add vagrant VM autoprovision environment for benchmarking (#1791) Jul 6, 2019
.gitmodules Start using the ci-scripts submodule (#232) Jan 25, 2017
.goreleaser.yml Use new quay docker repos with m3query, update docs, add m3query to g… Aug 7, 2018
.metalinter.json Merge m3aggregator into monorepo Oct 9, 2018
.nvmrc Merge m3ctl into monorepo Feb 25, 2019
CHANGELOG.md Support for numbered filesets (#1720) Jun 21, 2019
CONTRIBUTING.md Add pull request template (#1575) Apr 19, 2019
DEVELOPER.md [dbnode] Safe handling of concurrent bloom filters (#1837) Jul 28, 2019
LICENSE Update license to Apache 2 (#664) May 18, 2018
Makefile [DBNode] - Add support for passive replication between M3DB Clusters (#… Aug 14, 2019
NOTICES.txt Copy cors from etcd, since they eliminated the package (#1320) Jan 30, 2019
README.md [docs] Clarify Prom querying in single_node docs (#1612) May 8, 2019
TESTING.md Clean up files after monorepo merge (#734) May 30, 2018
docker-compose.yml [ci] build + release with go1.12 (again) (#1674) Jun 6, 2019
glide.lock [dbnode] Use already encoded tags when writing time series to commit … Aug 21, 2019
glide.yaml [dbnode] Use already encoded tags when writing time series to commit … Aug 21, 2019
issue_template.md Create an M3 issue template (#1172) Nov 14, 2018
mkdocs.yml [docker] dual-publish setcap; default raise limits (#1745) Aug 1, 2019
tools.json [kube] validate manifests in CI (#1456) Mar 14, 2019

README.md

M3 GoDoc Build Status Coverage Status FOSSA Status Gitter chat

M3 Logo

Distributed TSDB and Query Engine, Prometheus Sidecar, Metrics Aggregator, and more. "More" now includes Graphite storage and query engine!

More information:

Community meetings

M3 contributors and maintainers have monthly (every four weeks) meetings at 11:00 AM (USA Pacific) on Tuesday.

An initial agenda is posted to the community meeting shared Google doc a day before each meeting, and everyone is welcome to suggest additional topics or other agendas.

Please download and import the following iCalendar (.ics) files to calendar system.

Monthly: us04web.zoom.us/meeting/519434268/ics?icsToken=0509418cef48c753d095f78eabc177f7ee2ced032a53913644bc86bb88b54862

Join Zoom Meeting: us04web.zoom.us/j/519434268

Test it out

The easiest way to testing out M3 is to follow one of the guides from the documentation. For a fully comprehensive getting started guide, see our single node how-to.

Starting a node

# to build a local m3dbnode process
make m3dbnode (note that we currently require at least Go 1.10 or higher)

# run it with the sample configuration
./bin/m3dbnode -f ./src/dbnode/config/m3dbnode-local-etcd.yml

To cross-compile and build for Linux AMD64 build with make m3dbnode-linux-amd64.

Creating a namespace to store metrics

curl -X POST http://localhost:7201/api/v1/database/create -d '{
  "type": "local",
  "namespaceName": "default",
  "retentionTime": "2h"
}'

Test RPC

To test out some of the functionality of M3DB there are some user friendly HTTP JSON APIs that you can use. These use the DB node cluster service endpoints.

Note: performance sensitive users are expected to use the more performant endpoints via either the Go src/dbnode/client/Session API, or the GRPC endpoints exposed via src/coordinator.

Write a datapoint

curl http://localhost:9003/writetagged -s -X POST -d '{
  "namespace": "default",
  "id": "foo",
  "tags": [
    {
      "name": "__name__",
      "value": "user_login"
    },
    {
      "name": "city",
      "value": "new_york"
    },
    {
      "name": "endpoint",
      "value": "/request"
    }
  ],
  "datapoint": {
    "timestamp":'"$(date +"%s")"',
    "value": 42.123456789
  }
}'

Query for reverse indexed time series data

curl http://localhost:9003/query -s -X POST -d '{
  "namespace": "default",
  "query": {
    "regexp": {
      "field": "city",
      "regexp": ".*"
    }
  },
  "rangeStart": 0,
  "rangeEnd":'"$(date +"%s")"'
}' | jq .

Building with Docker

A Dockerfile is included for both development and production deployment purposes. It uses a multi-stage build in order to produce a lightweight production image from a single Dockerfile. Accordingly, it requires Docker 17.05 or later to build.

docker build -f docker/m3dbnode/Dockerfile -t m3dbnode:$(git rev-parse head) .
docker run --name m3dbnode m3dbnode:$(git rev-parse head)

If you wish to build an image with the source code included you can stop the build after the builder stage:

docker build -f docker/m3dbnode/Dockerfile -t m3dbnode:$(git rev-parse head) --target builder .

Configuration

The default Docker image will start a single m3dbnode process with an embedded etcd instance to mimic a production environment. If you would like to further customize the configuration, you must provide your own and mount it into the container:

docker run --name m3dbnode -v /host/config.yml:/etc/m3dbnode/myconfig.yml m3dbnode:tag -f /etc/m3dbnode/myconfig.yml

This project is released under the Apache License, Version 2.0.

You can’t perform that action at this time.