Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Add initial M3Aggregator documentation #1741

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
3 participants
@robskillington
Copy link
Collaborator

commented Jun 17, 2019

What this PR does / why we need it:

This documentation is not meant to be comprehensive but will serve as a start on providing documentation for running M3Aggregator.

Special notes for your reviewer:

Does this PR introduce a user-facing and/or backwards incompatible change?:

NONE

Does this PR require updating code package or user-facing documentation?:

Add initial M3Aggregator documentation

@robskillington robskillington changed the title [docs] Add initial aggregator documentation [docs] Add initial M3Aggregator documentation Jun 17, 2019


## Introduction

m3aggregator is used to cluster stateful downsampling and rollup of metrics before they are store in M3DB. The m3coordinator also performs this role but is not cluster aware. This means metrics will not get aggregated properly if you send metrics in round round fashion to multiple m3coordinators for the same metrics ingestion source (e.g. Prometheus server).

This comment has been minimized.

Copy link
@richardartoul

richardartoul Jun 18, 2019

Collaborator

store -> stored

This comment has been minimized.

Copy link
@richardartoul

richardartoul Jun 18, 2019

Collaborator

We usually say m3coordinator not "the m3coordinator"

This comment has been minimized.

Copy link
@richardartoul

richardartoul Jun 18, 2019

Collaborator

"m3coordinator can also perform this role, but is not clustered. This makes it difficult to scale out as metrics will not get aggregated properly when using round robin load balancing across multiple m3coordinators (since aggregation is a stateful operation)" or something like that


m3aggregator is used to cluster stateful downsampling and rollup of metrics before they are store in M3DB. The m3coordinator also performs this role but is not cluster aware. This means metrics will not get aggregated properly if you send metrics in round round fashion to multiple m3coordinators for the same metrics ingestion source (e.g. Prometheus server).

Metrics sent to m3aggregator are correctly routed to the instance resposible for aggregating each metric and there is multiple m3aggregator replicas to ensure no single point of failure during aggregation.

This comment has been minimized.

Copy link
@richardartoul

richardartoul Jun 18, 2019

Collaborator

"Similar to M3DB, m3aggregator supports clustering by default. This means that metrics are correctly routed to the instance(s) responsible for aggregating each metric and multiple m3aggregator replicas can be configured such that there are no single points of failure for aggregation"


Before setting up m3aggregator, make sure that you have at least [one M3DB node running](single_node.md) and a dedicated m3coordinator setup.

We highly recommend running with at least a replication factor 2 for a m3aggregator deployment.

This comment has been minimized.

Copy link
@richardartoul

richardartoul Jun 18, 2019

Collaborator

maybe add that if you run with R.F=1 you'll have downtime whenever you perform cluster operations like add/removes/joins


### Coordinator configuration

Metrics will still arrive at the m3coordinator, they simply need to be forwarded to m3aggregators. The m3coordinator then also needs to receive metrics that have been aggregated from the m3aggregator and store them in M3DB, so running an ingestion server should be configured.

This comment has been minimized.

Copy link
@richardartoul

richardartoul Jun 18, 2019

Collaborator

a digram here would be super helpful since its pretty confusing to have this prometheus --> m3coordinator --> m3aggregator --> m3coordinator flow.

Would also be good to say that the m3coordinators in that diagram can be different


```
make m3aggregator
./bin/m3aggregator -f ./src/aggregator/config/m3aggregator.yml

This comment has been minimized.

Copy link
@mmedvede

mmedvede Jun 27, 2019

Should sample m3aggregator.yml be added to the doc, or explicit mention about using the one from source? I am going through the setup and found that was the missing piece so far.

curl -vvvsSf -X POST http://m3coordinator01:7201/api/v1/topic/init -d '{
"numberOfShards": 64
}'
```

This comment has been minimized.

Copy link
@mmedvede

mmedvede Jul 11, 2019

Should the default topic name be mentioned? Looking through the source code, the default topic name used is aggregated_metrics. And there seems to be a way to specify the topic by setting topic-name in the http request header.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.