Skip to content
Quentin Manfroi edited this page Dec 4, 2020 · 12 revisions

🔗 Contents

Welcome to the terraform-signalfx-detectors wiki!

Goal

This repository is a collection of generic detectors usable in most of the situations. This goal will restrict the scope of available detectors.

It also provides an opinionated way to configure and manage his monitoring with SignalFx. First because it uses Terraform as deployment tool but also because it tries to enforce patterns, follow templating rules and provide a common and repeatable foundation for all detectors.

It does not aim to cover too specific usage, setup or implementation because it will not be reusable or does not address a more common need.

It is a good start for a new project, to detect known anomalies and usual problems, in common situations. Each module is focused on monitor one target from its own internal view.

Why

When designing detectors, it is good to understand this:

  • using highly complex formulas on multiple metrics from the same source is not a problem while it remains easy to setup and could be used in common situations.
  • but a very simple detector using only two metrics from different sources could be too difficult to setup or require dependencies not available for most of the people.

For example, correlate a simple CPU usage of an host (fully generic) with the latency of his own application (collected by a custom or apm metric): the detector is simple, easy to deploy but requirements cannot be generic by design.

Limits

Given this scope and because we want to preserve homogeneity and maintainability, some propositions do not belong here.

The behavior which fits to most of the people will always be favored compared to a better or more accurate approach which will only work in few situations.

This is also why we always recommend to create your own detectors, specific to your use case, data sources and situations. It will be always be a good complement and will provides a better monitoring coverage.

Where to start?

This wiki aims to bring a conceptual understanding of the project to go further either in usage or in development on this project. The docs in the repository, especially in the /docs directory aim to bring a more operational knowledge.

For a user who want to take advantages of the existing detectors and test them quickly we recommend to check the user guide to automatically build a stack ready to deploy with make stack command.

In the same way, for a contributor who want propose "common enough" changes you should check the developer guide. This will allow to easy remove, add, update detectors or modules to the code base.

Go further

If you are not satisfied with the way it works or it cannot fit your needs but you are still interesting by the approach and the goal of this project, please open an issue to discuss. But to go further in the usage or development of this project the wiki is the right place!

Keep in mind this project is highly dependent on:

  • Terraform tool and its HCL configuration language. To learn this we recommend to begin with the official learning tutorials.

  • SignalFx and especially its detector feature and its underlying SignalFlow language. To learn this we recommend to begin with this official documentation.

They are critical requirements to use and develop on this project so we suppose you know them a little before to go further.

As a user, if you want to go deeper in the customization of your deployed detectors
so you should check the Guidance documentation. The Getting started is optional; it show how to do manually what make stack does automatically.

As a developer, if you want to write advanced detectors that the generator does not support or even try to improve and add new features to the project so you should take a look at the Development_ section of this wiki which describe a module Structure and a detector Templating crucial to make important changes.