# [FEATURE] AutoRange

[vdaviot]

This pull-request is related to:

• cli: For the docker-compose.yml parsing and $ docker container stats format
• swarmkit: For the API update

What

AutoRange is a feature that helps to find and to apply the optimal limits for a service. It is an extension for the docker collector and require swarm mode to be enabled.

Why

This collector extension was thought as a way to monitor and predict the optimal consumption limits for a service. The goal was to find the point where a service could function properly, but still save as much resources as possible. It was written as a way to answer the question
How to optimize the number of services running on our infrastructure without losing quality of service?

How

The logic behind the feature can be described in 3 points:
- First, we collect the metrics and apply some transformations on it to generate two values.
Those values represent a "box" around the actual consumption.
- Then, we transform these values into time-series, using some of the key data collected previously to
weight our operations. The amplitude of change between values is monitored to know if it's time to stop measurements.
- Finally, we obtain refined values that we apply as limitation to the service. The data are then kept in a reduced form to limit memory usage.

Usage

The functionality is declared by adding the autorange key to the docker-compose.yml.
The mechanism works for cpu% and memory, with or without base values.
Below is an example of both.

services:
  myservice:
    autorange:
      memory:
      cpu%:

The available keys are:
- min (in octets)
- max (in octets)
- threshold% (only for memory, represents a security margin that will be refined by the algorithm)

services:
  myservice:
    autorange:
      memory:
          min: "110000"
          max: "120000"
          threshold%: "10"
      cpu%:
          min: "60"
          max: "70"

This functionality is deployed with $ docker stack deploy --compose-file=/your/compose/file and then $ docker container stats --format autorange (format is not necessary but shows the predicted values). The docker container stats is mandatory to start the collector. The collector needs to keep running to accumulate and predict values. You can always leave the docker container stats screen and come back later, the mechanism will be paused and the accumulated data won't be lost.

Improvements

Thing that could be improved:
- Compatibility with docker-compose, by removing the need for swarm labels.
- Avoid running $ docker container stats to start the collector, getting stats directly from the api.
- Implement a trigger to re-ignite the mechanism, using the old values as bases to further refine the predictions, in case of a change in behavior.

I'm open to any suggestion on how to refine the code/feature to better suit the docker scheme.

Cute animal

[thaJeztah]

oh! could you open this pull request in the upstream https://github.com/moby/moby repository? The docker/engine repository is only used to maintain release branches for the docker engine, but all feature development is done in the upstream moby/moby repository 🤗

[vdaviot]

Hi @thaJeztah,
Ok i will move this on https://github.com/moby/moby.