Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker Integration #632

Merged
merged 13 commits into from Feb 18, 2021
112 changes: 112 additions & 0 deletions packages/docker/_dev/build/docs/README.md
@@ -0,0 +1,112 @@
# Docker Integration

This Integration fetches metrics from [Docker](https://www.docker.com/) containers. The default data streams are: `container`, `cpu`, `diskio`, `healthcheck`, `info`, `memory` and `network`. The `image` metricset is not enabled by default.

## Compatibility

The Docker module is currently tested on Linux and Mac with the community
edition engine, versions 1.11 and 17.09.0-ce. It is not tested on Windows,
but it should also work there.

## Running from within Docker

The `docker` Integration will try to connect to the docker socket, by default at `unix:///var/run/docker.sock`.
If Elastic Agent is running inside docker, you'll need to mount the unix socket inside the container:

```
docker run -d \
fearful-symmetry marked this conversation as resolved.
Show resolved Hide resolved
--name=metricbeat \
--user=root \
--volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
docker.elastic.co/beats/metricbeat:latest metricbeat -e \
-E output.elasticsearch.hosts=["elasticsearch:9200"]
```

## Module-specific configuration notes

It is strongly recommended that you run Docker metricsets with a
<<metricset-period,`period`>> that is 3 seconds or longer. The request to the
Docker API already takes up to 2 seconds. Specifying less than 3 seconds will
result in requests that timeout, and no data will be reported for those
requests.

## Metrics

### Container

The Docker `container` data stream collects information and statistics about
running Docker containers.

{{fields "container"}}

fearful-symmetry marked this conversation as resolved.
Show resolved Hide resolved
{{event "container"}}

### CPU

The Docker `cpu` data stream collects runtime CPU metrics.

{{fields "cpu"}}

{{event "cpu"}}

### Diskio

The Docker `diskio` data stream collects disk I/O metrics.

{{fields "diskio"}}

{{event "diskio"}}

### Event

The Docker `event` data stream collects docker events

{{fields "event"}}

{{event "event"}}

### Healthcheck

The Docker `healthcheck` data stream collects healthcheck status metrics about
running Docker containers.

Healthcheck data will only be available from docker containers where the
docker `HEALTHCHECK` instruction has been used to build the docker image.

{{fields "healthcheck"}}

{{event "healthcheck"}}

### Image

The Docker `image` data stream collects metrics on docker images

{{fields "image"}}

{{event "image"}}

### Info

The Docker `info` data stream collects system-wide information based on the
https://docs.docker.com/engine/reference/api/docker_remote_api_v1.24/#/display-system-wide-information[Docker Remote API].

{{fields "info"}}

{{event "info"}}

### Memory

The Docker `memory` data stream collects memory metrics from docker.

{{fields "memory"}}

{{event "memory"}}


### Network

The Docker `network` data stream collects network metrics.

{{fields "network"}}

{{event "network"}}
8 changes: 8 additions & 0 deletions packages/docker/_dev/deploy/docker/Dockerfile
@@ -0,0 +1,8 @@
FROM friendsofgo/killgrave:0.4.0
WORKDIR /home

ADD imposters /home/imposters

EXPOSE 3000

ENTRYPOINT ["/go/bin/killgrave", "-host", "0.0.0.0", "-imposters", "/home/imposters"]
8 changes: 8 additions & 0 deletions packages/docker/_dev/deploy/docker/docker-compose.yml
@@ -0,0 +1,8 @@
version: '2.3'
services:
mocker:
build:
context: .
dockerfile: Dockerfile
ports:
- 3000
258 changes: 258 additions & 0 deletions packages/docker/_dev/deploy/docker/imposters/container_86a4b.json
@@ -0,0 +1,258 @@
{
"blkio_stats": {
"io_merged_recursive": [],
"io_queue_recursive": [],
"io_service_bytes_recursive": [
{
"major": 8,
"minor": 0,
"op": "Read",
"value": 311296
},
{
"major": 8,
"minor": 0,
"op": "Write",
"value": 283956224
},
{
"major": 8,
"minor": 0,
"op": "Sync",
"value": 284267520
},
{
"major": 8,
"minor": 0,
"op": "Async",
"value": 0
},
{
"major": 8,
"minor": 0,
"op": "Discard",
"value": 0
},
{
"major": 8,
"minor": 0,
"op": "Total",
"value": 284267520
},
{
"major": 253,
"minor": 0,
"op": "Read",
"value": 311296
},
{
"major": 253,
"minor": 0,
"op": "Write",
"value": 283956224
},
{
"major": 253,
"minor": 0,
"op": "Sync",
"value": 284267520
},
{
"major": 253,
"minor": 0,
"op": "Async",
"value": 0
},
{
"major": 253,
"minor": 0,
"op": "Discard",
"value": 0
},
{
"major": 253,
"minor": 0,
"op": "Total",
"value": 284267520
}
],
"io_service_time_recursive": [],
"io_serviced_recursive": [
{
"major": 8,
"minor": 0,
"op": "Read",
"value": 10
},
{
"major": 8,
"minor": 0,
"op": "Write",
"value": 3469
},
{
"major": 8,
"minor": 0,
"op": "Sync",
"value": 3479
},
{
"major": 8,
"minor": 0,
"op": "Async",
"value": 0
},
{
"major": 8,
"minor": 0,
"op": "Discard",
"value": 0
},
{
"major": 8,
"minor": 0,
"op": "Total",
"value": 3479
},
{
"major": 253,
"minor": 0,
"op": "Read",
"value": 10
},
{
"major": 253,
"minor": 0,
"op": "Write",
"value": 2394
},
{
"major": 253,
"minor": 0,
"op": "Sync",
"value": 2404
},
{
"major": 253,
"minor": 0,
"op": "Async",
"value": 0
},
{
"major": 253,
"minor": 0,
"op": "Discard",
"value": 0
},
{
"major": 253,
"minor": 0,
"op": "Total",
"value": 2404
}
],
"io_time_recursive": [],
"io_wait_time_recursive": [],
"sectors_recursive": []
},
"cpu_stats": {
"cpu_usage": {
"percpu_usage": [
8989285945449,
8802991575246,
8723071634953,
8812077512624
],
"total_usage": 35327426668272,
"usage_in_kernelmode": 2720580000000,
"usage_in_usermode": 32477900000000
},
"online_cpus": 4,
"system_cpu_usage": 7640294590000000,
"throttling_data": {
"periods": 0,
"throttled_periods": 0,
"throttled_time": 0
}
},
"id": "86a4d10d07bf0b1ded54c7156b4362e8b0dba9d450201a9695b4714c1492829c",
"memory_stats": {
"limit": 15620788224,
"max_usage": 420536320,
"stats": {
"active_anon": 59502592,
"active_file": 63746048,
"cache": 194330624,
"dirty": 0,
"hierarchical_memory_limit": 9223372036854771712,
"hierarchical_memsw_limit": 9223372036854771712,
"inactive_anon": 60268544,
"inactive_file": 130256896,
"mapped_file": 123408384,
"pgfault": 37852947,
"pgmajfault": 0,
"pgpgin": 25772802,
"pgpgout": 25697155,
"rss": 119738368,
"rss_huge": 4194304,
"total_active_anon": 59502592,
"total_active_file": 63746048,
"total_cache": 194330624,
"total_dirty": 0,
"total_inactive_anon": 60268544,
"total_inactive_file": 130256896,
"total_mapped_file": 123408384,
"total_pgfault": 37852947,
"total_pgmajfault": 0,
"total_pgpgin": 25772802,
"total_pgpgout": 25697155,
"total_rss": 119738368,
"total_rss_huge": 4194304,
"total_unevictable": 0,
"total_writeback": 0,
"unevictable": 0,
"writeback": 0
},
"usage": 326488064
},
"name": "/elastic-package-stack_elastic-agent_1",
"networks": {
"eth0": {
"rx_bytes": 23962833,
"rx_dropped": 0,
"rx_errors": 0,
"rx_packets": 128811,
"tx_bytes": 593646535,
"tx_dropped": 0,
"tx_errors": 0,
"tx_packets": 186477
}
},
"num_procs": 0,
"pids_stats": {
"current": 68
},
"precpu_stats": {
"cpu_usage": {
"percpu_usage": [
8989283978798,
8802978152064,
8723042900675,
8812068229234
],
"total_usage": 35327373260771,
"usage_in_kernelmode": 2720550000000,
"usage_in_usermode": 32477870000000
},
"online_cpus": 4,
"system_cpu_usage": 7640290600000000,
"throttling_data": {
"periods": 0,
"throttled_periods": 0,
"throttled_time": 0
}
},
"preread": "2021-02-04T22:02:38.129030814Z",
"read": "2021-02-04T22:02:39.138448063Z",
"storage_stats": {}
}