Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apparent memory leak introduced with v2.1.7 #6520

Closed
ginkel opened this issue Mar 19, 2020 · 2 comments
Closed

Apparent memory leak introduced with v2.1.7 #6520

ginkel opened this issue Mar 19, 2020 · 2 comments
Assignees
Projects
Milestone

Comments

@ginkel
Copy link

ginkel commented Mar 19, 2020

Do you want to request a feature or report a bug?

Bug

What did you do?

We have deployed Traefik v2.1 on a couple of nodes, limited to 128 MB memory each. This setup has been running stable ever since Trafik 2.x was released.

Last night our Traefik deployments have been automatically updated from v2.1.6 to v2.1.7.

Since then the instances are regularly OOM-killed. The following Grafana chart shows the difference in behavior (old: constant memory usage after a ramp-up, new: restart cycles due to OOMs):

image

image

What did you expect to see?

Traefik continues running as before.

What did you see instead?

Traefik is periodically killed by the Linux kernel due to an OOM (when hitting the 128 MB memory limit).

The issue can be reproduced more quickly by firing plenty of requests against the instance, e.g., using ab:

ab -n 100000 -c 10 https://<fqdn>/

Output of traefik version: (What version of Traefik are you using?)

$ docker exec -it traefik traefik version
Version:      2.1.7
Codename:     cantal
Go version:   go1.13.8
Built:        2020-03-18T16:43:27Z
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?

Linux amd64, running on Ubuntu 18.04 w/ Docker 19.03.8

accessLog:
    filePath: /log/access.log
api:
    dashboard: true
    debug: false
entryPoints:
    http:
        address: :80
    https:
        address: :443
    traefik:
        address: :81
log:
    format: json
    level: INFO
metrics:
    prometheus:
        entryPoint: traefik
providers:
    docker:
        exposedByDefault: false
    file:
        directory: /etc/traefik/rules
        watch: true
serversTransport:
    insecureSkipVerify: true
@ldez ldez added kind/bug/possible a possible bug that needs analysis before it is confirmed or fixed. priority/P0 needs hot fix and removed status/0-needs-triage labels Mar 19, 2020
@ldez ldez added this to issues in v2 via automation Mar 19, 2020
@ldez ldez added kind/bug/confirmed a confirmed bug (reproducible). area/middleware/metrics and removed kind/bug/possible a possible bug that needs analysis before it is confirmed or fixed. labels Mar 19, 2020
@ldez
Copy link
Member

ldez commented Mar 19, 2020

Thank you @ginkel 👍

We fixed that in #6522, and we will create a release today.

@traefiker traefiker added this to the 2.1 milestone Mar 19, 2020
@traefiker
Copy link
Contributor

Closed by #6522.

v2 automation moved this from issues to Done Mar 19, 2020
@traefik traefik locked and limited conversation to collaborators Apr 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
v2
Done
Development

No branches or pull requests

4 participants