The list of dropped targets grows forever

[simonpasquier]

What did you do?

Setup Prometheus with kubernetes_sd scrape configuration. Then go to the Status > Service discovery page in the Prometheus UI.

What did you expect to see?

The list of targets including those which have been dropped.

What did you see instead? Under which circumstances?

The list of dropped targets is larger than what it should be and continues to grow over time.

Environment

• System information:

Linux 4.4.0-104-generic x86_64

• Prometheus version:

prometheus, version 2.0.0 (branch: master, revision: ec94df4)
build user: pasquier-s@simon-trusty
build date: 20180104-09:35:39
go version: go1.9.2

[simonpasquier]

IIUC we're accumulating dropped targets and the list is never reset.

See

prometheus/retrieval/scrape.go

Line 263 in ec94df4

sp.droppedTargets = append(sp.droppedTargets, t)

[simonpasquier]

cc @Conorbro

[Conorbro]

Thanks, I'll take a look now.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.