prometheus 2.1.0 send alert return 400

[ZeeShen]

• Prometheus version: 2.1.0
• Alertmanager version: 0.14.0

Logs:
prometheus log:

Feb 27 09:10:00 prometheus-prod prometheus[5996]: level=error ts=2018-02-27T09:10:00.523796339Z caller=notifier.go:454 component=notifier alertmanager=http://xxx.com/api/v1/alerts count=1 msg="Error sending alert" err="bad response status 400 Bad Request"

alertmanager log:

Feb 27 09:10:00 prometheus-prod alertmanager[26860]: level=error ts=2018-02-27T09:10:00.523024004Z caller=api.go:826 msg="API error" err="bad_data: start time must be before end time"

alert request body dumped by tcpdump(not the same request in the log, but can also cause this 400 resp):

[{
  "labels": {
    "alertname": "TargetDown",
    "env": "production",
    "instance": "xxx",
    "job": "k8s-n1a-nodes",
    "k8s_cluster": "k8s-n1a",
    "severity": "critical"
  },
  "annotations": {
    "summary": "Target has been down for more than 5 minutes."
  },
  "startsAt": "0001-01-01T00:00:00Z",
  "endsAt": "2018-02-27T11:19:37.448602029Z",
  "generatorURL": "http://xxx.com/graph?g0.expr=xxx"
}]

When startsAt is zero, alertmanager will reset it to now, so this error happened. Maybe prometheus should not set startsAt as zero?

[krasi-georgiev]

looks like a dupe of #3543 which is fixed in 2.2

[simonpasquier]

Definitely a duplicate. This can be closed.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.