prometheus_rule_group_last_duration_seconds is in ms?

[sepich]

Bug Report

What did you do?
We're using alerts with prometheus_rule_group_last_duration_seconds.

What did you expect to see?
In v2.6.1 it was 0.005921854 which seems like seconds

What did you see instead? Under which circumstances?
Starting with 2.7.0 it start to return too large values

# HELP prometheus_rule_group_last_duration_seconds The duration of the last rule group evaluation.
# TYPE prometheus_rule_group_last_duration_seconds gauge
prometheus_rule_group_last_duration_seconds{rule_group="/etc/prometheus-shared/rules/app.yml;app"} 1.417883e+06

so rules are failing now:
prometheus_rule_group_last_duration_seconds/prometheus_rule_group_interval_seconds now in thousands

Environment

• System information:

Linux 4.9.0-8-amd64 x86_64

• Prometheus version:

prometheus, version 2.7.0 (branch: HEAD, revision: 410ee9e04acb8f59f400858752ca82b4ef88035e)
  build user:       root@3bc81b516055
  build date:       20190128-10:09:51
  go version:       go1.11.5

[codesome]

Can confirm this is a bug

prometheus/rules/manager.go

Line 372 in a2ef8cf

g.metrics.groupLastDuration.WithLabelValues(groupKey(g.file, g.name)).Set(float64(dur))

- I don't see it being changed to seconds here.

[cstyan]

@codesome this too, looks like the automatic closes #issue isn't working?

[simonpasquier]

@cstyan when the fix isn't made against master, GitHub doesn't auto-close the issue...

[cstyan]

Oh, didn't know that! Thanks @simonpasquier