修改了模板没有生效 #37

imlxh · 2020-06-28T07:27:10Z

修改了钉钉的模板，但是告警时还是发的老的模板内容，还有如果我想遍历所有label，怎么改模板哈

feiyu563 · 2020-06-28T08:22:36Z

你用的接口是模板接口：/prometheusalert吗，自定义模板必须对应自定义模板接口才能生效，并且模板的名称不能跟别的模板名称重复

---原始邮件--- 发件人: "imlxh"<notifications@github.com> 发送时间: 2020年6月28日(周日) 下午3:27 收件人: "feiyu563/PrometheusAlert"<PrometheusAlert@noreply.github.com>; 抄送: "Subscribed"<subscribed@noreply.github.com>; 主题: [feiyu563/PrometheusAlert] 修改了模板没有生效 (#37) 修改了钉钉的模板，但是告警时还是发的老的模板内容，还有如果我想遍历所有label，怎么改模板哈 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

imlxh · 2020-06-29T01:37:18Z

哦哦，用的不是您说的这个接口，但是发现提示是多个告警，发出来的只显示了第一个告警，这个感觉还需要优化下

feiyu563 · 2020-06-29T01:43:15Z

老的接口尚未接入自定义模板，下一版本会接入

…

---原始邮件--- 发件人: "imlxh"<notifications@github.com> 发送时间: 2020年6月29日(周一) 上午9:37 收件人: "feiyu563/PrometheusAlert"<PrometheusAlert@noreply.github.com>; 抄送: "Comment"<comment@noreply.github.com>;"feiyu563"<244217140@qq.com>; 主题: Re: [feiyu563/PrometheusAlert] 修改了模板没有生效 (#37) 哦哦，用的不是您说的这个接口，但是发现提示是多个告警，发出来的只显示了第一个告警，这个感觉还需要优化下 — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

imlxh · 2020-06-29T01:52:04Z

好的，我如果想把所有label都打印出来，模板该怎么设置呢，必须都写出来么

imlxh · 2020-06-29T01:54:01Z

类似于这种，每个JOB的内容可能都不一样，所以显示的时候也需要展示些关键信息

feiyu563 · 2020-06-29T01:54:35Z

参考#30

imlxh · 2020-06-29T01:56:31Z

这个我看了，是固定的，并没有展示所有内容，告警比较简练

xmx0632 · 2020-06-29T02:48:44Z

@feiyu563 你好，我在grafana里设置了发告警到 http://PrometheusAlert/grafana/dingding
从钉钉上可以接收到告警消息，如图

当是按照文档：https://github.com/feiyu563/PrometheusAlert/blob/master/doc/readme/tpltest.md 说的去日志中查看接收到的消息却没有如文档中说的关键字信息。

以下是我从日志中拿到的信息：

2020/06/29 02:37:15.400 [I] [value.go:460] [1593398235400197882] {"evalMatches":[],"message":"nacos db exception!!!","ruleId":27,"ruleName":"db exception alert","ruleUrl":"http://localhost:3000/d/Bz_QALEiz12/nacos-1?fullscreen\u0026edit\u0026tab=alert\u0026panelId=54\u0026orgId=1","state":"alerting","tags":{},"title":"[Alerting] db exception alert"}
2020/06/29 02:37:15.400 [I] [grafana.go:155] [1593398235400197882] [dingding] {"msgtype":"markdown","markdown":{"title":"PrometheusAlert故障告警信息","text":"## PrometheusAlertGrafana故障告警信息\n\n#### db exception alert\n\n###### 告警级别：灾难\n\n###### 开始时间：2020-06-29 02:37:15\n\n##### nacos db exception!!!\n\n"},"at":{"atMobiles":["15395105573"],"isAtAll":true}}

2020/06/29 02:37:15.883 [I] [grafana.go:155] [1593398235400197882] [dingding] {"errcode":0,"errmsg":"ok"}
2020/06/29 02:37:15.883 [I] [value.go:460] [1593398235400197882] 告警消息发送完成.
2020/06/29 02:37:15.883 [D] [server.go:2802] | 172.17.0.4| 200 | 483.872963ms| match| POST /grafana/dingding r:/grafana/dingding

请问这里需要如何定制模版呢？是不是文档与代码不同步？多谢

feiyu563 · 2020-06-29T02:54:49Z

这些就是grafana的json

{"evalMatches":[],"message":"nacos db exception!!!","ruleId":27,"ruleName":"db exception alert","ruleUrl":"http://localhost:3000/d/Bz_QALEiz12/nacos-1?fullscreen\u0026edit\u0026tab=alert\u0026panelId=54\u0026orgId=1","state":"alerting","tags":{},"title":"[Alerting] db exception alert"}

xmx0632 · 2020-06-29T03:00:14Z

这些就是grafana的json

{"evalMatches":[],"message":"nacos db exception!!!","ruleId":27,"ruleName":"db exception alert","ruleUrl":"http://localhost:3000/d/Bz_QALEiz12/nacos-1?fullscreen\u0026edit\u0026tab=alert\u0026panelId=54\u0026orgId=1","state":"alerting","tags":{},"title":"[Alerting] db exception alert"}

我在模版测试页面尝试发送消息会报错。可能是哪里的问题呢？多谢

feiyu563 · 2020-06-29T03:04:24Z

{{ $var := .externalURL}}{{ range $k,$v:=.alerts }}
{{if eq $v.status "resolved"}}
## [Prometheus恢复信息]({{$v.generatorURL}})
{{ range $x,$y:=$v.labels }}
###### {{ $x }}： {{ $y }}
{{end}}
###### 开始时间：{{$v.startsAt}}
###### 结束时间：{{$v.endsAt}}
##### [{{$v.annotations.description}}]({{$var}})
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)
{{else}}
## [Prometheus告警信息]({{$v.generatorURL}})
{{ range $x,$y:=$v.labels }}
###### {{ $x }}： {{ $y }}
{{end}}
###### 开始时间：{{$v.startsAt}}
###### 结束时间：{{$v.endsAt}}
##### [{{$v.annotations.description}}]({{$var}})
![Prometheus](https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png)
{{end}}
{{ end }}

遍历所有labels的模版试试这个

feiyu563 · 2020-06-29T03:05:41Z

你上面那个是grafana的告警消息，要用grafana的模版，而你用的prometheus的模版肯定会报错

xmx0632 · 2020-06-29T03:10:16Z

你上面那个是grafana的告警消息，要用grafana的模版，而你用的prometheus的模版肯定会报错

看到了，多谢

xmx0632 · 2020-06-29T03:19:16Z

换了模版测试有问题

日志中报错，可能是bug吗？

2020/06/29 03:16:34.596 [D] [value.go:460] [1593400594596360963] {"evalMatches":[],"message":"nacos db exception!!!","ruleId":27,"ruleName":"db exception alert","ruleUrl":"http://www.baidu.com","state":"alerting","tags":{},"title":"[Alerting] db exception alert"}
2020/06/29 03:16:34.596 [C] [panic.go:679] the request url is /prometheusalert
2020/06/29 03:16:34.596 [C] [panic.go:679] Handler crashed with error runtime error: invalid memory address or nil pointer dereference
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/runtime/panic.go:679
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/text/template/exec.go:164
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/runtime/panic.go:679
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/runtime/panic.go:199
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/runtime/signal_unix.go:394
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/text/template/exec.go:218
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/text/template/exec.go:204
2020/06/29 03:16:34.596 [C] [panic.go:679] /mnt/hgfs/code/golang/src/PrometheusAlert/controllers/prometheusalert.go:38
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/reflect/value.go:460
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/reflect/value.go:321
2020/06/29 03:16:34.597 [C] [panic.go:679] /mnt/hgfs/code/golang/pkg/mod/github.com/astaxie/beego@v1.12.1/router.go:853
2020/06/29 03:16:34.597 [C] [panic.go:679] /usr/lib/golang/src/net/http/server.go:2802
2020/06/29 03:16:34.597 [C] [panic.go:679] /usr/lib/golang/src/net/http/server.go:1890
2020/06/29 03:16:34.597 [C] [panic.go:679] /usr/lib/golang/src/runtime/asm_amd64.s:1357
2020/06/29 03:16:34.597 [server.go:3054] [HTTP] http: superfluous response.WriteHeader call from github.com/astaxie/beego/context.(*Response).WriteHeader (context.go:230)

imlxh · 2020-06-29T03:20:09Z

好的，我试下，非常感谢

xmx0632 · 2020-06-29T16:18:00Z

换了模版测试有问题

日志中报错，可能是bug吗？

2020/06/29 03:16:34.596 [D] [value.go:460] [1593400594596360963] {"evalMatches":[],"message":"nacos db exception!!!","ruleId":27,"ruleName":"db exception alert","ruleUrl":"http://www.baidu.com","state":"alerting","tags":{},"title":"[Alerting] db exception alert"}
2020/06/29 03:16:34.596 [C] [panic.go:679] the request url is /prometheusalert
2020/06/29 03:16:34.596 [C] [panic.go:679] Handler crashed with error runtime error: invalid memory address or nil pointer dereference
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/runtime/panic.go:679
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/text/template/exec.go:164
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/runtime/panic.go:679
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/runtime/panic.go:199
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/runtime/signal_unix.go:394
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/text/template/exec.go:218
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/text/template/exec.go:204
2020/06/29 03:16:34.596 [C] [panic.go:679] /mnt/hgfs/code/golang/src/PrometheusAlert/controllers/prometheusalert.go:38
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/reflect/value.go:460
2020/06/29 03:16:34.596 [C] [panic.go:679] /usr/lib/golang/src/reflect/value.go:321
2020/06/29 03:16:34.597 [C] [panic.go:679] /mnt/hgfs/code/golang/pkg/mod/github.com/astaxie/beego@v1.12.1/router.go:853
2020/06/29 03:16:34.597 [C] [panic.go:679] /usr/lib/golang/src/net/http/server.go:2802
2020/06/29 03:16:34.597 [C] [panic.go:679] /usr/lib/golang/src/net/http/server.go:1890
2020/06/29 03:16:34.597 [C] [panic.go:679] /usr/lib/golang/src/runtime/asm_amd64.s:1357
2020/06/29 03:16:34.597 [server.go:3054] [HTTP] http: superfluous response.WriteHeader call from github.com/astaxie/beego/context.(*Response).WriteHeader (context.go:230)

debug后发现是grafana原来设置的模版中包含time方法调用导致的，删除包含time的调用之后就正常了。

Grafana告警信息

{{.ruleName}}

告警级别：严重

开始时间：{{ (time.Now).Format time.Kitchen }}{{ ((time.Now).Add (time.Hour 2)).Format time.Kitchen }}

{{.message}}

与传入的消息转换时没有time对象导致返回 nil 报错，在

PrometheusAlert/controllers/prometheusalert.go

Line 37 in 92ce2eb

tpl,_:=template.New("").Parse(tpltext.Tpl)

中打印转换异常可以看到报错信息

tpl,tplErr:=template.New("").Parse(tpltext.Tpl)
if tplErr != nil {
logs.Error("parse tpl error",tplErr)
}

xmx0632 · 2020-06-29T16:21:29Z

接收到的Grafana消息为：

{"evalMatches":[{"value":0,"metric":"up{Category=\"micro-service\", Env=\"prod\", Name=\"fabric-sidecar-1\", Usage=\"fabric-sidecar\", instance=\"10.170.0.11:9876\", job=\"micro-service\"}","tags":{"Category":"micro-service","Env":"prod","Name":"fabric-sidecar-1","Usage":"fabric-sidecar","__name__":"up","instance":"10.170.0.11:9876","job":"micro-service"}}],"message":"微服务挂了！请联系管理员检查！","ruleId":37,"ruleName":"MicroService节点状态告警","ruleUrl":"http://localhost:3000/d/yCQyX6ZMz/blockchainservicemonitor?fullscreen\u0026edit\u0026tab=alert\u0026panelId=8\u0026orgId=1","state":"alerting","tags":{},"title":"[Alerting] MicroService节点状态告警"}

对应的 Grafana 告警模版改为：


## [Grafana告警信息]({{.ruleUrl}})
#### {{.ruleName}}
###### 告警级别：严重
##### {{.message}}
##### {{.value}}
{{range $i, $v := .evalMatches}}
{{ if $v.tags.Category }}
##### Category: {{ $v.tags.Category }}
{{end}}
{{ if $v.tags.Usage }}
##### Usage: {{ $v.tags.Usage }}
{{end}}
{{ if $v.tags.instance }}
##### Instance: {{ $v.tags.instance }}
{{end}}
{{end}}

feiyu563 · 2020-06-29T16:53:05Z

额，这是之前测试时候忘记删除了

…

---原始邮件--- 发件人: "xmx0632"<notifications@github.com> 发送时间: 2020年6月30日(周二) 凌晨0:21 收件人: "feiyu563/PrometheusAlert"<PrometheusAlert@noreply.github.com>; 抄送: "Mention"<mention@noreply.github.com>;"feiyu563"<244217140@qq.com>; 主题: Re: [feiyu563/PrometheusAlert] 修改了模板没有生效 (#37) 接收到的Grafana消息为： {"evalMatches":[{"value":0,"metric":"up{Category=\"micro-service\", Env=\"prod\", Name=\"fabric-sidecar-1\", Usage=\"fabric-sidecar\", instance=\"10.170.0.11:9876\", job=\"micro-service\"}","tags":{"Category":"micro-service","Env":"prod","Name":"fabric-sidecar-1","Usage":"fabric-sidecar","__name__":"up","instance":"10.170.0.11:9876","job":"micro-service"}}],"message":"微服务挂了！请联系管理员检查！","ruleId":37,"ruleName":"MicroService节点状态告警","ruleUrl":"http://localhost:3000/d/yCQyX6ZMz/blockchainservicemonitor?fullscreen\u0026edit\u0026tab=alert\u0026panelId=8\u0026orgId=1","state":"alerting","tags":{},"title":"[Alerting] MicroService节点状态告警"} 对应的 Grafana 告警模版改为： ## [Grafana告警信息]({{.ruleUrl}}) #### {{.ruleName}} ###### 告警级别：严重 ##### {{.message}} ##### {{.value}} {{range $i, $v := .evalMatches}} {{ if $v.tags.Category }} ##### Category: {{ $v.tags.Category }} {{end}} {{ if $v.tags.Usage }} ##### Usage: {{ $v.tags.Usage }} {{end}} {{ if $v.tags.instance }} ##### Instance: {{ $v.tags.instance }} {{end}} {{end}} — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

feiyu563 · 2020-07-07T02:35:34Z

issue我先关闭了

Zhang21 · 2020-07-07T06:39:51Z

@feiyu563 @xmx0632 请问两位的alertmanger是怎么配置的？

我按照文档配置，那不是wxurl已经被写死了？那自己配置的其它机器人是否就不生效了？还有就是，app.conf里面配置是否也不生效了？时间转换这些都不生效。

找到需要使用的自定义消息模版，复制表格中路径一列的地址内容，并将地址中[xxxxx]中的地址或手机号替换成你实际的配置，将其粘贴到对应的WebHook地址配置中即可。

我只是像使用自定义的模板，但我还是需要继续使用app.conf里面的内容。关于多个机器人，关于时区等等。怎么解决？

Zhang21 · 2020-07-07T06:45:24Z

我能不能只填写模板地址，不填写其它配置，让它从app.conf里取读取？

Zhang21 · 2020-07-07T07:15:49Z

我使用sqlite3连数据库都改了，使用默认的接口还是不生效。

这个默认的告警模板是写死在代码里面的了吗？我连数据库都改了，哎！

pokitpeng · 2021-12-10T05:50:30Z

+1 搞了半天，发现白忙活一场

feiyu563 closed this as completed Jul 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

修改了模板没有生效 #37

修改了模板没有生效 #37

imlxh commented Jun 28, 2020

feiyu563 commented Jun 28, 2020 via email

imlxh commented Jun 29, 2020

feiyu563 commented Jun 29, 2020 via email

imlxh commented Jun 29, 2020

imlxh commented Jun 29, 2020

feiyu563 commented Jun 29, 2020

imlxh commented Jun 29, 2020

xmx0632 commented Jun 29, 2020

feiyu563 commented Jun 29, 2020

xmx0632 commented Jun 29, 2020

feiyu563 commented Jun 29, 2020 •

edited

feiyu563 commented Jun 29, 2020

xmx0632 commented Jun 29, 2020 •

edited

xmx0632 commented Jun 29, 2020

imlxh commented Jun 29, 2020

xmx0632 commented Jun 29, 2020

Grafana告警信息

{{.ruleName}}

告警级别：严重

开始时间：{{ (time.Now).Format time.Kitchen }}{{ ((time.Now).Add (time.Hour 2)).Format time.Kitchen }}

{{.message}}

xmx0632 commented Jun 29, 2020

feiyu563 commented Jun 29, 2020 via email

feiyu563 commented Jul 7, 2020

Zhang21 commented Jul 7, 2020

Zhang21 commented Jul 7, 2020

Zhang21 commented Jul 7, 2020

pokitpeng commented Dec 10, 2021

修改了模板没有生效 #37

修改了模板没有生效 #37

Comments

imlxh commented Jun 28, 2020

feiyu563 commented Jun 28, 2020 via email

imlxh commented Jun 29, 2020

feiyu563 commented Jun 29, 2020 via email

imlxh commented Jun 29, 2020

imlxh commented Jun 29, 2020

feiyu563 commented Jun 29, 2020

imlxh commented Jun 29, 2020

xmx0632 commented Jun 29, 2020

feiyu563 commented Jun 29, 2020

xmx0632 commented Jun 29, 2020

feiyu563 commented Jun 29, 2020 • edited

feiyu563 commented Jun 29, 2020

xmx0632 commented Jun 29, 2020 • edited

xmx0632 commented Jun 29, 2020

imlxh commented Jun 29, 2020

xmx0632 commented Jun 29, 2020

Grafana告警信息

{{.ruleName}}

告警级别：严重

开始时间：{{ (time.Now).Format time.Kitchen }}{{ ((time.Now).Add (time.Hour 2)).Format time.Kitchen }}

{{.message}}

xmx0632 commented Jun 29, 2020

feiyu563 commented Jun 29, 2020 via email

feiyu563 commented Jul 7, 2020

Zhang21 commented Jul 7, 2020

Zhang21 commented Jul 7, 2020

Zhang21 commented Jul 7, 2020

pokitpeng commented Dec 10, 2021

feiyu563 commented Jun 29, 2020 •

edited

xmx0632 commented Jun 29, 2020 •

edited