Alertmanaged notify stoped working

[cloudziu]

Alertmanager stoped sending alerts to msteams with an error:

Current version: quay.io/prometheus/alertmanager:v0.27.0

2024-12-04T13:43:07.050Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="default/config-example/test/msteams[0]: notify retry canceled due to unrecoverable error after 1 attempts: template: :2: unexpected EOF"

It was working the day before, and there was no configuration changes in alertmaanger and/or environment.

Steps that I did to debug:

• wget from the Node where alermanager is running - passes,
• wget from a different Pod - passes
• wget from a debug container in Alertmanager pod - passes
• wget from the alertmanager pod - failes
• I bumped the container version to 0.28.0-rc.0 - didn't work
• I've mounted /etc/ssl/certs/ - didn't work

The failed output:

wget "https://prod-215.westeurope.logic.azure.com:443/workflows/..."
Connecting to prod-215.westeurope.logic.azure.com:443 (108.142.31.202:443)
wget: note: TLS certificate validation not implemented
wget: got bad TLS record (len:0) while expecting handshake record
wget: error getting response: Connection reset by peer
wget: got bad TLS record (len:0) while expecting handshake record
wget: error getting response: Connection reset by peer

Any ideas?

[cloudziu]

It was an issue with template,. closing