Don't use non-existent webhook as a placeholder

[nobuto-m]

Issue

Fixes: #237

Solution

The fake webhook fires the AlertmanagerNotificationsFailed alert continuously out of the box. Instead, use a blackhole receiver without any backend as a placeholder when no receiver is provided by an operator.

Context

[before patching]

global:
  http_config:
    tls_config:
      insecure_skip_verify: false
receivers:
- name: placeholder
  webhook_configs:
  - url: http://127.0.0.1:5001/
route:
  group_by:
  - juju_model
  - juju_application
  - juju_model_uuid
  group_interval: 5m
  group_wait: 30s
  receiver: placeholder
  repeat_interval: 1h

[after patching]

global:
  http_config:
    tls_config:
      insecure_skip_verify: false
receivers:
- name: placeholder
route:
  group_by:
  - juju_application
  - juju_model
  - juju_model_uuid
  group_interval: 5m
  group_wait: 30s
  receiver: placeholder
  repeat_interval: 1h

[After patching]

[The fix was applied in the middle of this timeline]

Testing Instructions

The steps are in #237.

1. deploy COS and workload
2. related COS with workload
3. check alerts

Upgrade Notes

N/A

[nobuto-m]

More context:

• Blackhole receiver prometheus/alertmanager#428
• https://www.unix-experience.fr/en/monitoring/alertmanager_ignore_alert/

[sed-i]

This is awesome @nobuto-m, I was not aware of this functionality!
Thanks for the change.