Slack Notifications Ignored by alarm-notify.sh

[hibbie]

Hello, I'm having difficulties with Netdata alarms going to Slack (or anywhere else). The server is behind an Nginx proxy but other than that it is a fairly pedestrian install on Ubuntu 18.04. The rest of Netdata works fine. It's just when I call the alarm script to test I receive the following:

su -s /bin/bash netdata
export NETDATA_ALARM_NOTIFY_DEBUG=1
/usr/libexec/netdata/plugins.d/alarm-notify.sh test

# SENDING TEST WARNING ALARM TO ROLE: sysadmin
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Loading config file '/usr/lib/netdata/conf.d/health_alarm_notify.conf'...
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Loading config file '/etc/netdata/health_alarm_notify.conf'...
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Cannot find sendmail command in the system path. Disabling email notifications.
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Cannot find aws command in the system path.  Disabling Amazon SNS notifications.
2019-01-24 16:22:38: alarm-notify.sh: FATAL: All notification methods are disabled. Not sending notification for host 'meriadoc', chart 'test.chart' to 'sysadmin' for 'test_alarm' = '100' for status 'WARNING'.
# FAILED

# SENDING TEST CRITICAL ALARM TO ROLE: sysadmin
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Loading config file '/usr/lib/netdata/conf.d/health_alarm_notify.conf'...
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Loading config file '/etc/netdata/health_alarm_notify.conf'...
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Cannot find sendmail command in the system path. Disabling email notifications.
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Cannot find aws command in the system path.  Disabling Amazon SNS notifications.
2019-01-24 16:22:38: alarm-notify.sh: FATAL: All notification methods are disabled. Not sending notification for host 'meriadoc', chart 'test.chart' to 'sysadmin' for 'test_alarm' = '100' for status 'CRITICAL'.
# FAILED

# SENDING TEST CLEAR ALARM TO ROLE: sysadmin
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Loading config file '/usr/lib/netdata/conf.d/health_alarm_notify.conf'...
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Loading config file '/etc/netdata/health_alarm_notify.conf'...
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Cannot find sendmail command in the system path. Disabling email notifications.
2019-01-24 16:22:38: alarm-notify.sh: DEBUG: Cannot find aws command in the system path.  Disabling Amazon SNS notifications.
2019-01-24 16:22:38: alarm-notify.sh: FATAL: All notification methods are disabled. Not sending notification for host 'meriadoc', chart 'test.chart' to 'sysadmin' for 'test_alarm' = '100' for status 'CLEAR'.
# FAILED

My Slack settings in health_alarm_notify.conf look like the following:

SEND_SLACK="YES"
SLACK_WEBHOOK_URL="https://hooks.slack.com/services/T*****/B*****/D*****"
DEFAULT_RECIPIENT_SLACK="#mychannelname"

I'm also able to manually send a test via curl to Slack that displays fine. I did try setting the various SEND_ settings to NO and YES to no avail. It seems very stuck on denying any and all services from working. Obvious gotcha like the web hook or channel being incorrect or curl not working don't seem to be at fault. The bash script doesn't seem to process my settings at all. I'm about to attempt a re-installation otherwise.

[cakrit]

Thanks for the thorough report. Please go through #5261, we just fixed this issue a few hours ago.

[hibbie]

Much appreciated!