Listen for alerts from Loggly and publish to Slack.
Loggly includes an integration with Slack to do this directly, however this integration has several deficiencies:
- It includes "@channel" which forces all users to be alerted (unless they mute the channel)
- All data for all log entries matching the alert are dumped into the alert message - in the case of JSON data it would be better to summarise only the key attributes of the log entry (i.e. message, etc)
- Log entries matching the alert are repeated in later alerts
This simple script attempts to format the Slack messages in a friendlier way. Key information is extracted from log messages and an attempt is made to skip repeated messages.
The JSON log entries sent by Loggly are sent escaped inside a JSON container. Once extracted, the log entries are rarely well-formed JSON. Most of the complexity in this script are attempts to remove invalid objects/strings in the JSON before attempting to parse is.
- Serverless (https://serverless.com/framework/docs/getting-started/)
- NodeJS 6.5+ & NPM
- AWS key
npm install
cp env.yaml.sample env.yaml
## replace env.yaml Slack keys with valid values:
## SLACK_TOKEN, SLACK_CHANNEL, SLACK_USERNAME
## populate test-data-json.txt with data from loggly endpoint
sls invoke local -f alert -p test-data-json.txt
## obtain aws access key pair
export AWS_ACCESS_KEY_ID=<your-key-here>
export AWS_SECRET_ACCESS_KEY=<your-secret-key-here>
## perform serverless deploy
sls deploy -s dev
## perform serverless deploy
sls deploy -s prod
Additonal stages can be created with different env data to target different Slack channels.
Setup a new Loggly Alert Endpoint at the following URL: https://.loggly.com/alerts/endpoint
The endpoint type should be HTTP/S, the URL should be set to this serverless endpoint and the HTTP Method should be set to "POST".
Assign this Alert Endpoint to one or more Loggly alerts. Once an alert is triggered, a message should be sent to Slack via this serverless endpoint.