Multiple slack notifications

[seanmcrw]

What happened:
Default installation on k8s(AWS EKS x7 EC2 nodes in cluster) with helm helm install --name falco stable/falco creates x7 duplicate notifications when using the slack integration.

What you expected to happen:
Only one notification should be posted to slack.

How to reproduce it (as minimally and precisely as possible):
Default helm install, with slack notifications enabled.

jsonOutput: true
  ....
    programOutput:
    enabled: true
    keepAlive: false
    program: "\"jq '{text: .output}' | curl -d @- -X POST https://hooks.slack.com/services/xxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxxxx\""

Anything else we need to know?:

Environment:

• Falco version (use falco --version):
  falco version 0.15.3
• System info

{
  "machine": "x86_64",
  "nodename": "falco-6l9sz",
  "release": "4.14.146-119.123.amzn2.x86_64",
  "sysname": "Linux",
  "version": "#1 SMP Mon Sep 23 16:58:43 UTC 2019"
}

• Cloud provider or hardware configuration:
  AWS
• OS (e.g: cat /etc/os-release):

NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
Amazon Linux release 2 (Karoo)
cpe:2.3:o:amazon:amazon_linux:2

• Kernel (e.g. uname -a):

Linux ip-10-0-73-85.eu-west-1.compute.internal 4.14.146-119.123.amzn2.x86_64 #1 SMP Mon Sep 23 16:58:43 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

• Install tools (e.g. in kubernetes, rpm, deb, from source):
• Others:

[Issif]

For more advanced notifications in Slack, you can take a look at : https://github.com/falcosecurity/falcosidekick

[seanmcrw]

Thanks, I've tried falcosidekick, but I still get x7 duplicate notifications(same number of nodes in the ds) for each triggered rule.

[Issif]

Your notifications are about a k8s event from audit logs? Seems logical that all pods of the daemonset detect it.

[seanmcrw]

This is a typical event that's repeated x7 times

14:43:20.636017541: Error File below /etc opened for writing (user=root command=exe / /var/lib/docker/overlay2/28f02df2e5c37c18e31d6a2b5795a1139d7d88464c4da3d38f2ab9709863dadf/diff parent=dockerd pcmdline=dockerd file=/etc/shadow program=exe gparent=systemd ggparent=<NA> gggparent=<NA>) k8s.ns=<NA> k8s.pod=<NA> container=host

On a different note I don't see the pod name or container name(host being a red herring #865 #629)

k8s.pod=<NA> container=host

[Issif]

Another user reported that kind of issue on Slack. cc @JPLachance. Have the name of related rule which triggers that?

For missing fields, it's known and people are figuring out how to solve that.

[fntlnz]

@seanmcrw the version of falco you are using is quite old 0.15.3. We had a lot of rules changes in 0.18.0 and every other version after that one and I believe this problem was solved by one of these #733 #736 #881 - could you try those out?

[seanmcrw]

Thanks guys, using 0.18.0 solved the problem. Closing.

[fntlnz]

Thanks for confirming @seanmcrw !