[DOCS] Add Endpoint troubleshooting document: Linux deadlock

[ferullo]

Description

We see that Endpoint users come across similar issues over and over. It would be nice to have a trouble shooting document that describes how to recognize and fix common problems.

Acceptance Test Criteria

Add the doc :)

Notes

To start, can we add troubleshooting for when Endpoint's policy fails due to potential system deadlock avoidance on Linux. Someone will follow up in the comments of this issue to described what needs to be document (assuming the feature that needs documenting ships in 8.4).

cc @nfritts @joeypoon

[joepeeples]

Note to self: Once docs are ready and published, we also need to revise the error message and docs link in the UI. Per elastic/kibana#136241, currently the message is minimal and the link points to the main page (instead of a targeted link to the anchor of the new t'shooting section).

[nicholasberlin]

@joepeeples hopefully there isn't too much formatting in the following blurb. let me know what is confusing/needs clarification, etc. Thanks.

Once a policy response contains Disabled due to potential system deadlock, Linux Malware will continue to be disabled until a related advanced policy option is updated.¹

Starting in 8.4 there are three advanced options for controlling Linux Malware behavior:

1. linux.advanced.fanotify.ignore_unknown_filesystems
2. linux.advanced.fanotify.monitored_filesystems
3. linux.advanced.fanotify. ignored_filesystems

Once adjusted, saving the Policy will re-enable Linux Malware.

linux.advanced.fanotify.ignore_unknown_filesystems

---

Whether fanotify should ignore unknown filesystems.

true:
Only CI tested filesystems will be marked by default. Additional filesystems can be added or removed with monitored_filesystems and ignored_filesystems, respectively.

false:
Only an internally curated list of filesystems will be ignored, all others will be marked. Additional filesystems can be ignored via ignored_filesystems. monitored_filesystems is ignored when ignore_unknown_filesystems is false.

The default (when the option is blank) is true. In the case of an upgrade from 8.3 to 8.4, the value will explicitly be false in order to preserve previous behavior.

linux.advanced.fanotify.monitored_filesystems

---

Additional filesystems for fanotify to monitor.

This option's format is a comma separated list of filesystem names as they appear in /proc/filesystems, e.g. 'jfs,ufs,ramfs'. It is recommended to avoid network-backed filesystems.

ignore_unknown_filesystems is false:
This option is ignored.²

ignore_unknown_filesystems is true:
Parsed entries of this option are monitored by fanotify unless overridden by entries in ignored_filesystems or internally known bad filesystems.

linux.advanced.fanotify.ignored_filesystems

---

Additional filesystems for fanotify to ignore.

This option's format is a comma separated list of filesystem names as they appear in /proc/filesystems, e.g. ext4,tmpfs.

ignore_unknown_filesystems is false:
Parsed entries of this option supplement internally known bad filesystems to be ignored.

ignore_unknown_filesystems is true:
Parsed entries of this option override entries in monitored_filesystems and internally CI tested filesystems.

How to determine which filesystems to add

Assuming a typical setup, Filebeat will be installed alongside Elastic Endpoint. Filebeat will automatically ship Elastic Endpoint logs to Elasticsearch. Elastic Endpoint will generate a log message about which file was being scanned when the issue occurred.

From the Security -> Hosts page, searching for the message: "Current sync path" will reveal the file path.
If the user has access to the endpoint, running findmnt -o FSTYPE -T <file path> will return the filesystem.
For example,

> findmnt -o FSTYPE -T /etc/passwd
FSTYPE
ext4

ext4 could then be added to linux.advanced.fanotify.ignored_filesystems.

Alternatively, the filesystem can also be gathered by correlating data from two further log message searches:

1. First search message: "Current fdinfo" to reveal the mnt_id of file path

pos:	12288
flags:	02500002
mnt_id:	29
ino:	2367737

2. Second search message: "Current mountinfo" to reveal what filesystem corresponds to the mnt_id.

<snip>
29 1 8:2 / / rw,relatime shared:1 - ext4 /dev/sda2 rw,errors=remount-ro
<snip>

The first number, 29, is the mnt_id, and the first field after the - is the filesystem, ext4

Footnotes

1. Technically, Linux Malware will be re-enabled if Elastic Endpoint is restarted on the host, but this will not address the underlying issue, and will likely result in the same state. Simply restarting is not a recommended course of action. ↩

2. If this is only option changed when ignore_unknown_filesystem is false, then Linux Malware will not be re-enabled. ↩