[DOC] Add overall documentation and instructions on how to connect data sources for security analytics

[sandervandegeijn]

What do you want to do?

• Request a change to existing documentation
• Add new documentation
• Report a technical problem with the documentation
• Other

The security analytics team has built something that is potentially very valuable, but using it is quite hard. For example:

Okay, network events. Sounds interesting. I want to use it, questions that come to mind:

• Network events is quite general, what is it?
• Which data sources should I use?
• How do I connect these data sources?
• How do I connect these data sources effectively so mapping efforts are minimal?

The documentation is very minimal, there is very little attention paid to how data is ingested while this is the foundation of the functionality of this module.

Ideally you would relate this based on something like the NIST security framework, which identifies risks and gives tips on how to mitigate. You can use this to drill down in the documentation:

High level risk - monitoring category - how to connect the different data sources for this monitoring category.

I.e.
DE.CM-1: The network is monitored to detect potential cybersecurity events - Network detection Response - How to connect Suricata and what document structure would be ideal (plus some instruction for configuring fluent-bit, logstash, data prepper

Throw this is a table for all data monitoring categories and sources and this would be very helpful to the users. The framework can also guide the team on what sources to add and in what order.

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

https://www.nist.gov/cyberframework/framework

[sandervandegeijn]

I got the question to open a new issue in https://forum.opensearch.org/t/using-security-analytics-is-hard-mostly-because-of-a-lack-of-documentation/16271 and fond my own issue :) So I can just reference this issue there, this sums up the needs quite well. I had a really good call with Xenia. I'm open to plan another call if needed.

How to ingest logs in such a manner that it can be processed by the plugin is totally unclear. Which log ingesters to use, do you need certain index pipelines, how to map the data? For each log type the flow needs to be described end to end.