-
Notifications
You must be signed in to change notification settings - Fork 205
[8.16] [Cloud Security] Third-party cloud data #6046
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
2ff4b69
[ESS] [8.16] Ingest Sysdig Falco data to Elasticsearch
benironside e14c668
fixes build error
benironside 63a1854
fixes kibana reference error
benironside 8c5c0d6
adds information about 3p workflows to cloud native sec section
benironside 7b9df3a
fix build error
benironside cc87e8b
save my work
benironside 22a1fef
Incorporates Nick's review
benironside 2f834c0
format tweaks for falco doc
benironside 58bdfc4
Incorporates Nick's second review
benironside cec408b
Incorporates Joe's review
benironside b09e43b
Merge branch 'main' into 5607-3p-cloud-data-8.16
benironside ec0b3a4
Update docs/cloud-native-security/wiz.asciidoc
benironside File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| [[ingest-aws-securityhub-data]] | ||
| = Ingest AWS Security Hub data | ||
|
|
||
| In order to enrich your {elastic-sec} workflows with third-party cloud security posture data collected by AWS Security Hub: | ||
|
|
||
| * Follow the steps to {integrations-docs}/aws/securityhub[set up the AWS Security Hub integration]. | ||
|
|
||
| * Make sure the integration version is at least 2.31.1. | ||
|
|
||
| * Ensure you have `read` privileges for the `security_solution-*.misconfiguration_latest` index. | ||
|
|
||
| * While configuring the AWS Security Hub integration, turn on **Collect AWS Security Hub Findings from AWS**. We recommend you also set the **Initial Interval** value to `2160h` (equivalent to 90 days) to ingest existing logs. | ||
|
|
||
| image::images/aws-config-finding-logs.png[AWS Security Hub integration settings showing the findings toggle] | ||
|
|
||
| After you've completed these steps, AWS Security Hub data will appear on the Misconfigurations tab of the <<cspm-findings-page, Findings>> page. | ||
|
|
||
| Any available findings data will also appear in the entity details flyouts for related <<insights-section, alerts>>. If alerts are present for a user or host that has findings data from AWS Security Hub, the findings will appear on the <<user-details-flyout,users>>, and <<host-details-flyout,hosts>> flyouts. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,115 @@ | ||
| [[ingest-falco]] | ||
| = Ingest CNCF Falco data | ||
|
|
||
| CNCF Falco is an open-source runtime security tool that detects anomalous activity in Linux hosts, containers, Kubernetes, and cloud environments. You can ingest Falco alerts into {es} to view them on {elastic-sec}'s Alerts page and incorporate them into your security workflows by using Falcosidekick, a proxy forwarder which can send alerts from your Falco deployments to {es}. | ||
|
|
||
| First, you'll need to configure {elastic-sec} to receive data from Falco, then you'll need to configure Falco and Falcosidekick to send data to {es}. | ||
|
|
||
| [discrete] | ||
| [[ingest-falco-setup-kibana]] | ||
| == Configure {elastic-sec} to receive Falco data | ||
|
|
||
| In {elastic-sec}: | ||
|
|
||
| . Click **Add integrations**. | ||
| . Search the Integrations page for `Falco`, then select it. | ||
| . Go to the Falco integration's **Settings** tab. | ||
| . Click **Install Falco**, then confirm by clicking **Install Falco** again. Installation should take less than a minute. | ||
|
|
||
| {elastic-sec} is now ready to receive data from Falco. The Falco integration page now has an **Assets** tab where you can inspect the newly installed assets that help to ingest Falco data. | ||
|
|
||
| Next, to make alerts from Falco appear on {elastic-sec}'s Alerts page: | ||
|
|
||
| . Find the **Rules** page in the navigation menu or by using the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. | ||
| . Click **Detection rules (SIEM)**. | ||
| . Search for a rule called `External Alerts`. Install it if necessary, and enable it. | ||
|
|
||
|
|
||
| [discrete] | ||
| [[ingest-falco-setup-falco]] | ||
| == Configure Falco and Falcosidekick | ||
|
|
||
| You can either: | ||
|
|
||
| * <<ingest-falco-setup-falco-vm,Send Falco data to {es} from virtual machines (VMs)>>; or, | ||
| * <<ingest-falco-setup-falco-kubernetes,Send Falco data to {es} from Kubernetes>>. | ||
|
|
||
| [discrete] | ||
| [[ingest-falco-setup-falco-vm]] | ||
| === Configure Falco and Falcosidekick for VMs | ||
|
|
||
| Multiple methods for configuring Falco to send data from VMs to {es} are available. This guide uses the https://github.com/falcosecurity/falcosidekick/blob/master/docs/outputs/elasticsearch.md[Falco sidekick on Docker using environment variables] method. | ||
|
|
||
| [discrete] | ||
| === Configure Falco for VMs: | ||
|
|
||
| . Refer to Falco's documentation to https://falco.org/docs/setup/packages/[install Falco on the Linux VMs you wish to monitor]. | ||
| . Once Falco is installed, update `/etc/falco/falco.yaml` as follows: | ||
| .. Enable JSON output: `json_output: true` | ||
| .. Enable HTTP output: under `http_output`, for the `url` value, enter the `url:port` where Falcosidekick will listen. For example, if Falcosidekick is running on localhost: | ||
| + | ||
| ``` | ||
| http_output: | ||
| enabled: true | ||
| url: "http://0.0.0.0:2801/" | ||
| ``` | ||
|
|
||
| [discrete] | ||
| [[falco-config-falco-for-vms]] | ||
| === Configure Falcosidekick for VMs: | ||
|
|
||
| . Refer to Falcosidekick's documentation to https://github.com/falcosecurity/falcosidekick?tab=readme-ov-file#installation[install Falcosidekick]. | ||
| . Use the https://github.com/falcosecurity/falcosidekick/blob/master/docs/outputs/elasticsearch.md[Falcosidekick on Docker using environment variables] method and set your environment variables as follows: | ||
| .. `ELASTICSEARCH_HOSTPORT`: Your {es} endpoint URL, which can be found under **Connection details** on the upper right of the **Integrations** page in {kib}. | ||
| .. `ELASTICSEARCH_INDEX`: The {es} index where you want to store Falco logs. | ||
| + | ||
| IMPORTANT: Your `ELASTICSEARCH_INDEX` value must match `logs-falco.alerts-*`. | ||
| + | ||
| .. `ELASTICSEARCH_SUFFIX`: The frequency with which you want the {es} index suffix to change. Either `daily`, `monthly`, `annually`, or `none`. | ||
| .. `ELASTICSEARCH_APIKEY`: The recommended way to authenticate to {es}, by providing an {kibana-ref}/api-keys.html[API key]. Note that support for this environment variable starts with Falcosidekick version 2.30. You can access the latest version on Falcosidekick's https://hub.docker.com/r/falcosecurity/falcosidekick[Docker Hub]. | ||
| .. `ELASTICSEARCH_USERNAME` and `ELASTICSEARCH_PASSWORD`: The username and password for an account on your {es} instance. Authentication using these environment variables is not supported on {ecloud} Serverless. | ||
| .. `ELASTICSEARCH_MUTUALTLS` and `ELASTICSEARCH_CHECKCERT`: For security reasons, we recommend setting these to `true`. | ||
|
|
||
| For example: | ||
|
|
||
| ``` | ||
| docker run -d -p 2801:2801 | ||
| -e ELASTICSEARCH_HOSTPORT=https://test-falco.es.us-west2.gcp.elastic-cloud.com | ||
| -e ELASTICSEARCH_INDEX=logs-falco.alerts-all | ||
| -e ELASTICSEARCH_SUFFIX=none | ||
| -e ELASTICSEARCH_APIKEY=XXXXXXXXXXXXX | ||
| -e ELASTICSEARCH_MUTUALTLS=true | ||
| -e ELASTICSEARCH_CHECKCERT=true falcosecurity/falcosidekick | ||
| ``` | ||
|
|
||
| IMPORTANT: The {es} account used to authenticate Falcosidekick only needs sufficient privileges to create and write to new indices. We recommend following the principle of least privilege when provisioning this account. | ||
|
|
||
| After installing and configuring Falcosidekick, restart Falco with `sudo systemctl restart falco`. Falcosidekick should start sending alerts to {es}. | ||
|
|
||
|
|
||
| [discrete] | ||
| [[ingest-falco-setup-falco-kubernetes]] | ||
| == Configure Falco and Falcosidekick for Kubernetes | ||
|
|
||
| 1. Add the Falco https://github.com/falcosecurity/charts/blob/master/README.md[Helm charts]: | ||
| + | ||
| ``` | ||
| helm repo add falcosecurity https://falcosecurity.github.io/charts | ||
| helm repo update | ||
| ``` | ||
| + | ||
| 2. Next, install Falco and Falcosidekick using the `falcosecurity/falco` Helm chart with https://github.com/falcosecurity/falcosidekick/blob/master/docs/outputs/elasticsearch.md[appropriate values] for each of the `falcosidekick.config.elasticsearch.*` fields: | ||
| + | ||
| ``` | ||
| helm install falco falcosecurity/falco \ | ||
| --set falcosidekick.enabled=true \ | ||
| --set tty=true \ | ||
| --set driver.kind=modern_ebpf \ | ||
| --set collectors.kubernetes.enabled=true \ | ||
| --set falcosidekick.config.elasticsearch.hostport="https://<ES host>" \ | ||
| --set falcosidekick.config.elasticsearch.username="<elastic>" \ | ||
| --set falcosidekick.config.elasticsearch.password="<password>" \ | ||
| --set falcosidekick.config.elasticsearch.index="logs-falco.alerts-all" \ | ||
| --set falcosidekick.config.elasticsearch.suffix="none" | ||
| ``` | ||
|
|
||
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| [[ingest-third-party-cloud-security-data]] | ||
| = Ingest third-party cloud security data | ||
|
|
||
| This section describes how to ingest cloud security data from third-party tools into {es}. Once ingested, this data can provide additional context and enrich your {elastic-sec} workflows. | ||
|
|
||
| You can ingest both third-party cloud workload protection data and third-party security posture and vulnerability data. | ||
|
|
||
| [discrete] | ||
| == Ingest third-party workload protection data | ||
|
|
||
| You can ingest third-party cloud security alerts into {elastic-sec} to view them on the <<alerts-page>> and incorporate them into your triage and threat hunting workflows. | ||
|
|
||
| * Learn to <<ingest-falco, ingest alerts from Sysdig Falco>>. | ||
|
|
||
| [discrete] | ||
| == Ingest third-party security posture and vulnerability data | ||
|
|
||
| You can ingest third-party data into {elastic-sec} to review and investigate it alongside data collected by {elastic-sec}'s native cloud security integrations. Once ingested, cloud security posture and vulnerability data appears on the <<cspm-findings-page,Findings>> page, on the <<cspm-posture-dashboard, Cloud Posture dashboard>>, and in the entity details flyouts for <<insights-section, alerts>>, <<user-details-flyout,users>>, and <<host-details-flyout,hosts>>. | ||
|
|
||
| * Learn to <<ingest-aws-securityhub-data, ingest cloud security posture data from AWS Security Hub>>. | ||
|
|
||
| * Learn to <<ingest-wiz-data, ingest cloud security posture and vulnerability data from Wiz>>. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| [[ingest-wiz-data]] | ||
| = Ingest Wiz data | ||
|
|
||
| In order to enrich your {elastic-sec} workflows with third-party cloud security posture and vulnerability data collected by Wiz: | ||
|
|
||
| * Follow the steps to {integrations-docs}/wiz[set up the Wiz integration]. | ||
|
|
||
| * Make sure the integration version is at least 2.0.1. | ||
|
|
||
| * Ensure you have `read` privileges for the following indices: `security_solution-*.misconfiguration_latest`, `security_solution-*.vulnerability_latest`. | ||
|
|
||
| * While configuring the Wiz integration, turn on **Cloud Configuration Finding logs** and **Vulnerability logs**. We recommend you also set the **Initial Interval** values for both settings to `2160h` (equivalent to 90 days) to ingest existing logs. | ||
|
|
||
| image::images/wiz-config-finding-logs.png[Wiz integration settings showing the findings toggle] | ||
|
|
||
| image::images/wiz-config-vuln-logs.png[Wiz integration settings showing the vulnerabilities toggle] | ||
|
|
||
| After you've completed these steps, Wiz data will appear on the <<cspm-findings-page, Misconfiguations>> and <<vuln-management-findings, Vulnerabilities>> tabs of the Findings page. | ||
|
|
||
| image::images/wiz-findings.png[Wiz data on the Findings page] | ||
|
|
||
| Any available findings data will also appear in the entity details flyouts for related <<insights-section, alerts>>. If alerts are present for a user or host that has findings data from Wiz, the findings will appear on the <<user-details-flyout,users>>, and <<host-details-flyout,hosts>> flyouts. | ||
|
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.