From 23a0a4266ec98b3f2ee303508735ac6d4c259e8e Mon Sep 17 00:00:00 2001 From: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com> Date: Tue, 12 Nov 2024 16:48:14 -0500 Subject: [PATCH 1/2] Updates nav references (#6147) (cherry picked from commit 3c3e4c825231843dd230947a57c8f3baece2d1e7) # Conflicts: # docs/serverless/AI-for-security/connect-to-azure-openai.asciidoc # docs/serverless/AI-for-security/connect-to-bedrock.asciidoc # docs/serverless/AI-for-security/connect-to-byo-llm.asciidoc # docs/serverless/AI-for-security/connect-to-openai.asciidoc # docs/serverless/AI-for-security/connect-to-vertex.asciidoc # docs/serverless/cloud-native-security/benchmark-rules.asciidoc # docs/serverless/cloud-native-security/cspm-get-started-azure.asciidoc # docs/serverless/cloud-native-security/cspm-get-started-gcp.asciidoc # docs/serverless/cloud-native-security/cspm-get-started.asciidoc # docs/serverless/cloud-native-security/d4c-get-started.asciidoc # docs/serverless/cloud-native-security/environment-variable-capture.asciidoc # docs/serverless/cloud-native-security/get-started-with-kspm.asciidoc # docs/serverless/cloud-native-security/kspm.asciidoc # docs/serverless/cloud-native-security/security-posture-management.asciidoc # docs/serverless/cloud-native-security/vuln-management-get-started.asciidoc --- .../connect-to-azure-openai.asciidoc | 2 +- .../connect-to-bedrock.asciidoc | 2 +- docs/AI-for-security/connect-to-byo.asciidoc | 2 +- .../connect-to-openai.asciidoc | 2 +- .../connect-to-vertex.asciidoc | 2 +- .../cspm-benchmark-rules.asciidoc | 2 +- .../cspm-get-started-aws.asciidoc | 4 +- .../cspm-get-started-azure.asciidoc | 5 +- .../cspm-get-started-gcp.asciidoc | 4 +- .../d4c-get-started.asciidoc | 7 +- .../environment-variable-capture.asciidoc | 7 +- .../kspm-benchmark-rules.asciidoc | 2 +- .../kspm-get-started.asciidoc | 4 +- docs/cloud-native-security/kspm.asciidoc | 2 +- .../security-posture-management.asciidoc | 4 +- .../vuln-management-get-started.asciidoc | 2 +- .../connect-to-azure-openai.asciidoc | 117 +++++ .../connect-to-bedrock.asciidoc | 167 +++++++ .../connect-to-byo-llm.asciidoc | 223 +++++++++ .../connect-to-openai.asciidoc | 70 +++ .../connect-to-vertex.asciidoc | 115 +++++ .../benchmark-rules.asciidoc | 61 +++ .../cspm-get-started-azure.asciidoc | 198 ++++++++ .../cspm-get-started-gcp.asciidoc | 205 ++++++++ .../cspm-get-started.asciidoc | 349 ++++++++++++++ .../d4c-get-started.asciidoc | 92 ++++ .../environment-variable-capture.asciidoc | 42 ++ .../get-started-with-kspm.asciidoc | 446 ++++++++++++++++++ .../cloud-native-security/kspm.asciidoc | 86 ++++ .../security-posture-management.asciidoc | 50 ++ .../vuln-management-get-started.asciidoc | 77 +++ 31 files changed, 2325 insertions(+), 26 deletions(-) create mode 100644 docs/serverless/AI-for-security/connect-to-azure-openai.asciidoc create mode 100644 docs/serverless/AI-for-security/connect-to-bedrock.asciidoc create mode 100644 docs/serverless/AI-for-security/connect-to-byo-llm.asciidoc create mode 100644 docs/serverless/AI-for-security/connect-to-openai.asciidoc create mode 100644 docs/serverless/AI-for-security/connect-to-vertex.asciidoc create mode 100644 docs/serverless/cloud-native-security/benchmark-rules.asciidoc create mode 100644 docs/serverless/cloud-native-security/cspm-get-started-azure.asciidoc create mode 100644 docs/serverless/cloud-native-security/cspm-get-started-gcp.asciidoc create mode 100644 docs/serverless/cloud-native-security/cspm-get-started.asciidoc create mode 100644 docs/serverless/cloud-native-security/d4c-get-started.asciidoc create mode 100644 docs/serverless/cloud-native-security/environment-variable-capture.asciidoc create mode 100644 docs/serverless/cloud-native-security/get-started-with-kspm.asciidoc create mode 100644 docs/serverless/cloud-native-security/kspm.asciidoc create mode 100644 docs/serverless/cloud-native-security/security-posture-management.asciidoc create mode 100644 docs/serverless/cloud-native-security/vuln-management-get-started.asciidoc diff --git a/docs/AI-for-security/connect-to-azure-openai.asciidoc b/docs/AI-for-security/connect-to-azure-openai.asciidoc index fbeffd1930..a84dec7cb5 100644 --- a/docs/AI-for-security/connect-to-azure-openai.asciidoc +++ b/docs/AI-for-security/connect-to-azure-openai.asciidoc @@ -101,7 +101,7 @@ The following video demonstrates these steps. Finally, configure the connector in {kib}: . Log in to {kib}. -. Go to **Stack Management → Connectors → Create connector → OpenAI**. +. Find the **Connectors** page in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. Then click **Create Connector**, and select **OpenAI**. . Give your connector a name to help you keep track of different models, such as `Azure OpenAI (GPT-4 Turbo v. 0125)`. . For **Select an OpenAI provider**, choose **Azure OpenAI**. . Update the **URL** field. We recommend doing the following: diff --git a/docs/AI-for-security/connect-to-bedrock.asciidoc b/docs/AI-for-security/connect-to-bedrock.asciidoc index 39ac30180d..2ee1ce2e84 100644 --- a/docs/AI-for-security/connect-to-bedrock.asciidoc +++ b/docs/AI-for-security/connect-to-bedrock.asciidoc @@ -147,7 +147,7 @@ The following video demonstrates these steps. Finally, configure the connector in {kib}: . Log in to {kib}. -. Go to **Stack Management → Connectors → Create connector → Amazon Bedrock**. +. . Find the **Connectors** page in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. Then click **Create Connector**, and select **Amazon Bedrock**. . Name your connector. . (Optional) Configure the Amazon Bedrock connector to use a different AWS region where Anthropic models are supported by editing the **URL** field, for example by changing `us-east-1` to `eu-central-1`. . (Optional) Add one of the following strings if you want to use a model other than the default: diff --git a/docs/AI-for-security/connect-to-byo.asciidoc b/docs/AI-for-security/connect-to-byo.asciidoc index f385c084aa..e1a3437a4a 100644 --- a/docs/AI-for-security/connect-to-byo.asciidoc +++ b/docs/AI-for-security/connect-to-byo.asciidoc @@ -178,7 +178,7 @@ image::images/lms-custom-logs-config.png[The configuration window for the custom Finally, configure the connector: 1. Log in to your Elastic deployment. -2. Navigate to **Stack Management → Connectors → Create Connector → OpenAI**. The OpenAI connector enables this use case because LM Studio uses the OpenAI SDK. +2. Find the **Connectors** page in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. Then click **Create Connector**, and select **OpenAI**. The OpenAI connector enables this use case because LM Studio uses the OpenAI SDK. 3. Name your connector to help keep track of the model version you are using. 4. Under **Select an OpenAI provider**, select **Other (OpenAI Compatible Service)**. 5. Under **URL**, enter the domain name specified in your Nginx configuration file, followed by `/v1/chat/completions`. diff --git a/docs/AI-for-security/connect-to-openai.asciidoc b/docs/AI-for-security/connect-to-openai.asciidoc index 830f657d23..3ee998637a 100644 --- a/docs/AI-for-security/connect-to-openai.asciidoc +++ b/docs/AI-for-security/connect-to-openai.asciidoc @@ -47,7 +47,7 @@ The following video demonstrates these steps. To integrate with {kib}: . Log in to {kib}. -. Navigate to **Stack Management → Connectors → Create Connector → OpenAI**. +. Find the **Connectors** page in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. Then click **Create Connector**, and select **OpenAI**. . Provide a name for your connector, such as `OpenAI (GPT-4 Turbo Preview)`, to help keep track of the model and version you are using. . Under **Select an OpenAI provider**, choose **OpenAI**. . The **URL** field can be left as default. diff --git a/docs/AI-for-security/connect-to-vertex.asciidoc b/docs/AI-for-security/connect-to-vertex.asciidoc index 5cd253f7fb..2825601ab8 100644 --- a/docs/AI-for-security/connect-to-vertex.asciidoc +++ b/docs/AI-for-security/connect-to-vertex.asciidoc @@ -93,7 +93,7 @@ The following video demonstrates these steps. Finally, configure the connector in your Elastic deployment: 1. Log in to your Elastic deployment. -2. Navigate to **Stack Management → Connectors → Create Connector → Google Gemini**. +2. Find the **Connectors** page in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. Then click **Create Connector**, select **Google Gemini**. 3. Name your connector to help keep track of the model version you are using. 4. Under **URL**, enter the URL for your region. 5. Enter your **GCP Region** and **GCP Project ID**. diff --git a/docs/cloud-native-security/cspm-benchmark-rules.asciidoc b/docs/cloud-native-security/cspm-benchmark-rules.asciidoc index 6bcebcf5b8..9240d918fa 100644 --- a/docs/cloud-native-security/cspm-benchmark-rules.asciidoc +++ b/docs/cloud-native-security/cspm-benchmark-rules.asciidoc @@ -23,7 +23,7 @@ NOTE: Benchmark rules are not editable. [discrete] == Review your benchmarks -To access your active benchmarks, go to **Rules -> Benchmarks**. From there, you can click a benchmark's name to view the benchmark rules associated with it. You can click a benchmark rule's name to see details including information about how to remediate it, and related links. +Find **Benchmarks** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. From there, you can click a benchmark's name to view the benchmark rules associated with it. You can click a benchmark rule's name to see details including information about how to remediate it, and related links. Benchmark rules are enabled by default, but you can disable some of them — at the benchmark level — to suit your environment. This means for example that if you have two integrations using the `CIS AWS` benchmark, disabling a rule for that benchmark affects both integrations. To enable or disable a rule, use the **Enabled** toggle on the right of the rules table. diff --git a/docs/cloud-native-security/cspm-get-started-aws.asciidoc b/docs/cloud-native-security/cspm-get-started-aws.asciidoc index bdcd180d80..bf077097cb 100644 --- a/docs/cloud-native-security/cspm-get-started-aws.asciidoc +++ b/docs/cloud-native-security/cspm-get-started-aws.asciidoc @@ -35,7 +35,7 @@ You can set up CSPM for AWS either by enrolling a single cloud account, or by en == Agentless deployment beta::[] -. From the Elastic Security *Get started* page, click *Add integrations*. +. Find **Integrations** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Search for `CSPM`, then click on the result. . Click *Add Cloud Security Posture Management (CSPM)*. . Select *AWS*, then either *AWS Organization* to onboard multiple accounts, or *Single Account* to onboard an individual account. @@ -53,7 +53,7 @@ beta::[] [discrete] [[cspm-add-and-name-integration]] === Add the CSPM integration -. From the Elastic Security *Get started* page, click *Add integrations*. +. Find **Integrations** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Search for `CSPM`, then click on the result. . Click *Add Cloud Security Posture Management (CSPM)*. . Select *AWS*, then either *AWS Organization* to onboard multiple accounts, or *Single Account* to onboard an individual account. diff --git a/docs/cloud-native-security/cspm-get-started-azure.asciidoc b/docs/cloud-native-security/cspm-get-started-azure.asciidoc index f47322abc0..865ebf02b0 100644 --- a/docs/cloud-native-security/cspm-get-started-azure.asciidoc +++ b/docs/cloud-native-security/cspm-get-started-azure.asciidoc @@ -35,7 +35,7 @@ You can set up CSPM for Azure by by enrolling an Azure organization (management == Agentless deployment beta::[] -. From the Elastic Security *Get started* page, click *Add integrations*. +. Find **Integrations** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Search for `CSPM`, then click on the result. . Click *Add Cloud Security Posture Management (CSPM)*. . Select *Azure*, then either *Azure Organization* to onboard your whole organization, or *Single Subscription* to onboard an individual subscription. @@ -51,7 +51,8 @@ beta::[] [discrete] [[cspm-add-and-name-integration-azure]] === Add your CSPM integration -. From the Elastic Security *Get started* page, click *Add integrations*. + +. Find **Integrations** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Search for `CSPM`, then click on the result. . Click *Add Cloud Security Posture Management (CSPM)*. . Under **Configure integration**, select **Azure**, then select either **Azure Organization** or **Single Subscription**, depending on which resources you want to monitor. diff --git a/docs/cloud-native-security/cspm-get-started-gcp.asciidoc b/docs/cloud-native-security/cspm-get-started-gcp.asciidoc index e7cf9d9f62..30d34c74c0 100644 --- a/docs/cloud-native-security/cspm-get-started-gcp.asciidoc +++ b/docs/cloud-native-security/cspm-get-started-gcp.asciidoc @@ -35,7 +35,7 @@ You can set up CSPM for GCP either by enrolling a single project, or by enrollin == Agentless deployment beta::[] -. From the Elastic Security *Get started* page, click *Add integrations*. +. Find **Integrations** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Search for `CSPM`, then click on the result. . Click *Add Cloud Security Posture Management (CSPM)*. . Select *GCP*, then either *GCP Organization* to onboard your whole organization, or *Single Account* to onboard an individual account. @@ -52,7 +52,7 @@ beta::[] [discrete] [[cspm-add-and-name-integration-gcp]] === Add your CSPM integration -. From the Elastic Security *Get started* page, click *Add integrations*. +. Find **Integrations** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Search for `CSPM`, then click on the result. . Click *Add Cloud Security Posture Management (CSPM)*. . Under *Configure integration*, select *GCP*, then either *GCP Organization* (recommended) or *Single Account*. diff --git a/docs/cloud-native-security/d4c-get-started.asciidoc b/docs/cloud-native-security/d4c-get-started.asciidoc index 585959d8f3..d73ecd5cb7 100644 --- a/docs/cloud-native-security/d4c-get-started.asciidoc +++ b/docs/cloud-native-security/d4c-get-started.asciidoc @@ -22,7 +22,7 @@ This page describes how to set up Cloud Workload Protection (CWP) for Kubernetes First, you'll need to deploy Elastic's Defend for Containers integration to the Kubernetes clusters you wish to monitor. -. Go to *Manage > Container Workload Security > Add D4C Integration*. +. Find **Container Workload Security** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. Click **Add D4C Integration**. . Name the integration. The default name, which you can change, is `cloud_defend-1`. . Optional — make any desired changes to the integration's policy by adjusting the *Selectors* and *Responses* sections. (For more information, refer to the <>). You can also change these later. . Under *Where to add this integration*, select an existing or new agent policy. @@ -54,7 +54,7 @@ In order to detect threats using this data, you'll need active < Rules > Detection rules (SIEM)*, then click *Add Elastic rules*. +. Find **Detection rules (SIEM)** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. Click **Add Elastic rules**. . Click the *Tags* filter next to the search bar, and search for the `Data Source: Elastic Defend for Containers` tag. . Select all the displayed rules, then click *Install _x_ selected rule(s)*. . Return to the *Rules* page. Click the *Tags* filter next to the search bar, and search for the `Data Source: Elastic Defend for Containers` tag. @@ -75,8 +75,7 @@ To enable drift detection, you can use the default D4C policy: To enable drift prevention, create a new policy: -. Add a new selector called `blockDrift`. -. Go to *Security > Manage > Container Workload Security > Your integration name*. +. Find **Container Workload Security** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field], then select your integration. . Under *Selectors*, click *Add selector > File Selector*. By default, it selects the operations `createExecutable` and `modifyExecutable`. . Name the selector, for example: `blockDrift`. . Scroll down to the *Responses* section and click *Add response > File Response*. diff --git a/docs/cloud-native-security/environment-variable-capture.asciidoc b/docs/cloud-native-security/environment-variable-capture.asciidoc index d93aa3beda..ec05a561b8 100644 --- a/docs/cloud-native-security/environment-variable-capture.asciidoc +++ b/docs/cloud-native-security/environment-variable-capture.asciidoc @@ -20,10 +20,11 @@ You can configure an {agent} policy to capture up to five environment variables To set up environment variable capture for an {agent} policy: -. Go to **Security -> Manage -> Policies**. + +. Find **Policies** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Select an {agent} policy. . Click *Show advanced settings*. -. Scroll down or search for `linux.advanced.capture_env_vars`. +. Scroll down or search for `linux.advanced.capture_env_vars` or `mac.advanced.capture_env_vars`. . Enter the names of env vars you want to capture, separated by commas. For example: `PATH,USER` . Click *Save*. @@ -37,7 +38,7 @@ Captured environment variables are associated with process events, and appear in To view environment variables in the *Events* table: -. Click the *Events* tab on the *Hosts*, *Network*, or *Users* pages (*Security -> Explore*), then click *Fields* in the Events table. +. Click the *Events* tab on the *Hosts*, *Network*, or *Users* pages, then click *Fields* in the Events table. . Search for the `process.env_vars` field, select it, and click *Close*. A new column appears containing captured environment variable data. diff --git a/docs/cloud-native-security/kspm-benchmark-rules.asciidoc b/docs/cloud-native-security/kspm-benchmark-rules.asciidoc index c8b44f6759..9f4aae3104 100644 --- a/docs/cloud-native-security/kspm-benchmark-rules.asciidoc +++ b/docs/cloud-native-security/kspm-benchmark-rules.asciidoc @@ -23,7 +23,7 @@ NOTE: Benchmark rules are not editable. [discrete] == Review your benchmarks -To access your active benchmarks, go to **Rules -> Benchmarks**. From there, you can click a benchmark's name to view the benchmark rules associated with it. You can click a benchmark rule's name to see details including information about how to remediate it, and related links. +Find **Benchmarks** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. From there, you can click a benchmark's name to view the benchmark rules associated with it. You can click a benchmark rule's name to see details including information about how to remediate it, and related links. Benchmark rules are enabled by default, but you can disable some of them — at the benchmark level — to suit your environment. This means for example that if you have two integrations using the `CIS AWS` benchmark, disabling a rule for that benchmark affects both integrations. To enable or disable a rule, use the **Enabled** toggle on the right of the rules table. diff --git a/docs/cloud-native-security/kspm-get-started.asciidoc b/docs/cloud-native-security/kspm-get-started.asciidoc index 7f05da0d61..2d85352a48 100644 --- a/docs/cloud-native-security/kspm-get-started.asciidoc +++ b/docs/cloud-native-security/kspm-get-started.asciidoc @@ -35,7 +35,7 @@ The instructions differ depending on whether you're installing on EKS or on unma [discrete] === Name your integration and select a Kubernetes Deployment type -1. Go to *Dashboards -> Cloud Security Posture*. +1. Find **Cloud Security Posture** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. 2. Click *Add a KSPM integration*. 3. Read the integration's description to understand how it works. Then, click {integrations-docs}/cloud_security_posture[*Add Kubernetes Security Posture Management*]. 4. Name your integration. Use a name that matches the purpose or team of the cluster(s) you want to monitor, for example, `IT-dev-k8s-clusters`. @@ -234,7 +234,7 @@ Follow these steps to deploy the KSPM integration to unmanaged clusters. Keep in === Configure the KSPM integration To install the integration on unmanaged clusters: -. Go to *Dashboards -> Cloud Security Posture*. +. Find **Cloud Security Posture** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Click *Add a KSPM integration*. . Read the integration's description to understand how it works. Then, click {integrations-docs}/cloud_security_posture[*Add Kubernetes Security Posture Management*]. . Name your integration. Use a name that matches the purpose or team of the cluster(s) you want to monitor, for example, `IT-dev-k8s-clusters`. diff --git a/docs/cloud-native-security/kspm.asciidoc b/docs/cloud-native-security/kspm.asciidoc index 971263e567..e0904cc918 100644 --- a/docs/cloud-native-security/kspm.asciidoc +++ b/docs/cloud-native-security/kspm.asciidoc @@ -62,7 +62,7 @@ To identify the Kubernetes resources generating the most failed findings: To identify risks in particular CIS sections: -. Go to the <> (*Dashboards -> Cloud Security Posture*). +. Go to the <>. . In the Failed findings by CIS section widget, click the name of a CIS section to view all failed findings for that section. Alternatively: diff --git a/docs/cloud-native-security/security-posture-management.asciidoc b/docs/cloud-native-security/security-posture-management.asciidoc index 8d6b12481e..a29392c8e1 100644 --- a/docs/cloud-native-security/security-posture-management.asciidoc +++ b/docs/cloud-native-security/security-posture-management.asciidoc @@ -23,14 +23,14 @@ Using the data generated by these features, you can: *Identify and secure misconfigured infrastructure:* -. Go to the Cloud Security Posture dashboard (*Dashboards > Cloud Security Posture*). +. Find **Cloud Security Posture** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Click *View all failed findings*, either for an individual resource or a group of resources. . Click a failed finding to open the Findings flyout. . Follow the steps under Remediation to fix the misconfiguration. *Identify the CIS Sections (security best practice categories) with which your resources are least compliant:* -. Go to the Cloud Security Posture dashboard (*Dashboards > Cloud Security Posture*). +. Find **Cloud Security Posture** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Do one of the following: .. Under Failed findings by CIS section, click the name of a CIS section to view all failed findings from that section. .. Go to the *Findings* page and filter by the `rule.section` field. For example, search for `rule.section : API Server` to view findings from the API Server category. diff --git a/docs/cloud-native-security/vuln-management-get-started.asciidoc b/docs/cloud-native-security/vuln-management-get-started.asciidoc index 7369634175..0f17f682ee 100644 --- a/docs/cloud-native-security/vuln-management-get-started.asciidoc +++ b/docs/cloud-native-security/vuln-management-get-started.asciidoc @@ -30,7 +30,7 @@ IMPORTANT: Do not add the integration to an existing {agent} policy. It should a [[vuln-management-setup-step-1]] === Step 1: Add the CNVM integration -. In the {security-app}, go to the **Get started** page, then click *Add security integrations*. +. Find **Integrations** in the navigation menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field]. . Search for **Cloud Native Vulnerability Management**, then click on the result. . Click *Add Cloud Native Vulnerability Management*. . Give your integration a name that matches its purpose or the AWS account region you want to scan for vulnerabilities (for example, `uswest2-aws-account`.) diff --git a/docs/serverless/AI-for-security/connect-to-azure-openai.asciidoc b/docs/serverless/AI-for-security/connect-to-azure-openai.asciidoc new file mode 100644 index 0000000000..3ff03fa1db --- /dev/null +++ b/docs/serverless/AI-for-security/connect-to-azure-openai.asciidoc @@ -0,0 +1,117 @@ +[[security-connect-to-azure-openai]] += Connect to Azure OpenAI + +// :description: Set up an Azure OpenAI LLM connector. +// :keywords: security, overview, get-started + +This page provides step-by-step instructions for setting up an Azure OpenAI connector for the first time. This connector type enables you to leverage large language models (LLMs) within {kib}. You'll first need to configure Azure, then configure the connector in {kib}. + +[discrete] +[[security-connect-to-azure-openai-configure-azure]] +== Configure Azure + +[discrete] +[[security-connect-to-azure-openai-configure-a-deployment]] +=== Configure a deployment + +First, set up an Azure OpenAI deployment: + +. Log in to the Azure console and search for Azure OpenAI. +. In **Azure AI services**, select **Create**. +. For the **Project Details**, select your subscription and resource group. If you don't have a resource group, select **Create new** to make one. +. For **Instance Details**, select the desired region and specify a name, such as `example-deployment-openai`. +. Select the **Standard** pricing tier, then click **Next**. +. Configure your network settings, click **Next**, optionally add tags, then click **Next**. +. Review your deployment settings, then click **Create**. When complete, select **Go to resource**. + +The following video demonstrates these steps. + +++++ + +++++ + +[discrete] +[[security-connect-to-azure-openai-configure-keys]] +=== Configure keys + +Next, create access keys for the deployment: + +. From within your Azure OpenAI deployment, select **Click here to manage keys**. +. Store your keys in a secure location. + +The following video demonstrates these steps. + +++++ + +++++ + +[discrete] +[[security-connect-to-azure-openai-configure-a-model]] +=== Configure a model + +Now, set up the Azure OpenAI model: + +. From within your Azure OpenAI deployment, select **Model deployments**, then click **Manage deployments**. +. On the **Deployments** page, select **Create new deployment**. +. Under **Select a model**, choose `gpt-4o` or `gpt-4 turbo`. +. Set the model version to "Auto-update to default". +. Under **Deployment type**, select **Standard**. +. Name your deployment. +. Slide the **Tokens per Minute Rate Limit** to the maximum. The following example supports 80,000 TPM, but other regions might support higher limits. +. Click **Create**. + +[IMPORTANT] +==== +The models available to you will depend on https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#model-summary-table-and-region-availability[region availability]. For best results, use `GPT-4o 2024-05-13` with the maximum Tokens-Per-Minute (TPM) capacity. For more information on how different models perform for different tasks, refer to the <>. +==== + +The following video demonstrates these steps. + +++++ + +++++ + +[discrete] +[[security-connect-to-azure-openai-configure-elastic-ai-assistant]] +== Configure Elastic AI Assistant + +Finally, configure the connector in {kib}: + +. Log in to {kib}. +. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **OpenAI**. +. Give your connector a name to help you keep track of different models, such as `Azure OpenAI (GPT-4 Turbo v. 0125)`. +. For **Select an OpenAI provider**, choose **Azure OpenAI**. +. Update the **URL** field. We recommend doing the following: ++ +** Navigate to your deployment in Azure AI Studio and select **Open in Playground**. The **Chat playground** screen displays. +** Select **View code**, then from the drop-down, change the **Sample code** to `Curl`. +** Highlight and copy the URL without the quotes, then paste it into the **URL** field in {kib}. +** (Optional) Alternatively, refer to the https://learn.microsoft.com/en-us/azure/ai-services/openai/reference[API documentation] to learn how to create the URL manually. +. Under **API key**, enter one of your API keys. +. Click **Save & test**, then click **Run**. + +The following video demonstrates these steps. + +++++ + +++++ diff --git a/docs/serverless/AI-for-security/connect-to-bedrock.asciidoc b/docs/serverless/AI-for-security/connect-to-bedrock.asciidoc new file mode 100644 index 0000000000..9581cf8e32 --- /dev/null +++ b/docs/serverless/AI-for-security/connect-to-bedrock.asciidoc @@ -0,0 +1,167 @@ +[[security-connect-to-bedrock]] += Connect to Amazon Bedrock + +// :description: Set up an Amazon Bedrock LLM connector. +// :keywords: security, overview, get-started + +This page provides step-by-step instructions for setting up an Amazon Bedrock connector for the first time. This connector type enables you to leverage large language models (LLMs) within {kib}. You'll first need to configure AWS, then configure the connector in {kib}. + +[NOTE] +==== +Only Amazon Bedrock's `Anthropic` models are supported: `Claude` and `Claude instant`. +==== + +[discrete] +[[security-connect-to-bedrock-configure-aws]] +== Configure AWS + +[discrete] +[[security-connect-to-bedrock-configure-an-iam-policy]] +=== Configure an IAM policy + +First, configure an IAM policy with the necessary permissions: + +. Log into the AWS console and search for Identity and Access Management (IAM). +. From the **IAM** menu, select **Policies** → **Create policy**. +. To provide the necessary permissions, paste the following JSON into the **Specify permissions** menu. + +[source,json] +---- +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "VisualEditor0", + "Effect": "Allow", + "Action": [ + "bedrock:InvokeModel", + "bedrock:InvokeModelWithResponseStream" + ], + "Resource": "*" + } + ] +} +---- + +[NOTE] +==== +These are the minimum required permissions. IAM policies with additional permissions are also supported. +==== + +. Click **Next**. Name your policy. + +The following video demonstrates these steps. + +++++ + +++++ + +[discrete] +[[security-connect-to-bedrock-configure-an-iam-user]] +=== Configure an IAM User + +Next, assign the policy you just created to a new user: + +. Return to the **IAM** menu. Select **Users** from the navigation menu, then click **Create User**. +. Name the user, then click **Next**. +. Select **Attach policies directly**. +. In the **Permissions policies** field, search for the policy you created earlier, select it, and click **Next**. +. Review the configuration then click **Create user**. + +The following video demonstrates these steps. + +++++ + +++++ + +[discrete] +[[security-connect-to-bedrock-create-an-access-key]] +=== Create an access key + +Create the access keys that will authenticate your Elastic connector: + +. Return to the **IAM** menu. Select **Users** from the navigation menu. +. Search for the user you just created, and click its name. +. Go to the **Security credentials** tab. +. Under **Access keys**, click **Create access key**. +. Select **Third-party service**, check the box under **Confirmation**, click **Next**, then click **Create access key**. +. Click **Download .csv file** to download the key. Store it securely. + +The following video demonstrates these steps. + +++++ + +++++ + +[discrete] +[[security-connect-to-bedrock-enable-model-access]] +=== Enable model access + +Make sure the supported Amazon Bedrock LLMs are enabled: + +. Search the AWS console for Amazon Bedrock. +. From the Amazon Bedrock page, click **Get started**. +. Select **Model access** from the left navigation menu, then click **Manage model access**. +. Check the boxes for **Claude** and/or **Claude Instant**, depending which model or models you plan to use. +. Click **Save changes**. + +The following video demonstrates these steps. + +++++ + +++++ + +[discrete] +[[security-connect-to-bedrock-configure-the-amazon-bedrock-connector]] +== Configure the Amazon Bedrock connector + +Finally, configure the connector in {kib}: + +. Log in to {kib}. +. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **Amazon Bedrock**. +. Name your connector. +. (Optional) Configure the Amazon Bedrock connector to use a different AWS region where Anthropic models are supported by editing the **URL** field, for example by changing `us-east-1` to `eu-central-1`. +. (Optional) Add one of the following strings if you want to use a model other than the default: ++ +** For Haiku: `anthropic.claude-3-haiku-20240307-v1:0` +** For Sonnet: `anthropic.claude-3-sonnet-20240229-v1:0` +** For Opus: `anthropic.claude-3-opus-20240229-v1:0` +. Enter the **Access Key** and **Secret** that you generated earlier, then click **Save**. + +Your LLM connector is now configured. For more information on using Elastic AI Assistant, refer to https://docs.elastic.co/security/ai-assistant[AI Assistant]. + +[IMPORTANT] +==== +If you're using https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html[provisioned throughput], your ARN becomes the model ID, and the connector settings **URL** value must be https://www.urlencoder.org/[encoded] to work. For example, if the non-encoded ARN is `arn:aws:bedrock:us-east-2:123456789102:provisioned-model/3Ztr7hbzmkrqy1`, the encoded ARN would be `arn%3Aaws%3Abedrock%3Aus-east-2%3A123456789102%3Aprovisioned-model%2F3Ztr7hbzmkrqy1`. +==== + +The following video demonstrates these steps. + +++++ + +++++ diff --git a/docs/serverless/AI-for-security/connect-to-byo-llm.asciidoc b/docs/serverless/AI-for-security/connect-to-byo-llm.asciidoc new file mode 100644 index 0000000000..6f5d6fbb3d --- /dev/null +++ b/docs/serverless/AI-for-security/connect-to-byo-llm.asciidoc @@ -0,0 +1,223 @@ +[[security-connect-to-byo-llm]] += Connect to your own local LLM + +// :description: Set up a connector to LM Studio so you can use a local model with AI Assistant. +// :keywords: security, overview, get-started + +This page provides instructions for setting up a connector to a large language model (LLM) of your choice using LM Studio. This allows you to use your chosen model within {elastic-sec}. You'll first need to set up a reverse proxy to communicate with {elastic-sec}, then set up LM Studio on a server, and finally configure the connector in your {elastic-sec} project. https://www.elastic.co/blog/ai-assistant-locally-hosted-models[Learn more about the benefits of using a local LLM]. + +This example uses a single server hosted in GCP to run the following components: + +* LM Studio with the https://mistral.ai/technology/#models[Mixtral-8x7b] model +* A reverse proxy using Nginx to authenticate to Elastic Cloud + +[role="screenshot"] +image::images/lms-studio-arch-diagram.png[Architecture diagram for this guide] + +[NOTE] +==== +For testing, you can use alternatives to Nginx such as https://learn.microsoft.com/en-us/azure/developer/dev-tunnels/overview[Azure Dev Tunnels] or https://ngrok.com/[Ngrok], but using Nginx makes it easy to collect additional telemetry and monitor its status by using Elastic's native Nginx integration. While this example uses cloud infrastructure, it could also be replicated locally without an internet connection. +==== + +[discrete] +[[security-connect-to-byo-llm-configure-your-reverse-proxy]] +== Configure your reverse proxy + +[NOTE] +==== +If your Elastic instance is on the same host as LM Studio, you can skip this step. +==== + +You need to set up a reverse proxy to enable communication between LM Studio and Elastic. For more complete instructions, refer to a guide such as https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-as-a-reverse-proxy-on-ubuntu-22-04[this one]. + +The following is an example Nginx configuration file: + +[source,txt] +---- +server { + listen 80; + listen [::]:80; + server_name ; + server_tokens off; + add_header x-xss-protection "1; mode=block" always; + add_header x-frame-options "SAMEORIGIN" always; + add_header X-Content-Type-Options "nosniff" always; + return 301 https://$server_name$request_uri; +} + +server { + + listen 443 ssl http2; + listen [::]:443 ssl http2; + server_name ; + server_tokens off; + ssl_certificate /etc/letsencrypt/live//fullchain.pem; + ssl_certificate_key /etc/letsencrypt/live//privkey.pem; + ssl_session_timeout 1d; + ssl_session_cache shared:SSL:50m; + ssl_session_tickets on; + ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256'; + ssl_protocols TLSv1.3 TLSv1.2; + ssl_prefer_server_ciphers on; + add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always; + add_header x-xss-protection "1; mode=block" always; + add_header x-frame-options "SAMEORIGIN" always; + add_header X-Content-Type-Options "nosniff" always; + add_header Referrer-Policy "strict-origin-when-cross-origin" always; + ssl_stapling on; + ssl_stapling_verify on; + ssl_trusted_certificate /etc/letsencrypt/live//fullchain.pem; + resolver 1.1.1.1; + location / { + + if ($http_authorization != "Bearer ") { + return 401; +} + + proxy_pass http://localhost:1234/; + } + +} +---- + +[IMPORTANT] +==== +* Replace `` with your actual token, and keep it safe since you'll need it to set up the {elastic-sec} connector. +* Replace `` with your actual domain name. +* Update the `proxy_pass` value at the bottom of the configuration if you decide to change the port number in LM Studio to something other than 1234. +==== + +[discrete] +[[security-connect-to-byo-llm-optional-set-up-performance-monitoring-for-your-reverse-proxy]] +=== (Optional) Set up performance monitoring for your reverse proxy + +You can use Elastic's https://www.elastic.co/docs/current/integrations/nginx[Nginx integration] to monitor performance and populate monitoring dashboards in the {security-app}. + +[discrete] +[[security-connect-to-byo-llm-configure-lm-studio-and-download-a-model]] +== Configure LM Studio and download a model + +First, install https://lmstudio.ai/[LM Studio]. LM Studio supports the OpenAI SDK, which makes it compatible with Elastic's OpenAI connector, allowing you to connect to any model available in the LM Studio marketplace. + +One current limitation of LM Studio is that when it is installed on a server, you must launch the application using its GUI before doing so using the CLI. For example, by using Chrome RDP with an https://cloud.google.com/architecture/chrome-desktop-remote-on-compute-engine[X Window System]. After you've opened the application the first time using the GUI, you can start it by using `sudo lms server start` in the CLI. + +Once you've launched LM Studio: + +. Go to LM Studio's Search window. +. Search for an LLM (for example, `Mixtral-8x7B-instruct`). Your chosen model must include `instruct` in its name in order to work with Elastic. +. Filter your search for "Compatibility Guess" to optimize results for your hardware. Results will be color coded: ++ +** Green means "Full GPU offload possible", which yields the best results. +** Blue means "Partial GPU offload possible", which may work. +** Red for "Likely too large for this machine", which typically will not work. +. Download one or more models. + +[IMPORTANT] +==== +For security reasons, before downloading a model, verify that it is from a trusted source. It can be helpful to review community feedback on the model (for example using a site like Hugging Face). +==== + +[role="screenshot"] +image::images/lms-model-select.png[The LM Studio model selection interface] + +In this example we used https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF[`TheBloke/Mixtral-8x7B-Instruct-v0.1.Q3_K_M.gguf`]. It has 46.7B total parameters, a 32,000 token context window, and uses GGUF https://huggingface.co/docs/transformers/main/en/quantization/overview[quanitization]. For more information about model names and format information, refer to the following table. + +|=== +| Model Name| Parameter Size| Tokens/Context Window| Quantization Format + +| Name of model, sometimes with a version number. +| LLMs are often compared by their number of parameters — higher numbers mean more powerful models. +| Tokens are small chunks of input information. Tokens do not necessarily correspond to characters. You can use https://platform.openai.com/tokenizer[Tokenizer] to see how many tokens a given prompt might contain. +| Quantization reduces overall parameters and helps the model to run faster, but reduces accuracy. + +| Examples: Llama, Mistral, Phi-3, Falcon. +| The number of parameters is a measure of the size and the complexity of the model. The more parameters a model has, the more data it can process, learn from, generate, and predict. +| The context window defines how much information the model can process at once. If the number of input tokens exceeds this limit, input gets truncated. +| Specific formats for quantization vary, most models now support GPU rather than CPU offloading. +|=== + +[discrete] +[[security-connect-to-byo-llm-load-a-model-in-lm-studio]] +== Load a model in LM Studio + +After downloading a model, load it in LM Studio using the GUI or LM Studio's https://lmstudio.ai/blog/lms[CLI tool]. + +[discrete] +[[security-connect-to-byo-llm-option-1-load-a-model-using-the-cli-recommended]] +=== Option 1: load a model using the CLI (Recommended) + +It is a best practice to download models from the marketplace using the GUI, and then load or unload them using the CLI. The GUI allows you to search for models, whereas the CLI only allows you to import specific paths, but the CLI provides a good interface for loading and unloading. + +Use the following commands in your CLI: + +. Verify LM Studio is installed: `lms` +. Check LM Studio's status: `lms status` +. List all downloaded models: `lms ls` +. Load a model: `lms load` + +[role="screenshot"] +image::images/lms-cli-welcome.png[The CLI interface during execution of initial LM Studio commands] + +After the model loads, you should see a `Model loaded successfully` message in the CLI. + +[role="screenshot"] +image::images/lms-studio-model-loaded-msg.png[The CLI message that appears after a model loads] + +To verify which model is loaded, use the `lms ps` command. + +[role="screenshot"] +image::images/lms-ps-command.png[The CLI message that appears after running lms ps] + +If your model uses NVIDIA drivers, you can check the GPU performance with the `sudo nvidia-smi` command. + +[discrete] +[[security-connect-to-byo-llm-option-2-load-a-model-using-the-gui]] +=== Option 2: load a model using the GUI + +Refer to the following video to see how to load a model using LM Studio's GUI. You can change the **port** setting, which is referenced in the Nginx configuration file. Note that the **GPU offload** was set to **Max**. + +++++ + + +++++ + +[discrete] +[[security-connect-to-byo-llm-optional-collect-logs-using-elastics-custom-logs-integration]] +== (Optional) Collect logs using Elastic's Custom Logs integration + +You can monitor the performance of the host running LM Studio using Elastic's https://www.elastic.co/docs/current/integrations/log[Custom Logs integration]. This can also help with troubleshooting. Note that the default path for LM Studio logs is `/tmp/lmstudio-server-log.txt`, as in the following screenshot: + +[role="screenshot"] +image::images/lms-custom-logs-config.png[The configuration window for the custom logs integration] + +[discrete] +[[security-connect-to-byo-llm-configure-the-connector-in-elastic-sec]] +== Configure the connector in {elastic-sec} + +Finally, configure the connector in your Security project: + +. Log in to your Security project. +. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **OpenAI**. The OpenAI connector enables this use case because LM Studio uses the OpenAI SDK. +. Name your connector to help keep track of the model version you are using. +. Under **Select an OpenAI provider**, select **Other (OpenAI Compatible Service)**. +. Under **URL**, enter the domain name specified in your Nginx configuration file, followed by `/v1/chat/completions`. +. Under **Default model**, enter `local-model`. +. Under **API key**, enter the secret token specified in your Nginx configuration file. +. Click **Save**. + +[role="screenshot"] +image::images/lms-edit-connector.png[The Edit connector page in the {security-app}, with appropriate values populated] + +Setup is now complete. You can use the model you've loaded in LM Studio to power Elastic's generative AI features. You can test a variety of models as you interact with AI Assistant to see what works best without having to update your connector. + +[NOTE] +==== +While local models work well for <>, we recommend you use one of <> for interacting with <>. As local models become more performant over time, this is likely to change. +==== diff --git a/docs/serverless/AI-for-security/connect-to-openai.asciidoc b/docs/serverless/AI-for-security/connect-to-openai.asciidoc new file mode 100644 index 0000000000..39804f59fd --- /dev/null +++ b/docs/serverless/AI-for-security/connect-to-openai.asciidoc @@ -0,0 +1,70 @@ +[[security-connect-to-openai]] += Connect to OpenAI + +// :description: Set up an OpenAI LLM connector. +// :keywords: security, overview, get-started + +This page provides step-by-step instructions for setting up an OpenAI connector for the first time. This connector type enables you to leverage OpenAI's large language models (LLMs) within {kib}. You'll first need to create an OpenAI API key, then configure the connector in {kib}. + +[discrete] +[[security-connect-to-openai-configure-openai]] +== Configure OpenAI + +[discrete] +[[security-connect-to-openai-select-a-model]] +=== Select a model + +Before creating an API key, you must choose a model. Refer to the https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4[OpenAI docs] to select a model. Take note of the specific model name (for example `gpt-4-turbo`); you'll need it when configuring {kib}. + +[NOTE] +==== +`GPT-4o` offers increased performance over previous versions. For more information on how different models perform for different tasks, refer to the <>. +==== + +[discrete] +[[security-connect-to-openai-create-an-api-key]] +=== Create an API key + +To generate an API key: + +. Log in to the OpenAI platform and navigate to **API keys**. +. Select **Create new secret key**. +. Name your key, select an OpenAI project, and set the desired permissions. +. Click **Create secret key** and then copy and securely store the key. It will not be accessible after you leave this screen. + +The following video demonstrates these steps. + +++++ + +++++ + +[discrete] +[[security-connect-to-openai-configure-the-openai-connector]] +== Configure the OpenAI connector + +Finally, configure the connector in {kib}: + +. Log in to {kib}. +. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **OpenAI**. +. Provide a name for your connector, such as `OpenAI (GPT-4 Turbo Preview)`, to help keep track of the model and version you are using. +. Under **Select an OpenAI provider**, choose **OpenAI**. +. The **URL** field can be left as default. +. Under **Default model**, specify which https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4[model] you want to use. +. Paste the API key that you created into the corresponding field. +. Click **Save**. + +The following video demonstrates these steps. + +++++ + +++++ diff --git a/docs/serverless/AI-for-security/connect-to-vertex.asciidoc b/docs/serverless/AI-for-security/connect-to-vertex.asciidoc new file mode 100644 index 0000000000..f33ae56329 --- /dev/null +++ b/docs/serverless/AI-for-security/connect-to-vertex.asciidoc @@ -0,0 +1,115 @@ +[[security-connect-to-google-vertex]] += Connect to Google Vertex AI + +// :description: Set up a Google Vertex LLM connector. +// :keywords: security, overview, get-started + +This page provides step-by-step instructions for setting up a Google Vertex AI connector for the first time. This connector type enables you to leverage Vertex AI's large language models (LLMs) within {elastic-sec}. You'll first need to enable Vertex AI, then generate an API key, and finally configure the connector in your {elastic-sec} project. + +[IMPORTANT] +==== +Before continuing, you should have an active project in one of Google Vertex AI's https://cloud.google.com/vertex-ai/docs/general/locations#feature-availability[supported regions]. +==== + +[discrete] +[[security-connect-to-google-vertex-enable-the-vertex-ai-api]] +== Enable the Vertex AI API + +. Log in to the GCP console and navigate to **Vertex AI → Vertex AI Studio → Overview**. +. If you're new to Vertex AI, the **Get started with Vertex AI Studio** popup appears. Click **Vertex AI API**, then click **ENABLE**. + +The following video demonstrates these steps. + +++++ + + +++++ + +[NOTE] +==== +For more information about enabling the Vertex AI API, refer to https://cloud.google.com/vertex-ai/docs/start/cloud-environment[Google's documentation]. +==== + +[discrete] +[[security-connect-to-google-vertex-create-a-vertex-ai-service-account]] +== Create a Vertex AI service account + +. In the GCP console, navigate to **APIs & Services → Library**. +. Search for **Vertex AI API**, select it, and click **MANAGE**. +. In the left menu, navigate to **Credentials** then click **+ CREATE CREDENTIALS** and select **Service account**. +. Name the new service account, then click **CREATE AND CONTINUE**. +. Under **Select a role**, select **Vertex AI User**, then click **CONTINUE**. +. Click **Done**. + +The following video demonstrates these steps. + +++++ + + +++++ + +[discrete] +[[security-connect-to-google-vertex-generate-an-api-key]] +== Generate an API key + +. Return to Vertex AI's **Credentials** menu and click **Manage service accounts**. +. Search for the service account you just created, select it, then click the link that appears under **Email**. +. Go to the **KEYS** tab, click **ADD KEY**, then select **Create new key**. +. Select **JSON**, then click **CREATE** to download the key. Keep it somewhere secure. + +The following video demonstrates these steps. + +++++ + + +++++ + +[discrete] +[[security-connect-to-google-vertex-configure-the-google-gemini-connector]] +== Configure the Google Gemini connector + +Finally, configure the connector in {kib}: + +. Log in to {kib}. +. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **Google Gemini**. +. Name your connector to help keep track of the model version you are using. +. Under **URL**, enter the URL for your region. +. Enter your **GCP Region** and **GCP Project ID**. +. Under **Default model**, specify either `gemini-1.5.pro` or `gemini-1.5-flash`. https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models[Learn more about the models]. +. Under **Authentication**, enter your API key. +. Click **Save**. + +The following video demonstrates these steps. + +++++ + + +++++ diff --git a/docs/serverless/cloud-native-security/benchmark-rules.asciidoc b/docs/serverless/cloud-native-security/benchmark-rules.asciidoc new file mode 100644 index 0000000000..d9fcc2eb7b --- /dev/null +++ b/docs/serverless/cloud-native-security/benchmark-rules.asciidoc @@ -0,0 +1,61 @@ +[[security-benchmark-rules]] += Benchmarks + +// :description: Review the cloud security benchmark rules used by the CSPM and KSPM integrations. +// :keywords: serverless, security, overview, cloud security + +:append: + +// tag::content[] + +preview:[] + +The Benchmarks page lets you view the cloud security posture (CSP) benchmarks for the <> (CSPM) and <> (KSPM) integrations. + +[role="screenshot"] +image::images/benchmark-rules/-cloud-native-security-benchmark-rules.png[Benchmark rules page] + +[discrete] +[id="security-benchmark-rules-what-are-benchmarks{append}"] +== What are benchmarks? + +Each benchmark contains benchmark rules which are used by the CSPM and KSPM integrations to identify configuration risks in your cloud infrastructure. There are different benchmarks for different cloud services, such as AWS, GCP, or Azure. They are based on the Center for Internet Security's (CIS) https://www.cisecurity.org/cis-benchmarks/[secure configuration benchmarks]. + +Each benchmark rule checks to see if a specific type of resource is configured according to a CIS Benchmark. The names of rules describe what they check, for example: + +* `Ensure Kubernetes Secrets are encrypted using Customer Master Keys (CMKs) managed in AWS KMS` +* `Ensure the default namespace is not in use` +* `Ensure IAM policies that allow full "*:*" administrative privileges are not attached` +* `Ensure the default namespace is not in use` + +When benchmark rules are evaluated, the resulting <> data appears on the <>. + +[NOTE] +==== +Benchmark rules are not editable. +==== + +[discrete] +[id="security-benchmark-rules-review-your-benchmarks{append}"] +== Review your benchmarks + +Find **Benchmarks** in the navigation menu or use the global search field. From there, you can click a benchmark's name to view the benchmark rules associated with it. You can click a benchmark rule's name to see details including information about how to remediate it, and related links. + +Benchmark rules are enabled by default, but you can disable some of them — at the benchmark level — to suit your environment. This means for example that if you have two CSPM integrations using the `CIS AWS` benchmark, disabling a rule for that benchmark affects both integrations. To enable or disable a rule, use the **Enabled** toggle on the right of the rules table. + +[NOTE] +==== +Disabling a benchmark rule automatically disables any associated detection rules and alerts. Re-enabling a benchmark rule **does not** automatically re-enable them. +==== + +[discrete] +[id="security-benchmark-rules-how-benchmark-rules-work{append}"] +== How benchmark rules work + +. When a security posture management integration is deployed, and every four hours after that, {agent} fetches relevant cloud resources. +. After resources are fetched, they are evaluated against all applicable enabled benchmark rules. +. Finding values of `pass` or `fail` indicate whether the standards defined by benchmark rules were met. + +// end::content[] + +:append!: diff --git a/docs/serverless/cloud-native-security/cspm-get-started-azure.asciidoc b/docs/serverless/cloud-native-security/cspm-get-started-azure.asciidoc new file mode 100644 index 0000000000..01c42f26e1 --- /dev/null +++ b/docs/serverless/cloud-native-security/cspm-get-started-azure.asciidoc @@ -0,0 +1,198 @@ +[[security-cspm-get-started-azure]] += Get started with CSPM for Azure + +// :description: Start monitoring the security posture of your Azure cloud assets. +// :keywords: serverless, security, overview, cloud security + +preview:[] + +[discrete] +[[cspm-overview-azure]] +== Overview + +This page explains how to get started monitoring the security posture of your cloud assets using the Cloud Security Posture Management (CSPM) feature. + +.Requirements +[NOTE] +==== +* CSPM only works in the `Default` {kib} space. Installing the CSPM integration on a different {kib} space will not work. +* CSPM is supported only on AWS, GCP, and Azure commercial cloud platforms, and AWS GovCloud. Other government cloud platforms are not supported (https://github.com/elastic/kibana/issues/new/choose[request support]). +* To view posture data, you need `read` privileges for the following {es} indices: ++ +** `logs-cloud_security_posture.findings_latest-*` +** `logs-cloud_security_posture.scores-*` +** `logs-cloud_security_posture.findings` +* The user who gives the CSPM integration permissions in Azure must be an Azure subscription `admin`. +==== + +[discrete] +[[cspm-setup-azure]] +== Set up CSPM for Azure + +You can set up CSPM for Azure by by enrolling an Azure organization (management group) containing multiple subscriptions, or by enrolling a single subscription. Either way, first add the CSPM integration, then enable cloud account access. Two deployment technologies are available: agentless, and agent-based. <> allows you to collect cloud posture data without having to manage the deployment of an agent in your cloud. <> requires you to deploy and manage an agent in the cloud account you want to monitor. + +[discrete] +[[cspm-azure-agentless]] +== Agentless deployment + +beta:[] + +. Find **Integrations** in the navigation menu or use the global search field. +. Search for `CSPM`, then click on the result. +. Click **Add Cloud Security Posture Management (CSPM)**. +. Select **Azure**, then either **Azure Organization** to onboard your whole organization, or **Single Subscription** to onboard an individual subscription. +. Give your integration a name that matches the purpose or team of the Azure subscription/organization you want to monitor, for example, `dev-azure-account`. +. Click **Advanced options**, then select **Agentless (BETA)**. +. Next, you'll need to authenticate to Azure by providing a **Client ID**, **Tenant ID**, and **Client Secret**. To learn how to generate them, refer to <>. +. Once you've provided the necessary credentials, click **Save and continue** to finish deployment. Your data should start to appear within a few minutes. + +[discrete] +[[cspm-azure-agent-based]] +== Agent-based deployment + +[discrete] +[[cspm-add-and-name-integration-azure]] +=== Add your CSPM integration + +. Find **Integrations** in the navigation menu or use the global search field. +. Search for `CSPM`, then click on the result. +. Click **Add Cloud Security Posture Management (CSPM)**. +. Under **Configure integration**, select **Azure**, then select either **Azure Organization** or **Single Subscription**, depending on which resources you want to monitor. +. Give your integration a name that matches the purpose or team of the Azure resources you want to monitor, for example, `azure-CSPM-dev-1`. + +[discrete] +[[cspm-set-up-cloud-access-section-azure]] +=== Set up cloud account access + +To set up CSPM for an Azure organization or subscription, you will need admin privileges for that organization or subscription. + +For most users, the simplest option is to use an Azure Resource Manager (ARM) template to automatically provision the necessary resources and permissions in Azure. If you prefer a more hands-on approach or require a specific configuration not supported by the ARM template, you can use one of the manual setup options described below. + +[discrete] +[[cspm-set-up-ARM]] +=== ARM template setup (recommended) + +. Under **Setup Access**, select **ARM Template**. +. Under **Where to add this integration**: ++ +.. Select **New Hosts**. +.. Name the {agent} policy. Use a name that matches the resources you want to monitor, for example, `azure-dev-policy`. Click **Save and continue**. The **ARM Template deployment** window appears. +.. In a new tab, log in to the Azure portal, then return to {kib} and click **Launch ARM Template**. This will open the ARM template in Azure. +.. If you are deploying to an Azure organization, select the management group you want to monitor from the drop-down menu. Next, enter the subscription ID of the subscription where you want to deploy the VM that will scan your resources. +.. Copy the `Fleet URL` and `Enrollment Token` that appear in {kib} to the corresponding fields in the ARM Template, then click **Review + create**. +.. (Optional) Change the `Resource Group Name` parameter. Otherwise, the name of the resource group defaults to a timestamp prefixed with `cloudbeat-`. +. Return to {kib} and wait for the confirmation of data received from your new integration. Then you can click **View Assets** to see your data. + +[discrete] +[[cspm-set-up-manual-azure]] +=== Manual setup + +For manual setup, multiple authentication methods are available: + +. Managed identity (recommended) +. Service principal with client secret +. Service principal with client certificate + +[discrete] +[[cspm-azure-managed-identity-setup]] +=== Option 1: Managed identity (recommended) + +This method involves creating an Azure VM (or using an existing one), giving it read access to the resources you want to monitor with CSPM, and installing {agent} on it. + +. Go to the Azure portal to create a new Azure VM. +. Follow the setup process, and make sure you enable **System assigned managed identity** under the **Management** tab. +. Go to your Azure subscription list and select the subscription or management group you want to monitor with CSPM. +. Go to **Access control (IAM)**, and select **Add Role Assignment**. +. Select the `Reader` role, assign access to **Managed Identity**, then select your VM. + +After assigning the role: + +. Return to the **Add CSPM** page in {kib}. +. Under **Configure integration**, select **Azure**. Under **Setup access**, select **Manual**. +. Under **Where to add this integration**, select **New hosts**. +. Click **Save and continue**, then follow the instructions to install {agent} on your Azure VM. + +Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data. + +[discrete] +[[cspm-azure-client-secret]] +=== Option 2: Service principal with client secret + +Before using this method, you must have set up a https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#get-tenant-and-app-id-values-for-signing-in[Microsoft Entra application and service principal that can access resources]. + +. On the **Add Cloud Security Posture Management (CSPM) integration** page, scroll to the **Setup access** section, then select **Manual**. +. Under **Preferred manual method**, select **Service principal with Client Secret**. +. Go to the **Registered apps** section of https://ms.portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/RegisteredApps[Microsoft Entra ID]. +. Click on **New Registration**, name your app and click **Register**. +. Copy your new app's `Directory (tenant) ID` and `Application (client) ID`. Paste them into the corresponding fields in {kib}. +. Return to the Azure portal. Select **Certificates & secrets**, then go to the **Client secrets** tab. Click **New client secret**. +. Copy the new secret. Paste it into the corresponding field in {kib}. +. Return to Azure. Go to your Azure subscription list and select the subscription or management group you want to monitor with CSPM. +. Go to **Access control (IAM)** and select **Add Role Assignment**. +. Select the `Reader` function role, assign access to **User, group, or service principal**, and select your new app. +. Return to the **Add CSPM** page in {kib}. +. Under **Where to add this integration**, select **New hosts**. +. Click **Save and continue**, then follow the instructions to install {agent} on your selected host. + +Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data. + +[discrete] +[[cspm-azure-client-certificate]] +=== Option 3: Service principal with client certificate + +Before using this method, you must have set up a https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#get-tenant-and-app-id-values-for-signing-in[Microsoft Entra application and service principal that can access resources]. + +. On the **Add Cloud Security Posture Management (CSPM) integration** page, under **Setup access**, select **Manual**. +. Under **Preferred manual method**, select **Service principal with client certificate**. +. Go to the **Registered apps** section of https://ms.portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/RegisteredApps[Microsoft Entra ID]. +. Click on **New Registration**, name your app and click **Register**. +. Copy your new app's `Directory (tenant) ID` and `Application (client) ID`. Paste them into the corresponding fields in {kib}. +. Return to Azure. Go to your Azure subscription list and select the subscription or management group you want to monitor with CSPM. +. Go to **Access control (IAM)** and select **Add Role Assignment**. +. Select the `Reader` function role, assign access to **User, group, or service principal**, and select your new app. + +Next, create a certificate. If you intend to use a password-protected certificate, you must use a pkcs12 certificate. Otherwise, you must use a pem certificate. + +Create a pkcs12 certificate, for example: + +[source,shell] +---- +# Create PEM file +openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes + +# Create pkcs12 bundle using legacy flag (CLI will ask for export password) +openssl pkcs12 -legacy -export -out bundle.p12 -inkey key.pem -in cert.pem +---- + +Create a PEM certificate, for example: + +[source,shell] +---- +# Generate certificate signing request (csr) and key +openssl req -new -newkey rsa:4096 -nodes -keyout cert.key -out cert.csr + +# Generate PEM and self-sign with key +openssl x509 -req -sha256 -days 365 -in cert.csr -signkey cert.key -out signed.pem + +# Create bundle +cat cert.key > bundle.pem +cat signed.pem >> bundle.pem +---- + +. Return to Azure. +. Navigate to the **Certificates & secrets** menu. Select the **Certificates** tab. +. Click **Upload certificate**. ++ +.. If you're using a PEM certificate that was created using the example commands above, upload `signed.pem`. +.. If you're using a pkcs12 certificate that was created using the example commands above, upload `cert.pem`. +. Upload the certificate bundle to the VM where you will deploy {agent}. ++ +.. If you're using a PEM certificate that was created using the example commands above, upload `bundle.pem`. +.. If you're using a pkcs12 certificate that was created using the example commands above, upload `bundle.p12`. +. Return to the **Add CSPM** page in {kib}. +. For **Client Certificate Path**, enter the full path to the certificate that you uploaded to the host where you will install {agent}. +. If you used a pkcs12 certificate, enter its password under **Client Certificate Password**. +. Under **Where to add this integration**, select **New hosts**. +. Click **Save and continue**, then follow the instructions to install {agent} on your selected host. + +Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data. diff --git a/docs/serverless/cloud-native-security/cspm-get-started-gcp.asciidoc b/docs/serverless/cloud-native-security/cspm-get-started-gcp.asciidoc new file mode 100644 index 0000000000..4eea50b7bf --- /dev/null +++ b/docs/serverless/cloud-native-security/cspm-get-started-gcp.asciidoc @@ -0,0 +1,205 @@ +[[security-cspm-get-started-gcp]] += Get started with CSPM for GCP + +// :description: Start monitoring the security posture of your GCP cloud assets. +// :keywords: serverless, security, overview, cloud security + +preview:[] + +[discrete] +[[cspm-overview-gcp]] +== Overview + +This page explains how to get started monitoring the security posture of your cloud assets using the Cloud Security Posture Management (CSPM) feature. + +.Requirements +[NOTE] +==== +* CSPM only works in the `Default` {kib} space. Installing the CSPM integration on a different {kib} space will not work. +* CSPM is supported only on AWS, GCP, and Azure commercial cloud platforms, and AWS GovCloud. Other government cloud platforms are not supported (https://github.com/elastic/kibana/issues/new/choose[request support]). +* To view posture data, you need the appropriate user role to read the following {es} indices: ++ +** `logs-cloud_security_posture.findings_latest-*` +** `logs-cloud_security_posture.scores-*` +** `Logs-cloud_security_posture.findings` +* The user who gives the CSPM integration GCP permissions must be a GCP project `admin`. +==== + +[discrete] +[[cspm-setup-gcp]] +== Set up CSPM for GCP + +You can set up CSPM for GCP either by enrolling a single project, or by enrolling an organization containing multiple projects. Either way, you need to first add the CSPM integration, then enable cloud account access. Two deployment technologies are available: agentless, and agent-based. <> allows you to collect cloud posture data without having to manage the deployment of an agent in your cloud. <> requires you to deploy and manage an agent in the cloud account you want to monitor. + +[discrete] +[[cspm-gcp-agentless]] +== Agentless deployment + +beta:[] + +. Find **Integrations** in the navigation menu or use the global search field. +. Search for `CSPM`, then click on the result. +. Click **Add Cloud Security Posture Management (CSPM)**. +. Select **GCP**, then either **GCP Organization** to onboard your whole organization, or **Single Account** to onboard an individual account. +. Give your integration a name that matches the purpose or team of the GCP subscription/organization you want to monitor, for example, `dev-gcp-account`. +. Click **Advanced options**, then select **Agentless (BETA)**. +. Next, you'll need to authenticate to GCP. Expand the **Steps to Generate GCP Account Credentials** section, then follow the instructions that appear to automatically create the necessary credentials using Google Cloud Shell. +. Once you've provided the necessary credentials, click **Save and continue** to finish deployment. Your data should start to appear within a few minutes. + +[discrete] +[[cspm-gcp-agent-based]] +== Agent-based deployment + +[discrete] +[[cspm-add-and-name-integration-gcp]] +=== Add your CSPM integration + +. Find **Integrations** in the navigation menu or use the global search field. +. Search for `CSPM`, then click on the result. +. Click **Add Cloud Security Posture Management (CSPM)**. +. Under **Configure integration**, select **GCP**, then either **GCP Organization** (recommended) or **Single Account**. +. Give your integration a name that matches the purpose or team of the GCP account you want to monitor, for example, `dev-gcp-project`. + +[discrete] +[[cspm-set-up-cloud-access-section-gcp]] +=== Set up cloud account access + +To set up CSPM for a GCP project, you need admin privileges for the project. + +For most users, the simplest option is to use a Google Cloud Shell script to automatically provision the necessary resources and permissions in your GCP account. This method, as well as two manual options, are described below. + +[discrete] +[[cspm-set-up-cloudshell]] +== Cloud Shell script setup (recommended) + +. Under **Setup Access**, select **Google Cloud Shell**. Enter your GCP Project ID, and for GCP Organization deployments, your GCP Organization ID. +. Under **Where to add this integration**: ++ +.. Select **New Hosts**. +.. Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`. +.. Click **Save and continue**, then **Add {agent} to your hosts**. The **Add agent** wizard appears and provides {agent} binaries, which you can download and deploy to a VM in your GCP account. +. Click **Save and continue**. +. Copy the command that appears, then click **Launch Google Cloud Shell**. It opens in a new window. +. Check the box to trust Elastic's `cloudbeat` repo, then click **Confirm** ++ +[role="screenshot"] +image::images/cspm-get-started-gcp/-cloud-native-security-cspm-cloudshell-trust.png[The cloud shell confirmation popup] +. In Google Cloud Shell, execute the command you copied. Once it finishes, return to {kib} and wait for the confirmation of data received from your new integration. Then you can click **View Assets** to see your data. + +[NOTE] +==== +During Cloud Shell setup, the CSPM integration adds roles to Google's default service account, which enables custom role creation and attachment of the service account to a compute instance. +After setup, these roles are removed from the service account. If you attempt to delete the deployment but find the deployment manager lacks necessary permissions, consider adding the missing roles to the service account: +https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectIamAdmin[Project IAM Admin], https://cloud.google.com/iam/docs/understanding-roles#iam.roleAdmin[Role Administrator]. +==== + +[discrete] +[[cspm-manual-auth-org]] +== Manual authentication (GCP organization) + +To authenticate manually to monitor a GCP organization, you'll need to create a new GCP service account, assign it the necessary roles, generate credentials, then provide those credentials to the CSPM integration. + +Use the following commands, after replacing `` with the name of your new service account, `` with your GCP organization's ID, and `` with the GCP project ID of the project where you want to provision the compute instance that will run CSPM. + +Create a new service account: + +[source,shell] +---- +gcloud iam service-accounts create \ + --description="Elastic agent service account for CSPM" \ + --display-name="Elastic agent service account for CSPM" \ + --project= +---- + +Assign the necessary roles to the service account: + +[source,shell] +---- +gcloud organizations add-iam-policy-binding \ + --member=serviceAccount:@.iam.gserviceaccount.com \ + --role=roles/cloudasset.viewer + +gcloud organizations add-iam-policy-binding \ + --member=serviceAccount:@.iam.gserviceaccount.com \ + --role=roles/browser +---- + +The `Cloud Asset Viewer` role grants read access to cloud asset metadata. The `Browser` role grants read access to the project hierarchy. + +Download the credentials JSON (first, replace `` with the location where you want to save it): + +[source,shell] +---- +gcloud iam service-accounts keys create \ + --iam-account=@.iam.gserviceaccount.com +---- + +Keep the credentials JSON in a secure location; you will need it later. + +Provide credentials to the CSPM integration: + +. On the CSPM setup screen under **Setup Access**, select **Manual**. +. Enter your GCP **Organization ID**. Enter the GCP **Project ID** of the project where you want to provision the compute instance that will run CSPM. +. Select **Credentials JSON**, and enter the value you generated earlier. +. Under **Where to add this integration**, select **New Hosts**. +. Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`. +. Click **Save and continue**, then follow the instructions to install {agent} in your chosen GCP project. + +Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data. + +[discrete] +[[cspm-manual-auth-proj]] +== Manual authentication (GCP project) + +To authenticate manually to monitor an individual GCP project, you'll need to create a new GCP service account, assign it the necessary roles, generate credentials, then provide those credentials to the CSPM integration. + +Use the following commands, after replacing `` with the name of your new service account, and `` with your GCP project ID. + +Create a new service account: + +[source,shell] +---- +gcloud iam service-accounts create \ + --description="Elastic agent service account for CSPM" \ + --display-name="Elastic agent service account for CSPM" \ + --project= +---- + +Assign the necessary roles to the service account: + +[source,shell] +---- +gcloud projects add-iam-policy-binding \ + --member=serviceAccount:@.iam.gserviceaccount.com \ + --role=roles/cloudasset.viewer + +gcloud projects add-iam-policy-binding \ + --member=serviceAccount:@.iam.gserviceaccount.com \ + --role=roles/browser +---- + +[NOTE] +==== +The `Cloud Asset Viewer` role grants read access to cloud asset metadata. The `Browser` role grants read access to the project hierarchy. +==== + +Download the credentials JSON (first, replace `` with the location where you want to save it): + +[source,shell] +---- +gcloud iam service-accounts keys create \ + --iam-account=@.iam.gserviceaccount.com +---- + +Keep the credentials JSON in a secure location; you will need it later. + +Provide credentials to the CSPM integration: + +. On the CSPM setup screen under **Setup Access**, select **Manual**. +. Enter your GCP **Project ID**. +. Select **Credentials JSON**, and enter the value you generated earlier. +. Under **Where to add this integration**, select **New Hosts**. +. Name the policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`. +. Click **Save and continue**, then follow the instructions to install the agent in your chosen GCP project. + +Wait for the confirmation that Kibana received data from your new integration. Then you can click **View Assets** to see your data. diff --git a/docs/serverless/cloud-native-security/cspm-get-started.asciidoc b/docs/serverless/cloud-native-security/cspm-get-started.asciidoc new file mode 100644 index 0000000000..5883b97b99 --- /dev/null +++ b/docs/serverless/cloud-native-security/cspm-get-started.asciidoc @@ -0,0 +1,349 @@ +[[security-cspm-get-started]] += Get started with CSPM for AWS + +// :description: Start monitoring the security posture of your AWS cloud assets. +// :keywords: serverless, security, overview, cloud security + +preview:[] + +[discrete] +[[cspm-overview]] +== Overview + +This page explains how to get started monitoring the security posture of your cloud assets using the Cloud Security Posture Management (CSPM) feature. + +.Requirements +[NOTE] +==== +* CSPM only works in the `Default` {kib} space. Installing the CSPM integration on a different {kib} space will not work. +* CSPM is supported only on AWS, GCP, and Azure commercial cloud platforms, and AWS GovCloud. Other government cloud platforms are not supported (https://github.com/elastic/kibana/issues/new/choose[request support]). +* To view posture data, you need the appropriate user role to read the following {es} indices: ++ +** `logs-cloud_security_posture.findings_latest-*` +** `logs-cloud_security_posture.scores-*` +** `Logs-cloud_security_posture.findings` +* The user who gives the CSPM integration AWS permissions must be an AWS account `admin`. +==== + +[discrete] +[[cspm-setup]] +== Set up CSPM for AWS + +You can set up CSPM for AWS either by enrolling a single cloud account, or by enrolling an organization containing multiple accounts. Either way, first you will add the CSPM integration, then enable cloud account access. Two deployment technologies are available: agentless, and agent-based. <> allows you to collect cloud posture data without having to manage the deployment of an {agent} in your cloud. <> requires you to deploy and manage an {agent} in the cloud account you want to monitor. + +[discrete] +[[cspm-aws-agentless]] +== Agentless deployment + +beta:[] + +. Find **Integrations** in the navigation menu or use the global search field. +. Search for `CSPM`, then click on the result. +. Click *Add Cloud Security Posture Management (CSPM)*. +. Select *AWS*, then either *AWS Organization* to onboard multiple accounts, or *Single Account* to onboard an individual account. +. Give your integration a name that matches the purpose or team of the AWS account/organization you want to monitor, for example, `dev-aws-account`. +. Click **Advanced options**, then select **Agentless (BETA)**. +. Next, you'll need to authenticate to AWS. Two methods are available: +.. Option 1: Direct access keys/CloudFormation (Recommended). Under **Preferred method** select **Direct access keys**. Expand the **Steps to Generate AWS Account Credentials** section, then follow the displayed instructions to automatically create the necessary credentials using CloudFormation. +.. Option 2: Temporary keys. To authenticate using temporary keys, refer to the instructions for <>. +. Once you've selected an authentication method and provided all necessary credentials, click **Save and continue** to finish deployment. Your data should start to appear within a few minutes. + +[discrete] +[[cspm-aws-agent-based]] +== Agent-based deployment + +[discrete] +[[cspm-add-and-name-integration]] +=== Add the CSPM integration + +. Find **Integrations** in the navigation menu or use the global search field. +. Search for `CSPM`, then click on the result. +. Click **Add Cloud Security Posture Management (CSPM)**. +. Select **AWS**, then either **AWS Organization** to onboard multiple accounts, or **Single Account** to onboard an individual account. +. Give your integration a name that matches the purpose or team of the AWS account/organization you want to monitor, for example, `dev-aws-account`. + +[discrete] +[[cspm-set-up-cloud-access-section]] +=== Set up cloud account access + +The CSPM integration requires access to AWS's built-in https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_job-functions.html#jf_security-auditor[`SecurityAudit` IAM policy] in order to discover and evaluate resources in your cloud account. There are several ways to provide access. + +For most use cases, the simplest option is to use AWS CloudFormation to automatically provision the necessary resources and permissions in your AWS account. This method, as well as several manual options, are described below. + +[discrete] +[[cspm-set-up-cloudformation]] +=== CloudFormation (recommended) + +. In the **Add Cloud Security Posture Management (CSPM) integration** menu, under **Setup Access**, select **CloudFormation**. +. In a new browser tab or window, log in as an admin to the AWS account or organization you want to onboard. +. Return to your {kib} tab. Click **Save and continue** at the bottom of the page. +. Review the information, then click **Launch CloudFormation**. +. A CloudFormation template appears in a new browser tab. +. For organization-level deployments only, you must enter the ID of the organizational unit where you want to deploy into the `OrganizationalUnitIds` field in the CloudFormation template. You can find it in the AWS console under **AWS Organizations → AWS Accounts** (it appears under the organization name). +. (Optional) Switch to the AWS region where you want to deploy using the controls in the upper right corner. +. Tick the checkbox under **Capabilities** to authorize the creation of necessary resources. ++ +[role="screenshot"] +image::images/cspm-get-started/-cloud-native-security-cspm-cloudformation-template.png[The Add permissions screen in AWS] +. At the bottom of the template, select **Create stack**. + +When you return to {kib}, click **View assets** to review the data being collected by your new integration. + +[discrete] +[[cspm-setup-organization-manual]] +=== Manual authentication for organization-level onboarding + +[NOTE] +==== +If you're onboarding a single account instead of an organization, skip this section. +==== + +When using manual authentication to onboard at the organization level, you need to configure the necessary permissions using the AWS console for the organization where you want to deploy: + +* In the organization's management account (root account), create an IAM role called `cloudbeat-root` (the name is important). The role needs several policies: ++ +** The following inline policy: ++ +.Click to expand policy +[%collapsible] +===== +[source,json] +---- +{ + "Version": "2012-10-17", + "Statement": [ + { + "Action": [ + "organizations:List*", + "organizations:Describe*" + ], + "Resource": "*", + "Effect": "Allow" + }, + { + "Action": [ + "sts:AssumeRole" + ], + "Resource": "*", + "Effect": "Allow" + } + ] +} +---- +===== ++ +** The following trust policy: ++ +.Click to expand policy +[%collapsible] +===== +[source,json] +---- +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": "arn:aws:iam:::root" + }, + "Action": "sts:AssumeRole" + }, + { + "Effect": "Allow", + "Principal": { + "Service": "ec2.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +} +---- +===== ++ +** The AWS-managed `SecurityAudit` policy. + +[IMPORTANT] +==== +You must replace `` in the trust policy with your AWS account ID. +==== + +* Next, for each account you want to scan in the organization, create an IAM role named `cloudbeat-securityaudit` with the following policies: ++ +** The AWS-managed `SecurityAudit` policy. +** The following trust policy: ++ +.Click to expand policy +[%collapsible] +===== +[source,json] +---- +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": "arn:aws:iam:::role/cloudbeat-root" + }, + "Action": "sts:AssumeRole" + } + ] +} +---- +===== + +[IMPORTANT] +==== +You must replace `` in the trust policy with your AWS account ID. +==== + +After creating the necessary roles, authenticate using one of the manual authentication methods. + +[IMPORTANT] +==== +When deploying to an organization using any of the authentication methods below, you need to make sure that the credentials you provide grant permission to assume `cloudbeat-root` privileges. +==== + +[discrete] +[[cspm-set-up-manual]] +=== Manual authentication methods + +* <> +* <> +* <> +* <> +* <> + +[IMPORTANT] +==== +Whichever method you use to authenticate, make sure AWS’s built-in https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_job-functions.html#jf_security-auditor[`SecurityAudit` IAM policy] is attached. +==== + +[discrete] +[[cspm-use-instance-role]] +==== Option 1 - Default instance role + +[NOTE] +==== +If you are deploying to an AWS organization instead of an AWS account, you should already have <>, `cloudbeat-root`. Skip to step 2 "Attach your new IAM role to an EC2 instance", and attach this role. You can use either an existing or new EC2 instance. +==== + +Follow AWS's https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html[IAM roles for Amazon EC2] documentation to create an IAM role using the IAM console, which automatically generates an instance profile. + +. Create an IAM role: ++ +.. In AWS, go to your IAM dashboard. Click **Roles**, then **Create role**. +.. On the **Select trusted entity** page, under **Trusted entity type**, select **AWS service**. +.. Under **Use case**, select **EC2**. Click **Next**. ++ +[role="screenshot"] +image::images/cspm-get-started/-cloud-native-security-cspm-aws-auth-1.png[The Select trusted entity screen in AWS] +.. On the **Add permissions** page, search for and select `SecurityAudit`. Click **Next**. ++ +[role="screenshot"] +image::images/cspm-get-started/-cloud-native-security-cspm-aws-auth-2.png[The Add permissions screen in AWS] +.. On the **Name, review, and create** page, name your role, then click **Create role**. +. Attach your new IAM role to an EC2 instance: ++ +.. In AWS, select an EC2 instance. +.. Select **Actions → Security → Modify IAM role**. ++ +[role="screenshot"] +image::images/cspm-get-started/-cloud-native-security-cspm-aws-auth-3.png[The EC2 page in AWS, showing the Modify IAM role option] +.. On the **Modify IAM role** page, search for and select your new IAM role. +.. Click **Update IAM role**. +.. Return to {kib} and <>. + +[IMPORTANT] +==== +Make sure to deploy the CSPM integration to this EC2 instance. When completing setup in {kib}, in the **Setup Access** section, select **Assume role**. Leave **Role ARN** empty for agentless deployments. For agent-based deployments, leave it empty unless you want to specify a role the {agent} should assume instead of the default role for your EC2 instance. Click **Save and continue**. +==== + +[discrete] +[[cspm-use-keys-directly]] +==== Option 2 - Direct access keys + +Access keys are long-term credentials for an IAM user or AWS account root user. To use access keys as credentials, you must provide the `Access key ID` and the `Secret Access Key`. After you provide credentials, <>. + +For more details, refer to https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html[Access Keys and Secret Access Keys]. + +[IMPORTANT] +==== +You must select **Programmatic access** when creating the IAM user. +==== + +[discrete] +[[cspm-use-temp-credentials]] +==== Option 3 - Temporary security credentials + +You can configure temporary security credentials in AWS to last for a specified duration. They consist of an access key ID, a secret access key, and a session token, which is typically found using `GetSessionToken`. + +Because temporary security credentials are short term, once they expire, you will need to generate new ones and manually update the integration's configuration to continue collecting cloud posture data. Update the credentials before they expire to avoid data loss. + +[NOTE] +==== +IAM users with multi-factor authentication (MFA) enabled need to submit an MFA code when calling `GetSessionToken`. For more details, refer to AWS's https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html[Temporary Security Credentials] documentation. +==== + +You can use the AWS CLI to generate temporary credentials. For example, you could use the following command if you have MFA enabled: + +[source,console] +---- +sts get-session-token --serial-number arn:aws:iam::1234:mfa/your-email@example.com --duration-seconds 129600 --token-code 123456 +---- + +The output from this command includes the following fields, which you should provide when configuring the KSPM integration: + +* `Access key ID`: The first part of the access key. +* `Secret Access Key`: The second part of the access key. +* `Session Token`: The required token when using temporary security credentials. + +After you provide credentials, <>. + +[discrete] +[[cspm-use-a-shared-credentials-file]] +==== Option 4 - Shared credentials file + +If you use different AWS credentials for different tools or applications, you can use profiles to define multiple access keys in the same configuration file. For more details, refer to AWS' https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html[Shared Credentials Files] documentation. + +Instead of providing the `Access key ID` and `Secret Access Key` to the integration, provide the information required to locate the access keys within the shared credentials file: + +* `Credential Profile Name`: The profile name in the shared credentials file. +* `Shared Credential File`: The directory of the shared credentials file. + +If you don't provide values for all configuration fields, the integration will use these defaults: + +* If `Access key ID`, `Secret Access Key`, and `ARN Role` are not provided, then the integration will check for `Credential Profile Name`. +* If there is no `Credential Profile Name`, the default profile will be used. +* If `Shared Credential File` is empty, the default directory will be used. ++ +** For Linux or Unix, the shared credentials file is located at `~/.aws/credentials`. + +After providing credentials, <>. + +[discrete] +[[cspm-use-iam-arn]] +==== Option 5 - IAM role Amazon Resource Name (ARN) + +An IAM role Amazon Resource Name (ARN) is an IAM identity that you can create in your AWS account. You define the role's permissions. Roles do not have standard long-term credentials such as passwords or access keys. Instead, when you assume a role, it provides temporary security credentials for your session. + +To use an IAM role ARN, select **Assume role** under **Preferred manual method**, enter the ARN, and continue to Finish manual setup. + +[discrete] +[[cspm-finish-manual]] +=== Finish manual setup + +Once you’ve provided AWS credentials, under **Where to add this integration**: + +If you want to monitor an AWS account or organization where you have not yet deployed {agent}: + +* Select **New Hosts**. +* Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-aws-account`. +* Click **Save and continue**, then **Add {agent} to your hosts**. The **Add agent** wizard appears and provides {agent} binaries, which you can download and deploy to your AWS account. + +If you want to monitor an AWS account or organization where you have already deployed {agent}: + +* Select **Existing hosts**. +* Select an agent policy that applies the AWS account you want to monitor. +* Click **Save and continue**. diff --git a/docs/serverless/cloud-native-security/d4c-get-started.asciidoc b/docs/serverless/cloud-native-security/d4c-get-started.asciidoc new file mode 100644 index 0000000000..e23b27fc8d --- /dev/null +++ b/docs/serverless/cloud-native-security/d4c-get-started.asciidoc @@ -0,0 +1,92 @@ +[[security-d4c-get-started]] += Get started with CWP + +// :description: Secure your containerized workloads and start detecting threats and vulnerabilities. +// :keywords: security, how-to, get-started, cloud security + +preview:[] + +beta:[] + +This page describes how to set up Cloud Workload Protection (CWP) for Kubernetes. + +.Requirements +[NOTE] +==== +* Kubernetes node operating systems must have Linux kernels 5.10.16 or higher. +==== + +[discrete] +[[security-d4c-get-started-initial-setup]] +== Initial setup + +First, you'll need to deploy Elastic's Defend for Containers integration to the Kubernetes clusters you wish to monitor. + +. Find **Container Workload Security** in the navigation menu or use the global search field. Click **Add D4C Integration**. +. Name the integration. The default name, which you can change, is `cloud_defend-1`. +. Optional — make any desired changes to the integration's policy by adjusting the **Selectors** and **Responses** sections. (For more information, refer to the <>). You can also change these later. +. Under **Where to add this integration**, select an existing or new agent policy. +. Click **Save & Continue**, then **Add {agent} to your hosts**. +. On the {agent} policy page, click **Add agent** to open the Add agent flyout. +. In the flyout, go to step 3 (**Install {agent} on your host**) and select the **Kubernetes** tab. +. Download or copy the manifest (`elastic-agent-managed-kubernetes.yml`). +. Open the manifest using your favorite editor, and uncomment the `#capabilities` section: ++ +[source,console] +---- +#capabilities: +# add: +# - BPF # (since Linux 5.8) allows loading of BPF programs, create most map types, load BTF, iterate programs and maps. +# - PERFMON # (since Linux 5.8) allows attaching of BPF programs used for performance metrics and observability operations. +# - SYS_RESOURCE # Allow use of special resources or raising of resource limits. Used by 'Defend for Containers' to modify 'rlimit_memlock' +---- +. From the directory where you saved the manifest, run the command `kubectl apply -f elastic-agent-managed-kubernetes.yml`. +. Wait for the **Confirm agent enrollment** dialogue to show that data has started flowing from your newly-installed agent, then click **Close**. + +[discrete] +[[d4c-get-started-threat]] +== Get started with threat detection + +One of the <> sends process telemetry events (`fork` and `exec`) to {es}. + +In order to detect threats using this data, you'll need active <>. Elastic has prebuilt detection rules designed for this data. (You can also create your own <>.) + +To install and enable the prebuilt rules: + +. Find **Detection rules (SIEM)** in the navigation menu or use the global search field, then click **Add Elastic rules**. +. Click the **Tags** filter next to the search bar, and search for the `Data Source: Elastic Defend for Containers` tag. +. Select all the displayed rules, then click **Install _x_ selected rule(s)**. +. Return to the **Rules** page. Click the **Tags** filter next to the search bar, and search for the `Data Source: Elastic Defend for Containers` tag. +. Select all the rules with the tag, and then click **Bulk actions → Enable**. + +[discrete] +[[d4c-get-started-drift]] +== Get started with drift detection and prevention + +{elastic-sec} defines container drift as the creation or modification of an executable within a container. Blocking drift restricts the number of attack vectors available to bad actors by prohibiting them from using external tools. + +To enable drift detection, you can use the default D4C policy: + +. Make sure the <> is active. +. Make sure you enabled at least the "Container Workload Protection" rule, by following the steps to install prebuilt rules, above. + +To enable drift prevention, create a new policy: + +. Find **Container Workload Security** in the navigation menu or use the global search field, then select your integration. +. Under **Selectors**, click **Add selector → File Selector**. By default, it selects the operations `createExecutable` and `modifyExecutable`. +. Name the selector, for example: `blockDrift`. +. Scroll down to the **Responses** section and click **Add response → File Response**. +. Under **Match selectors**, add the name of your new selector, for example: `blockDrift`. +. Select the **Alert** and **Block** actions. +. Click **Save integration**. + +[IMPORTANT] +==== +Before you enable blocking, we strongly recommend you observe a production workload that's using the default D4C policy to ensure that the workload does not create or modify executables as part of its normal operation. +==== + +[discrete] +[[d4c-get-started-validation]] +== Policy validation + +To ensure the stability of your production workloads, you should test policy changes before implementing them in production workloads. We also recommend you test policy changes on a simulated environment with workloads similar to production. This approach allows you to test that policy changes prevent undesirable behavior without disrupting your production workloads. diff --git a/docs/serverless/cloud-native-security/environment-variable-capture.asciidoc b/docs/serverless/cloud-native-security/environment-variable-capture.asciidoc new file mode 100644 index 0000000000..311796c7a3 --- /dev/null +++ b/docs/serverless/cloud-native-security/environment-variable-capture.asciidoc @@ -0,0 +1,42 @@ +[[security-environment-variable-capture]] += Capture environment variables + +// :description: Capture environment variables from monitored Linux sessions. +// :keywords: serverless, security, overview, cloud security + +preview:[] + +You can configure an {agent} policy to capture up to five environment variables (`env vars`). + +[NOTE] +==== +* Env var names must be no more than 63 characters, and env var values must be no more than 1023 characters. Values outside these limits are silently ignored. +* Env var names are case sensitive. +==== + +To set up environment variable capture for an {agent} policy: + +. Find **Policies** in the navigation menu or use the global search field. +. Select an {agent} policy. +. Click **Show advanced settings**. +. Scroll down or search for `linux.advanced.capture_env_vars`, or `mac.advanced.capture_env_vars`. +. Enter the names of env vars you want to capture, separated by commas. For example: `PATH,USER` +. Click **Save**. + +[role="screenshot"] +image::images/environment-variable-capture/-cloud-native-security-env-var-capture.png[The "linux.advanced.capture_env_vars" advanced agent policy setting] + +[discrete] +[[find-cap-env-vars]] +== Find captured environment variables + +Captured environment variables are associated with process events, and appear in each event's `process.env_vars` field. + +To view environment variables in the **Events** table: + +. Click the **Events** tab on the **Hosts**, **Network**, or **Users** pages, then click **Fields** in the Events table. +. Search for the `process.env_vars` field, select it, and click **Close**. +A new column appears containing captured environment variable data. + +[role="screenshot"] +image::images/environment-variable-capture/-cloud-native-security-env-var-capture-detail.png[The Events table with the "process.env_vars" column highlighted] diff --git a/docs/serverless/cloud-native-security/get-started-with-kspm.asciidoc b/docs/serverless/cloud-native-security/get-started-with-kspm.asciidoc new file mode 100644 index 0000000000..2380fb9fef --- /dev/null +++ b/docs/serverless/cloud-native-security/get-started-with-kspm.asciidoc @@ -0,0 +1,446 @@ +[[security-get-started-with-kspm]] += Get started with KSPM + +// :keywords: serverless, security, overview, cloud security + +preview:[] + +This page explains how to configure the Kubernetes Security Posture Management (KSPM) integration. + +.Requirements +[NOTE] +==== +* KSPM only works in the `Default` {kib} space. Installing the KSPM integration on a different {kib} space will not work. +* KSPM is not supported on EKS clusters in AWS GovCloud (https://github.com/elastic/kibana/issues/new/choose[request support]). +* To view posture data, ensure you have the appropriate user role to read the following {es} indices: + +* `logs-cloud_security_posture.findings_latest-*` +* `logs-cloud_security_posture.scores-*` +* `logs-cloud_security_posture.findings` +==== + +The instructions differ depending on whether you're installing on EKS or on unmanaged clusters. + +* Install on EKS-managed clusters: ++ +.. <> +.. <> +.. <> +.. <> +* Install on unmanaged clusters: ++ +.. <> +.. <> + +[discrete] +[[kspm-setup-eks-start]] +== Set up KSPM for Amazon EKS clusters + +[discrete] +[[security-get-started-with-kspm-name-your-integration-and-select-a-kubernetes-deployment-type]] +=== Name your integration and select a Kubernetes Deployment type + +. Find **Cloud Security Posture** in the navigation menu or use the global search field. +. Click **Add a KSPM integration**. +. Read the integration's description to understand how it works. Then, click {integrations-docs}/cloud_security_posture[_Add Kubernetes Security Posture Management_]. +. Name your integration. Use a name that matches the purpose or team of the cluster(s) you want to monitor, for example, `IT-dev-k8s-clusters`. +. Select **EKS** from the **Kubernetes Deployment** menu. A new section for AWS credentials will appear. + +[discrete] +[[kspm-setup-eks-auth]] +=== Authenticate to AWS + +There are several options for how to provide AWS credentials: + +* <> +* <> +* <> +* <> +* <> +* <> + +Regardless of which option you use, you'll need to grant the following permissions: + +[source,console] +---- +ecr:GetRegistryPolicy, +eks:ListTagsForResource +elasticloadbalancing:DescribeTags +ecr-public:DescribeRegistries +ecr:DescribeRegistry +elasticloadbalancing:DescribeLoadBalancerPolicyTypes +ecr:ListImages +ecr-public:GetRepositoryPolicy +elasticloadbalancing:DescribeLoadBalancerAttributes +elasticloadbalancing:DescribeLoadBalancers +ecr-public:DescribeRepositories +eks:DescribeNodegroup +ecr:DescribeImages +elasticloadbalancing:DescribeLoadBalancerPolicies +ecr:DescribeRepositories +eks:DescribeCluster +eks:ListClusters +elasticloadbalancing:DescribeInstanceHealth +ecr:GetRepositoryPolicy +---- + +If you are using the AWS visual editor to create and modify your IAM Policies, you can copy and paste this IAM policy JSON object: + +.Click to view JSON object +[%collapsible] +===== +[source,json] +---- +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "VisualEditor0", + "Effect": "Allow", + "Action": [ + "ecr:GetRegistryPolicy", + "eks:ListTagsForResource", + "elasticloadbalancing:DescribeTags", + "ecr-public:DescribeRegistries", + "ecr:DescribeRegistry", + "elasticloadbalancing:DescribeLoadBalancerPolicyTypes", + "ecr:ListImages", + "ecr-public:GetRepositoryPolicy", + "elasticloadbalancing:DescribeLoadBalancerAttributes", + "elasticloadbalancing:DescribeLoadBalancers", + "ecr-public:DescribeRepositories", + "eks:DescribeNodegroup", + "ecr:DescribeImages", + "elasticloadbalancing:DescribeLoadBalancerPolicies", + "ecr:DescribeRepositories", + "eks:DescribeCluster", + "eks:ListClusters", + "elasticloadbalancing:DescribeInstanceHealth", + "ecr:GetRepositoryPolicy" + ], + "Resource": "*" + } + ] +} +---- +===== + +[discrete] +[[kspm-use-irsa]] +==== Option 1 - [Recommended] Use Kubernetes Service Account to assume IAM role + +Follow AWS's https://aws.github.io/aws-eks-best-practices/security/docs/iam/#iam-roles-for-service-accounts-irsa[EKS Best Practices] documentation to use the https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html[IAM Role to Kubernetes Service-Account] (IRSA) feature to get temporary credentials and scoped permissions. + +[IMPORTANT] +==== +During setup, do not fill in any option in the "Setup Access" section. Click **Save and continue**. +==== + +[discrete] +[[kspm-use-instance-role]] +==== Option 2 - Use default instance role + +Follow AWS's https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html[IAM roles for Amazon EC2] documentation to create an IAM role using the IAM console, which automatically generates an instance profile. + +[IMPORTANT] +==== +During setup, do not fill in any option in the "Setup Access" section. Click **Save and continue**. +==== + +[discrete] +[[kspm-use-keys-directly]] +==== Option 3 - Use access keys directly + +Access keys are long-term credentials for an IAM user or AWS account root user. To use access keys as credentials, you must provide the `Access key ID` and the `Secret Access Key`. + +For more details, refer to AWS' https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html[Access Keys and Secret Access Keys] documentation. + +[IMPORTANT] +==== +You must select "Programmatic access" when creating the IAM user. +==== + +[discrete] +[[kspm-use-temp-credentials]] +==== Option 4 - Use temporary security credentials + +You can configure temporary security credentials in AWS to last for a specified duration. They consist of an access key ID, a secret access key, and a security token, which is typically found using `GetSessionToken`. + +Because temporary security credentials are short term, once they expire, you will need to generate new ones and manually update the integration's configuration to continue collecting cloud posture data. Update the credentials before they expire to avoid data loss. + +[NOTE] +==== +IAM users with multi-factor authentication (MFA) enabled need to submit an MFA code when calling `GetSessionToken`. For more details, refer to AWS' https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html[Temporary Security Credentials] documentation. +==== + +You can use the AWS CLI to generate temporary credentials. For example, you could use the following command if you have MFA enabled: + +[source,console] +---- +`sts get-session-token --serial-number arn:aws:iam::1234:mfa/your-email@example.com --duration-seconds 129600 --token-code 123456` +---- + +The output from this command includes the following fields, which you should provide when configuring the KSPM integration: + +* `Access key ID`: The first part of the access key. +* `Secret Access Key`: The second part of the access key. +* `Session Token`: A token required when using temporary security credentials. + +[discrete] +[[kspm-use-a-shared-credentials-file]] +==== Option 5 - Use a shared credentials file + +If you use different AWS credentials for different tools or applications, you can use profiles to define multiple access keys in the same configuration file. For more details, refer to AWS' https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html[Shared Credentials Files] documentation. + +Instead of providing the `Access key ID` and `Secret Access Key` to the integration, provide the information required to locate the access keys within the shared credentials file: + +* `Credential Profile Name`: The profile name in the shared credentials file. +* `Shared Credential File`: The directory of the shared credentials file. + +If you don't provide values for all configuration fields, the integration will use these defaults: + +* If `Access key ID`, `Secret Access Key`, and `ARN Role` are not provided, then the integration will check for `Credential Profile Name`. +* If there is no `Credential Profile Name`, the default profile will be used. +* If `Shared Credential File` is empty, the default directory will be used. ++ +** For Linux or Unix, the shared credentials file is located at `~/.aws/credentials`. + +[discrete] +[[kspm-use-iam-arn]] +==== Option 6 - Use an IAM role Amazon Resource Name (ARN) + +An IAM role Amazon Resource Name (ARN) is an IAM identity that you can create in your AWS account. You define the role's permissions. +Roles do not have standard long-term credentials such as passwords or access keys. +Instead, when you assume a role, it provides temporary security credentials for your session. +An IAM role's ARN can be used to specify which AWS IAM role to use to generate temporary credentials. + +For more details, refer to AWS' https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html[AssumeRole API] documentation. +Follow AWS' instructions to https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html[create an IAM user], and define the IAM role's permissions using the JSON permissions policy above. + +To use an IAM role's ARN, you need to provide either a <> or <> along with the `ARN role`. +The `ARN Role` value specifies which AWS IAM role to use for generating temporary credentials. + +[NOTE] +==== +If `ARN Role` is present, the integration will check if `Access key ID` and `Secret Access Key` are present. +If not, the package will check for a `Credential Profile Name`. +If a `Credential Profile Name` is not present, the default credential profile will be used. +==== + +[discrete] +[[kspm-setup-eks-finish]] +=== Finish configuring the KSPM integration for EKS + +Once you've provided AWS credentials, finish configuring the KSPM integration: + +. If you want to monitor Kubernetes clusters that aren’t yet enrolled in {fleet}, select **New Hosts** under “where to add this integration”. +. Name the {agent} policy. Use a name that matches the purpose or team of the cluster(s) you want to monitor. For example, `IT-dev-k8s-clusters`. +. Click **Save and continue**, then **Add agent to your hosts**. The **Add agent** wizard appears and provides a DaemonSet manifest `.yaml` file with pre-populated configuration information, such as the `Fleet ID` and `Fleet URL`. + +[discrete] +[[kspm-setup-eks-modify-deploy]] +=== Deploy the KSPM integration to EKS clusters + +The **Add agent** wizard helps you deploy the KSPM integration on the Kubernetes clusters you wish to monitor. For each cluster: + +. Download the manifest and make any necessary revisions to its configuration to suit the needs of your environment. +. Apply the manifest using the `kubectl apply -f` command. For example: `kubectl apply -f elastic-agent-managed-kubernetes.yaml` + +After a few minutes, a message confirming the {agent} enrollment appears, followed by a message confirming that data is incoming. You can then click **View assets** to see where the newly-collected configuration information appears, including the <> and the <>. + +[discrete] +[[kspm-setup-unmanaged]] +== Set up KSPM for unmanaged Kubernetes clusters + +Follow these steps to deploy the KSPM integration to unmanaged clusters. Keep in mind credentials are NOT required for unmanaged deployments. + +[discrete] +[[security-get-started-with-kspm-configure-the-kspm-integration]] +=== Configure the KSPM integration + +To install the integration on unmanaged clusters: + +. Find **Connectors** in the navigation menu or use the global search field. +. Click **Add a KSPM integration**. +. Read the integration's description to understand how it works. Then, click {integrations-docs}/cloud_security_posture[_Add Kubernetes Security Posture Management_]. +. Name your integration. Use a name that matches the purpose or team of the cluster(s) you want to monitor, for example, `IT-dev-k8s-clusters`. +. Select **Unmanaged Kubernetes** from the **Kubernetes Deployment** menu. +. If you want to monitor Kubernetes clusters that aren’t yet enrolled in {fleet}, select **New Hosts** when choosing the {agent} policy. +. Select the {agent} policy where you want to add the integration. +. Click **Save and continue**, then **Add agent to your hosts**. The **Add agent** wizard appears and provides a DaemonSet manifest `.yaml` file with pre-populated configuration information, such as the `Fleet ID` and `Fleet URL`. + +[role="screenshot"] +image::images/get-started-with-kspm/-cloud-native-security-kspm-add-agent-wizard.png[The KSPM integration's Add agent wizard] + +[discrete] +[[kspm-setup-unmanaged-modify-deploy]] +=== Deploy the KSPM integration to unmanaged clusters + +The **Add agent** wizard helps you deploy the KSPM integration on the Kubernetes clusters you wish to monitor. To do this, for each cluster: + +. Download the manifest and make any necessary revisions to its configuration to suit the needs of your environment. +. Apply the manifest using the `kubectl apply -f` command. For example: `kubectl apply -f elastic-agent-managed-kubernetes.yaml` + +After a few minutes, a message confirming the {agent} enrollment appears, followed by a message confirming that data is incoming. You can then click **View assets** to see where the newly-collected configuration information appears, including the <> and the <>. + +[discrete] +[[kspm-eck]] +=== Set up KSPM on ECK deployments + +To run KSPM on an https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html[ECK] deployment, +you must edit the https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-configuration.html[Elastic Agent CRD] and https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-configuration.html#k8s-elastic-agent-role-based-access-control[Elastic Agent Cluster-Role] `.yaml` files. + +.Patch Elastic Agent +[%collapsible] +===== +Add `volumes` and `volumeMounts` to `podTemplate`: + +[source,yaml] +---- +podTemplate: + spec: + containers: + - name: agent + volumeMounts: + - name: proc + mountPath: /hostfs/proc + readOnly: true + - name: cgroup + mountPath: /hostfs/sys/fs/cgroup + readOnly: true + - name: varlibdockercontainers + mountPath: /var/lib/docker/containers + readOnly: true + - name: varlog + mountPath: /var/log + readOnly: true + - name: etc-full + mountPath: /hostfs/etc + readOnly: true + - name: var-lib + mountPath: /hostfs/var/lib + readOnly: true + - name: etc-mid + mountPath: /etc/machine-id + readOnly: true + volumes: + - name: proc + hostPath: + path: /proc + - name: cgroup + hostPath: + path: /sys/fs/cgroup + - name: varlibdockercontainers + hostPath: + path: /var/lib/docker/containers + - name: varlog + hostPath: + path: /var/log + - name: etc-full + hostPath: + path: /etc + - name: var-lib + hostPath: + path: /var/lib + # Mount /etc/machine-id from the host to determine host ID + # Needed for Elastic Security integration + - name: etc-mid + hostPath: + path: /etc/machine-id + type: File +---- +===== + +.Patch RBAC +[%collapsible] +===== +Make sure that the `elastic-agent` service-account has the following Role and ClusterRole: + +[source,yaml] +---- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + namespace: default + name: elastic-agent +subjects: +- kind: ServiceAccount + name: elastic-agent + namespace: default +roleRef: + kind: Role + name: elastic-agent + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: elastic-agent + labels: + k8s-app: elastic-agent +rules: +- apiGroups: [""] + resources: + - nodes + - namespaces + - events + - pods + - services + - configmaps + - serviceaccounts + - persistentvolumes + - persistentvolumeclaims + verbs: ["get", "list", "watch"] +- apiGroups: ["extensions"] + resources: + - replicasets + verbs: ["get", "list", "watch"] +- apiGroups: ["apps"] + resources: + - statefulsets + - deployments + - replicasets + - daemonsets + verbs: ["get", "list", "watch"] +- apiGroups: + - "" + resources: + - nodes/stats + verbs: + - get +- apiGroups: [ "batch" ] + resources: + - jobs + - cronjobs + verbs: [ "get", "list", "watch" ] +- nonResourceURLs: + - "/metrics" + verbs: + - get +- apiGroups: ["rbac.authorization.k8s.io"] + resources: + - clusterrolebindings + - clusterroles + - rolebindings + - roles + verbs: ["get", "list", "watch"] +- apiGroups: ["policy"] + resources: + - podsecuritypolicies + verbs: ["get", "list", "watch"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: elastic-agent + namespace: default + labels: + k8s-app: elastic-agent +rules: + - apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: ["get", "create", "update"] +---- +===== diff --git a/docs/serverless/cloud-native-security/kspm.asciidoc b/docs/serverless/cloud-native-security/kspm.asciidoc new file mode 100644 index 0000000000..a5a9e8b200 --- /dev/null +++ b/docs/serverless/cloud-native-security/kspm.asciidoc @@ -0,0 +1,86 @@ +[[security-kspm]] += Kubernetes security posture management + +// :description: Identify configuration risks in your Kubernetes clusters. +// :keywords: serverless, security, overview, cloud security + +preview:[] + +[discrete] +[[kspm-overview]] +== Overview + +The Kubernetes Security Posture Management (KSPM) integration allows you to identify configuration risks in the various components that make up your Kubernetes cluster. +It does this by evaluating your Kubernetes clusters against secure configuration guidelines defined by the Center for Internet Security (CIS) and generating findings with step-by-step instructions for remediating potential security risks. + +This integration supports Amazon EKS and unmanaged Kubernetes clusters. For setup instructions, refer to <>. + +.Requirements +[NOTE] +==== +* KSPM only works in the `Default` {kib} space. Installing the KSPM integration on a different {kib} space will not work. +* KSPM is not supported on EKS clusters in AWS GovCloud (https://github.com/elastic/kibana/issues/new/choose[request support]). +* To view posture data, ensure you have the appropriate user role to read the following {es} indices: + +* `logs-cloud_security_posture.findings_latest-*` +* `logs-cloud_security_posture.scores-*` +* `logs-cloud_security_posture.findings` +==== + +[discrete] +[[kspm-how-kspm-works]] +== How KSPM works + +. When you add a KSPM integration, it generates a Kubernetes manifest. When applied to a cluster, the manifest deploys an {agent} as a https://kubernetes.io/docs/concepts/workloads/controllers/daemonset[DaemonSet] to ensure all nodes are evaluated. +. Upon deployment, the integration immediately assesses the security posture of your Kubernetes resources. The evaluation process repeats every four hours. +. After each evaluation, the integration sends findings to {es}. Findings appear on the <> and the <> page. + +[discrete] +[[kspm-use-cases]] +== Use cases + +The KSPM integration helps you to: + +* Identify and remediate `failed` findings +* Identify the most misconfigured resources +* Identify risks in particular CIS benchmark sections + +[discrete] +[[kspm-remediate-failed-findings]] +=== Identify and remediate failed findings + +To identify and remediate failed failed findings: + +. Go to the <>. +. Click **View all failed findings**, either for an individual cluster or for all monitored clusters. +. Click a failed finding. The findings flyout opens. +. Follow the steps under **Remediation** to correct the misconfiguration. ++ +[NOTE] +==== +Remediation steps typically include commands for you to execute. These sometimes contain placeholder values that you must replace before execution. +==== + +[discrete] +[[kspm-identify-misconfigured-resources]] +=== Identify the most misconfigured Kubernetes resources + +To identify the Kubernetes resources generating the most failed findings: + +. Go to the <> page. +. Click the **Group by** menu near the search box and select **Resource** to view a list of resources sorted by their total number of failed findings. +. Click a resource ID to view the findings associated with that resource. + +[discrete] +[[kspm-identify-config-risks-by-section]] +=== Identify configuration risks by CIS section + +To identify risks in particular CIS sections: + +. Go to the <>. +. In the Failed findings by CIS section widget, click the name of a CIS section to view all failed findings for that section. + +Alternatively: + +. Go to the Findings page. +. Filter by the `rule.section` field. For example, search for `rule.section : API Server` to view findings for benchmark rules in the API Server category. diff --git a/docs/serverless/cloud-native-security/security-posture-management.asciidoc b/docs/serverless/cloud-native-security/security-posture-management.asciidoc new file mode 100644 index 0000000000..0975975752 --- /dev/null +++ b/docs/serverless/cloud-native-security/security-posture-management.asciidoc @@ -0,0 +1,50 @@ +[[security-posture-management]] += Security posture management overview + +// :description: Discovers and evaluates your cloud services and resources against security best practices. +// :keywords: serverless, security, overview, cloud security + +preview:[] + +[discrete] +== Overview + +Elastic's <> (CSPM) and <> (KSPM) features help you discover and evaluate the services and resources in your cloud environment — like storage, compute, IAM, and more — against security guidelines defined by the Center for Internet Security (CIS). They help you identify and remediate configuration risks that could undermine the confidentiality, integrity, and availability of your cloud assets, such as publicly exposed storage buckets or overly permissive networking objects. + +The KSPM feature assesses the security of your Kubernetes assets, while the CSPM feature assesses the security of your AWS resources such as storage, compute, IAM, and more. + +[discrete] +[[security-posture-management-get-started]] +== Getting started + +For setup instructions, refer to: + +* <> +* <> + +[discrete] +[[security-posture-use-cases]] +== Use cases + +Using the data generated by these features, you can: + +**Identify and secure misconfigured infrastructure:** + +. Find **Cloud Security Posture** in the navigation menu or use the global search field. +. Click **View all failed findings**, either for an individual resource or a group of resources. +. Click a failed finding to open the Findings flyout. +. Follow the steps under Remediation to fix the misconfiguration. + +**Identify the CIS Sections (security best practice categories) with which your resources are least compliant:** + +. Find **Cloud Security Posture** in the navigation menu or use the global search field. +. Do one of the following: ++ +.. Under Failed findings by CIS section, click the name of a CIS section to view all failed findings from that section. +.. Go to the **Findings** page and filter by the `rule.section` field. For example, search for `rule.section : API Server` to view findings from the API Server category. + +**Identify your least compliant cloud resources** + +. Go to the **Findings** page. +. Click the **Group by** menu near the search box, and select **Resource** to sort resources by their number of failed findings. +. Click a resource ID to view associated findings. diff --git a/docs/serverless/cloud-native-security/vuln-management-get-started.asciidoc b/docs/serverless/cloud-native-security/vuln-management-get-started.asciidoc new file mode 100644 index 0000000000..b2a2157621 --- /dev/null +++ b/docs/serverless/cloud-native-security/vuln-management-get-started.asciidoc @@ -0,0 +1,77 @@ +[[security-vuln-management-get-started]] += Get started with CNVM + +// :description: Set up cloud native vulnerability management. +// :keywords: serverless, security, overview, cloud security + +preview:[] + +This page explains how to set up Cloud Native Vulnerability Management (CNVM). + +.Requirements +[NOTE] +==== +* CNVM only works in the `Default` {kib} space. Installing the CNVM integration on a different {kib} space will not work. +* Requires {agent} version 8.8 or higher. +* CNVM can only be deployed on ARM-based VMs. +* To view vulnerability scan findings, you need the appropriate user role to read the following indices: ++ +** `logs-cloud_security_posture.vulnerabilities-*` +** `logs-cloud_security_posture.vulnerabilities_latest-*` +* You need an AWS user account with permissions to perform the following actions: run CloudFormation templates, create IAM Roles and InstanceProfiles, and create EC2 SecurityGroups and Instances. +==== + +[NOTE] +==== +CNVM currently only supports AWS EC2 Linux workloads. +==== + +[discrete] +[[vuln-management-setup]] +== Set up CNVM for AWS + +To set up the CNVM integration for AWS, install the integration on a new {agent} policy, sign into the AWS account you want to scan, and run the https://docs.aws.amazon.com/cloudformation/index.html[CloudFormation] template. + +[IMPORTANT] +==== +Do not add the integration to an existing {agent} policy. It should always be added to a new policy since it should not run on VMs with existing workloads. For more information, refer to <>. +==== + +[discrete] +[[vuln-management-setup-step-1]] +=== Step 1: Add the CNVM integration + +. Find **Integrations** in the navigation menu or use the global search field. +. Search for **Cloud Native Vulnerability Management**, then click on the result. +. Click **Add Cloud Native Vulnerability Management**. +. Give your integration a name that matches its purpose or the AWS account region you want to scan for vulnerabilities (for example, `uswest2-aws-account`.) ++ +[role="screenshot"] +image::images/vuln-management-get-started/-dashboards-cnvm-setup-1.png[The CNVM integration setup page] +. Click **Save and continue**. The integration will create a new {agent} policy. +. Click **Add {agent} to your hosts**. + +[discrete] +[[vuln-management-setup-step-2]] +=== Step 2: Sign in to the AWS management console + +. Open a new browser tab and use it to sign into your AWS management console. +. Switch to the cloud region with the workloads that you want to scan for vulnerabilities. + +[IMPORTANT] +==== +The integration will only scan VMs in the region you select. To scan multiple regions, repeat this setup process for each region. +==== + +[discrete] +[[vuln-management-setup-step-3]] +=== Step 3: Run the CloudFormation template + +. Switch back to the tab with Elastic Security. +. Click **Launch CloudFormation**. The CloudFormation page appears. ++ +[role="screenshot"] +image::images/vuln-management-get-started/-dashboards-cnvm-cloudformation.png[The cloud formation template] +. Click **Create stack**. To avoid authentication problems, you can only make configuration changes to the VM InstanceType, which you could make larger to increase scanning speed. +. Wait for the confirmation that {agent} was enrolled. +. Your data will start to appear on the **Vulnerabilities** tab of the <>. From 2dd15cc9d2f1902bdbc8073144991a9b6548e703 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Tue, 12 Nov 2024 21:50:04 +0000 Subject: [PATCH 2/2] Delete docs/serverless directory and its contents --- .../connect-to-azure-openai.asciidoc | 117 ----- .../connect-to-bedrock.asciidoc | 167 ------- .../connect-to-byo-llm.asciidoc | 223 --------- .../connect-to-openai.asciidoc | 70 --- .../connect-to-vertex.asciidoc | 115 ----- .../benchmark-rules.asciidoc | 61 --- .../cspm-get-started-azure.asciidoc | 198 -------- .../cspm-get-started-gcp.asciidoc | 205 -------- .../cspm-get-started.asciidoc | 349 -------------- .../d4c-get-started.asciidoc | 92 ---- .../environment-variable-capture.asciidoc | 42 -- .../get-started-with-kspm.asciidoc | 446 ------------------ .../cloud-native-security/kspm.asciidoc | 86 ---- .../security-posture-management.asciidoc | 50 -- .../vuln-management-get-started.asciidoc | 77 --- 15 files changed, 2298 deletions(-) delete mode 100644 docs/serverless/AI-for-security/connect-to-azure-openai.asciidoc delete mode 100644 docs/serverless/AI-for-security/connect-to-bedrock.asciidoc delete mode 100644 docs/serverless/AI-for-security/connect-to-byo-llm.asciidoc delete mode 100644 docs/serverless/AI-for-security/connect-to-openai.asciidoc delete mode 100644 docs/serverless/AI-for-security/connect-to-vertex.asciidoc delete mode 100644 docs/serverless/cloud-native-security/benchmark-rules.asciidoc delete mode 100644 docs/serverless/cloud-native-security/cspm-get-started-azure.asciidoc delete mode 100644 docs/serverless/cloud-native-security/cspm-get-started-gcp.asciidoc delete mode 100644 docs/serverless/cloud-native-security/cspm-get-started.asciidoc delete mode 100644 docs/serverless/cloud-native-security/d4c-get-started.asciidoc delete mode 100644 docs/serverless/cloud-native-security/environment-variable-capture.asciidoc delete mode 100644 docs/serverless/cloud-native-security/get-started-with-kspm.asciidoc delete mode 100644 docs/serverless/cloud-native-security/kspm.asciidoc delete mode 100644 docs/serverless/cloud-native-security/security-posture-management.asciidoc delete mode 100644 docs/serverless/cloud-native-security/vuln-management-get-started.asciidoc diff --git a/docs/serverless/AI-for-security/connect-to-azure-openai.asciidoc b/docs/serverless/AI-for-security/connect-to-azure-openai.asciidoc deleted file mode 100644 index 3ff03fa1db..0000000000 --- a/docs/serverless/AI-for-security/connect-to-azure-openai.asciidoc +++ /dev/null @@ -1,117 +0,0 @@ -[[security-connect-to-azure-openai]] -= Connect to Azure OpenAI - -// :description: Set up an Azure OpenAI LLM connector. -// :keywords: security, overview, get-started - -This page provides step-by-step instructions for setting up an Azure OpenAI connector for the first time. This connector type enables you to leverage large language models (LLMs) within {kib}. You'll first need to configure Azure, then configure the connector in {kib}. - -[discrete] -[[security-connect-to-azure-openai-configure-azure]] -== Configure Azure - -[discrete] -[[security-connect-to-azure-openai-configure-a-deployment]] -=== Configure a deployment - -First, set up an Azure OpenAI deployment: - -. Log in to the Azure console and search for Azure OpenAI. -. In **Azure AI services**, select **Create**. -. For the **Project Details**, select your subscription and resource group. If you don't have a resource group, select **Create new** to make one. -. For **Instance Details**, select the desired region and specify a name, such as `example-deployment-openai`. -. Select the **Standard** pricing tier, then click **Next**. -. Configure your network settings, click **Next**, optionally add tags, then click **Next**. -. Review your deployment settings, then click **Create**. When complete, select **Go to resource**. - -The following video demonstrates these steps. - -++++ - -++++ - -[discrete] -[[security-connect-to-azure-openai-configure-keys]] -=== Configure keys - -Next, create access keys for the deployment: - -. From within your Azure OpenAI deployment, select **Click here to manage keys**. -. Store your keys in a secure location. - -The following video demonstrates these steps. - -++++ - -++++ - -[discrete] -[[security-connect-to-azure-openai-configure-a-model]] -=== Configure a model - -Now, set up the Azure OpenAI model: - -. From within your Azure OpenAI deployment, select **Model deployments**, then click **Manage deployments**. -. On the **Deployments** page, select **Create new deployment**. -. Under **Select a model**, choose `gpt-4o` or `gpt-4 turbo`. -. Set the model version to "Auto-update to default". -. Under **Deployment type**, select **Standard**. -. Name your deployment. -. Slide the **Tokens per Minute Rate Limit** to the maximum. The following example supports 80,000 TPM, but other regions might support higher limits. -. Click **Create**. - -[IMPORTANT] -==== -The models available to you will depend on https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#model-summary-table-and-region-availability[region availability]. For best results, use `GPT-4o 2024-05-13` with the maximum Tokens-Per-Minute (TPM) capacity. For more information on how different models perform for different tasks, refer to the <>. -==== - -The following video demonstrates these steps. - -++++ - -++++ - -[discrete] -[[security-connect-to-azure-openai-configure-elastic-ai-assistant]] -== Configure Elastic AI Assistant - -Finally, configure the connector in {kib}: - -. Log in to {kib}. -. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **OpenAI**. -. Give your connector a name to help you keep track of different models, such as `Azure OpenAI (GPT-4 Turbo v. 0125)`. -. For **Select an OpenAI provider**, choose **Azure OpenAI**. -. Update the **URL** field. We recommend doing the following: -+ -** Navigate to your deployment in Azure AI Studio and select **Open in Playground**. The **Chat playground** screen displays. -** Select **View code**, then from the drop-down, change the **Sample code** to `Curl`. -** Highlight and copy the URL without the quotes, then paste it into the **URL** field in {kib}. -** (Optional) Alternatively, refer to the https://learn.microsoft.com/en-us/azure/ai-services/openai/reference[API documentation] to learn how to create the URL manually. -. Under **API key**, enter one of your API keys. -. Click **Save & test**, then click **Run**. - -The following video demonstrates these steps. - -++++ - -++++ diff --git a/docs/serverless/AI-for-security/connect-to-bedrock.asciidoc b/docs/serverless/AI-for-security/connect-to-bedrock.asciidoc deleted file mode 100644 index 9581cf8e32..0000000000 --- a/docs/serverless/AI-for-security/connect-to-bedrock.asciidoc +++ /dev/null @@ -1,167 +0,0 @@ -[[security-connect-to-bedrock]] -= Connect to Amazon Bedrock - -// :description: Set up an Amazon Bedrock LLM connector. -// :keywords: security, overview, get-started - -This page provides step-by-step instructions for setting up an Amazon Bedrock connector for the first time. This connector type enables you to leverage large language models (LLMs) within {kib}. You'll first need to configure AWS, then configure the connector in {kib}. - -[NOTE] -==== -Only Amazon Bedrock's `Anthropic` models are supported: `Claude` and `Claude instant`. -==== - -[discrete] -[[security-connect-to-bedrock-configure-aws]] -== Configure AWS - -[discrete] -[[security-connect-to-bedrock-configure-an-iam-policy]] -=== Configure an IAM policy - -First, configure an IAM policy with the necessary permissions: - -. Log into the AWS console and search for Identity and Access Management (IAM). -. From the **IAM** menu, select **Policies** → **Create policy**. -. To provide the necessary permissions, paste the following JSON into the **Specify permissions** menu. - -[source,json] ----- -{ - "Version": "2012-10-17", - "Statement": [ - { - "Sid": "VisualEditor0", - "Effect": "Allow", - "Action": [ - "bedrock:InvokeModel", - "bedrock:InvokeModelWithResponseStream" - ], - "Resource": "*" - } - ] -} ----- - -[NOTE] -==== -These are the minimum required permissions. IAM policies with additional permissions are also supported. -==== - -. Click **Next**. Name your policy. - -The following video demonstrates these steps. - -++++ - -++++ - -[discrete] -[[security-connect-to-bedrock-configure-an-iam-user]] -=== Configure an IAM User - -Next, assign the policy you just created to a new user: - -. Return to the **IAM** menu. Select **Users** from the navigation menu, then click **Create User**. -. Name the user, then click **Next**. -. Select **Attach policies directly**. -. In the **Permissions policies** field, search for the policy you created earlier, select it, and click **Next**. -. Review the configuration then click **Create user**. - -The following video demonstrates these steps. - -++++ - -++++ - -[discrete] -[[security-connect-to-bedrock-create-an-access-key]] -=== Create an access key - -Create the access keys that will authenticate your Elastic connector: - -. Return to the **IAM** menu. Select **Users** from the navigation menu. -. Search for the user you just created, and click its name. -. Go to the **Security credentials** tab. -. Under **Access keys**, click **Create access key**. -. Select **Third-party service**, check the box under **Confirmation**, click **Next**, then click **Create access key**. -. Click **Download .csv file** to download the key. Store it securely. - -The following video demonstrates these steps. - -++++ - -++++ - -[discrete] -[[security-connect-to-bedrock-enable-model-access]] -=== Enable model access - -Make sure the supported Amazon Bedrock LLMs are enabled: - -. Search the AWS console for Amazon Bedrock. -. From the Amazon Bedrock page, click **Get started**. -. Select **Model access** from the left navigation menu, then click **Manage model access**. -. Check the boxes for **Claude** and/or **Claude Instant**, depending which model or models you plan to use. -. Click **Save changes**. - -The following video demonstrates these steps. - -++++ - -++++ - -[discrete] -[[security-connect-to-bedrock-configure-the-amazon-bedrock-connector]] -== Configure the Amazon Bedrock connector - -Finally, configure the connector in {kib}: - -. Log in to {kib}. -. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **Amazon Bedrock**. -. Name your connector. -. (Optional) Configure the Amazon Bedrock connector to use a different AWS region where Anthropic models are supported by editing the **URL** field, for example by changing `us-east-1` to `eu-central-1`. -. (Optional) Add one of the following strings if you want to use a model other than the default: -+ -** For Haiku: `anthropic.claude-3-haiku-20240307-v1:0` -** For Sonnet: `anthropic.claude-3-sonnet-20240229-v1:0` -** For Opus: `anthropic.claude-3-opus-20240229-v1:0` -. Enter the **Access Key** and **Secret** that you generated earlier, then click **Save**. - -Your LLM connector is now configured. For more information on using Elastic AI Assistant, refer to https://docs.elastic.co/security/ai-assistant[AI Assistant]. - -[IMPORTANT] -==== -If you're using https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html[provisioned throughput], your ARN becomes the model ID, and the connector settings **URL** value must be https://www.urlencoder.org/[encoded] to work. For example, if the non-encoded ARN is `arn:aws:bedrock:us-east-2:123456789102:provisioned-model/3Ztr7hbzmkrqy1`, the encoded ARN would be `arn%3Aaws%3Abedrock%3Aus-east-2%3A123456789102%3Aprovisioned-model%2F3Ztr7hbzmkrqy1`. -==== - -The following video demonstrates these steps. - -++++ - -++++ diff --git a/docs/serverless/AI-for-security/connect-to-byo-llm.asciidoc b/docs/serverless/AI-for-security/connect-to-byo-llm.asciidoc deleted file mode 100644 index 6f5d6fbb3d..0000000000 --- a/docs/serverless/AI-for-security/connect-to-byo-llm.asciidoc +++ /dev/null @@ -1,223 +0,0 @@ -[[security-connect-to-byo-llm]] -= Connect to your own local LLM - -// :description: Set up a connector to LM Studio so you can use a local model with AI Assistant. -// :keywords: security, overview, get-started - -This page provides instructions for setting up a connector to a large language model (LLM) of your choice using LM Studio. This allows you to use your chosen model within {elastic-sec}. You'll first need to set up a reverse proxy to communicate with {elastic-sec}, then set up LM Studio on a server, and finally configure the connector in your {elastic-sec} project. https://www.elastic.co/blog/ai-assistant-locally-hosted-models[Learn more about the benefits of using a local LLM]. - -This example uses a single server hosted in GCP to run the following components: - -* LM Studio with the https://mistral.ai/technology/#models[Mixtral-8x7b] model -* A reverse proxy using Nginx to authenticate to Elastic Cloud - -[role="screenshot"] -image::images/lms-studio-arch-diagram.png[Architecture diagram for this guide] - -[NOTE] -==== -For testing, you can use alternatives to Nginx such as https://learn.microsoft.com/en-us/azure/developer/dev-tunnels/overview[Azure Dev Tunnels] or https://ngrok.com/[Ngrok], but using Nginx makes it easy to collect additional telemetry and monitor its status by using Elastic's native Nginx integration. While this example uses cloud infrastructure, it could also be replicated locally without an internet connection. -==== - -[discrete] -[[security-connect-to-byo-llm-configure-your-reverse-proxy]] -== Configure your reverse proxy - -[NOTE] -==== -If your Elastic instance is on the same host as LM Studio, you can skip this step. -==== - -You need to set up a reverse proxy to enable communication between LM Studio and Elastic. For more complete instructions, refer to a guide such as https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-as-a-reverse-proxy-on-ubuntu-22-04[this one]. - -The following is an example Nginx configuration file: - -[source,txt] ----- -server { - listen 80; - listen [::]:80; - server_name ; - server_tokens off; - add_header x-xss-protection "1; mode=block" always; - add_header x-frame-options "SAMEORIGIN" always; - add_header X-Content-Type-Options "nosniff" always; - return 301 https://$server_name$request_uri; -} - -server { - - listen 443 ssl http2; - listen [::]:443 ssl http2; - server_name ; - server_tokens off; - ssl_certificate /etc/letsencrypt/live//fullchain.pem; - ssl_certificate_key /etc/letsencrypt/live//privkey.pem; - ssl_session_timeout 1d; - ssl_session_cache shared:SSL:50m; - ssl_session_tickets on; - ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256'; - ssl_protocols TLSv1.3 TLSv1.2; - ssl_prefer_server_ciphers on; - add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always; - add_header x-xss-protection "1; mode=block" always; - add_header x-frame-options "SAMEORIGIN" always; - add_header X-Content-Type-Options "nosniff" always; - add_header Referrer-Policy "strict-origin-when-cross-origin" always; - ssl_stapling on; - ssl_stapling_verify on; - ssl_trusted_certificate /etc/letsencrypt/live//fullchain.pem; - resolver 1.1.1.1; - location / { - - if ($http_authorization != "Bearer ") { - return 401; -} - - proxy_pass http://localhost:1234/; - } - -} ----- - -[IMPORTANT] -==== -* Replace `` with your actual token, and keep it safe since you'll need it to set up the {elastic-sec} connector. -* Replace `` with your actual domain name. -* Update the `proxy_pass` value at the bottom of the configuration if you decide to change the port number in LM Studio to something other than 1234. -==== - -[discrete] -[[security-connect-to-byo-llm-optional-set-up-performance-monitoring-for-your-reverse-proxy]] -=== (Optional) Set up performance monitoring for your reverse proxy - -You can use Elastic's https://www.elastic.co/docs/current/integrations/nginx[Nginx integration] to monitor performance and populate monitoring dashboards in the {security-app}. - -[discrete] -[[security-connect-to-byo-llm-configure-lm-studio-and-download-a-model]] -== Configure LM Studio and download a model - -First, install https://lmstudio.ai/[LM Studio]. LM Studio supports the OpenAI SDK, which makes it compatible with Elastic's OpenAI connector, allowing you to connect to any model available in the LM Studio marketplace. - -One current limitation of LM Studio is that when it is installed on a server, you must launch the application using its GUI before doing so using the CLI. For example, by using Chrome RDP with an https://cloud.google.com/architecture/chrome-desktop-remote-on-compute-engine[X Window System]. After you've opened the application the first time using the GUI, you can start it by using `sudo lms server start` in the CLI. - -Once you've launched LM Studio: - -. Go to LM Studio's Search window. -. Search for an LLM (for example, `Mixtral-8x7B-instruct`). Your chosen model must include `instruct` in its name in order to work with Elastic. -. Filter your search for "Compatibility Guess" to optimize results for your hardware. Results will be color coded: -+ -** Green means "Full GPU offload possible", which yields the best results. -** Blue means "Partial GPU offload possible", which may work. -** Red for "Likely too large for this machine", which typically will not work. -. Download one or more models. - -[IMPORTANT] -==== -For security reasons, before downloading a model, verify that it is from a trusted source. It can be helpful to review community feedback on the model (for example using a site like Hugging Face). -==== - -[role="screenshot"] -image::images/lms-model-select.png[The LM Studio model selection interface] - -In this example we used https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF[`TheBloke/Mixtral-8x7B-Instruct-v0.1.Q3_K_M.gguf`]. It has 46.7B total parameters, a 32,000 token context window, and uses GGUF https://huggingface.co/docs/transformers/main/en/quantization/overview[quanitization]. For more information about model names and format information, refer to the following table. - -|=== -| Model Name| Parameter Size| Tokens/Context Window| Quantization Format - -| Name of model, sometimes with a version number. -| LLMs are often compared by their number of parameters — higher numbers mean more powerful models. -| Tokens are small chunks of input information. Tokens do not necessarily correspond to characters. You can use https://platform.openai.com/tokenizer[Tokenizer] to see how many tokens a given prompt might contain. -| Quantization reduces overall parameters and helps the model to run faster, but reduces accuracy. - -| Examples: Llama, Mistral, Phi-3, Falcon. -| The number of parameters is a measure of the size and the complexity of the model. The more parameters a model has, the more data it can process, learn from, generate, and predict. -| The context window defines how much information the model can process at once. If the number of input tokens exceeds this limit, input gets truncated. -| Specific formats for quantization vary, most models now support GPU rather than CPU offloading. -|=== - -[discrete] -[[security-connect-to-byo-llm-load-a-model-in-lm-studio]] -== Load a model in LM Studio - -After downloading a model, load it in LM Studio using the GUI or LM Studio's https://lmstudio.ai/blog/lms[CLI tool]. - -[discrete] -[[security-connect-to-byo-llm-option-1-load-a-model-using-the-cli-recommended]] -=== Option 1: load a model using the CLI (Recommended) - -It is a best practice to download models from the marketplace using the GUI, and then load or unload them using the CLI. The GUI allows you to search for models, whereas the CLI only allows you to import specific paths, but the CLI provides a good interface for loading and unloading. - -Use the following commands in your CLI: - -. Verify LM Studio is installed: `lms` -. Check LM Studio's status: `lms status` -. List all downloaded models: `lms ls` -. Load a model: `lms load` - -[role="screenshot"] -image::images/lms-cli-welcome.png[The CLI interface during execution of initial LM Studio commands] - -After the model loads, you should see a `Model loaded successfully` message in the CLI. - -[role="screenshot"] -image::images/lms-studio-model-loaded-msg.png[The CLI message that appears after a model loads] - -To verify which model is loaded, use the `lms ps` command. - -[role="screenshot"] -image::images/lms-ps-command.png[The CLI message that appears after running lms ps] - -If your model uses NVIDIA drivers, you can check the GPU performance with the `sudo nvidia-smi` command. - -[discrete] -[[security-connect-to-byo-llm-option-2-load-a-model-using-the-gui]] -=== Option 2: load a model using the GUI - -Refer to the following video to see how to load a model using LM Studio's GUI. You can change the **port** setting, which is referenced in the Nginx configuration file. Note that the **GPU offload** was set to **Max**. - -++++ - - -++++ - -[discrete] -[[security-connect-to-byo-llm-optional-collect-logs-using-elastics-custom-logs-integration]] -== (Optional) Collect logs using Elastic's Custom Logs integration - -You can monitor the performance of the host running LM Studio using Elastic's https://www.elastic.co/docs/current/integrations/log[Custom Logs integration]. This can also help with troubleshooting. Note that the default path for LM Studio logs is `/tmp/lmstudio-server-log.txt`, as in the following screenshot: - -[role="screenshot"] -image::images/lms-custom-logs-config.png[The configuration window for the custom logs integration] - -[discrete] -[[security-connect-to-byo-llm-configure-the-connector-in-elastic-sec]] -== Configure the connector in {elastic-sec} - -Finally, configure the connector in your Security project: - -. Log in to your Security project. -. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **OpenAI**. The OpenAI connector enables this use case because LM Studio uses the OpenAI SDK. -. Name your connector to help keep track of the model version you are using. -. Under **Select an OpenAI provider**, select **Other (OpenAI Compatible Service)**. -. Under **URL**, enter the domain name specified in your Nginx configuration file, followed by `/v1/chat/completions`. -. Under **Default model**, enter `local-model`. -. Under **API key**, enter the secret token specified in your Nginx configuration file. -. Click **Save**. - -[role="screenshot"] -image::images/lms-edit-connector.png[The Edit connector page in the {security-app}, with appropriate values populated] - -Setup is now complete. You can use the model you've loaded in LM Studio to power Elastic's generative AI features. You can test a variety of models as you interact with AI Assistant to see what works best without having to update your connector. - -[NOTE] -==== -While local models work well for <>, we recommend you use one of <> for interacting with <>. As local models become more performant over time, this is likely to change. -==== diff --git a/docs/serverless/AI-for-security/connect-to-openai.asciidoc b/docs/serverless/AI-for-security/connect-to-openai.asciidoc deleted file mode 100644 index 39804f59fd..0000000000 --- a/docs/serverless/AI-for-security/connect-to-openai.asciidoc +++ /dev/null @@ -1,70 +0,0 @@ -[[security-connect-to-openai]] -= Connect to OpenAI - -// :description: Set up an OpenAI LLM connector. -// :keywords: security, overview, get-started - -This page provides step-by-step instructions for setting up an OpenAI connector for the first time. This connector type enables you to leverage OpenAI's large language models (LLMs) within {kib}. You'll first need to create an OpenAI API key, then configure the connector in {kib}. - -[discrete] -[[security-connect-to-openai-configure-openai]] -== Configure OpenAI - -[discrete] -[[security-connect-to-openai-select-a-model]] -=== Select a model - -Before creating an API key, you must choose a model. Refer to the https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4[OpenAI docs] to select a model. Take note of the specific model name (for example `gpt-4-turbo`); you'll need it when configuring {kib}. - -[NOTE] -==== -`GPT-4o` offers increased performance over previous versions. For more information on how different models perform for different tasks, refer to the <>. -==== - -[discrete] -[[security-connect-to-openai-create-an-api-key]] -=== Create an API key - -To generate an API key: - -. Log in to the OpenAI platform and navigate to **API keys**. -. Select **Create new secret key**. -. Name your key, select an OpenAI project, and set the desired permissions. -. Click **Create secret key** and then copy and securely store the key. It will not be accessible after you leave this screen. - -The following video demonstrates these steps. - -++++ - -++++ - -[discrete] -[[security-connect-to-openai-configure-the-openai-connector]] -== Configure the OpenAI connector - -Finally, configure the connector in {kib}: - -. Log in to {kib}. -. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **OpenAI**. -. Provide a name for your connector, such as `OpenAI (GPT-4 Turbo Preview)`, to help keep track of the model and version you are using. -. Under **Select an OpenAI provider**, choose **OpenAI**. -. The **URL** field can be left as default. -. Under **Default model**, specify which https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4[model] you want to use. -. Paste the API key that you created into the corresponding field. -. Click **Save**. - -The following video demonstrates these steps. - -++++ - -++++ diff --git a/docs/serverless/AI-for-security/connect-to-vertex.asciidoc b/docs/serverless/AI-for-security/connect-to-vertex.asciidoc deleted file mode 100644 index f33ae56329..0000000000 --- a/docs/serverless/AI-for-security/connect-to-vertex.asciidoc +++ /dev/null @@ -1,115 +0,0 @@ -[[security-connect-to-google-vertex]] -= Connect to Google Vertex AI - -// :description: Set up a Google Vertex LLM connector. -// :keywords: security, overview, get-started - -This page provides step-by-step instructions for setting up a Google Vertex AI connector for the first time. This connector type enables you to leverage Vertex AI's large language models (LLMs) within {elastic-sec}. You'll first need to enable Vertex AI, then generate an API key, and finally configure the connector in your {elastic-sec} project. - -[IMPORTANT] -==== -Before continuing, you should have an active project in one of Google Vertex AI's https://cloud.google.com/vertex-ai/docs/general/locations#feature-availability[supported regions]. -==== - -[discrete] -[[security-connect-to-google-vertex-enable-the-vertex-ai-api]] -== Enable the Vertex AI API - -. Log in to the GCP console and navigate to **Vertex AI → Vertex AI Studio → Overview**. -. If you're new to Vertex AI, the **Get started with Vertex AI Studio** popup appears. Click **Vertex AI API**, then click **ENABLE**. - -The following video demonstrates these steps. - -++++ - - -++++ - -[NOTE] -==== -For more information about enabling the Vertex AI API, refer to https://cloud.google.com/vertex-ai/docs/start/cloud-environment[Google's documentation]. -==== - -[discrete] -[[security-connect-to-google-vertex-create-a-vertex-ai-service-account]] -== Create a Vertex AI service account - -. In the GCP console, navigate to **APIs & Services → Library**. -. Search for **Vertex AI API**, select it, and click **MANAGE**. -. In the left menu, navigate to **Credentials** then click **+ CREATE CREDENTIALS** and select **Service account**. -. Name the new service account, then click **CREATE AND CONTINUE**. -. Under **Select a role**, select **Vertex AI User**, then click **CONTINUE**. -. Click **Done**. - -The following video demonstrates these steps. - -++++ - - -++++ - -[discrete] -[[security-connect-to-google-vertex-generate-an-api-key]] -== Generate an API key - -. Return to Vertex AI's **Credentials** menu and click **Manage service accounts**. -. Search for the service account you just created, select it, then click the link that appears under **Email**. -. Go to the **KEYS** tab, click **ADD KEY**, then select **Create new key**. -. Select **JSON**, then click **CREATE** to download the key. Keep it somewhere secure. - -The following video demonstrates these steps. - -++++ - - -++++ - -[discrete] -[[security-connect-to-google-vertex-configure-the-google-gemini-connector]] -== Configure the Google Gemini connector - -Finally, configure the connector in {kib}: - -. Log in to {kib}. -. Find **Connectors** in the navigation menu or use the global search field. Then click **Create Connector**, and select **Google Gemini**. -. Name your connector to help keep track of the model version you are using. -. Under **URL**, enter the URL for your region. -. Enter your **GCP Region** and **GCP Project ID**. -. Under **Default model**, specify either `gemini-1.5.pro` or `gemini-1.5-flash`. https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models[Learn more about the models]. -. Under **Authentication**, enter your API key. -. Click **Save**. - -The following video demonstrates these steps. - -++++ - - -++++ diff --git a/docs/serverless/cloud-native-security/benchmark-rules.asciidoc b/docs/serverless/cloud-native-security/benchmark-rules.asciidoc deleted file mode 100644 index d9fcc2eb7b..0000000000 --- a/docs/serverless/cloud-native-security/benchmark-rules.asciidoc +++ /dev/null @@ -1,61 +0,0 @@ -[[security-benchmark-rules]] -= Benchmarks - -// :description: Review the cloud security benchmark rules used by the CSPM and KSPM integrations. -// :keywords: serverless, security, overview, cloud security - -:append: - -// tag::content[] - -preview:[] - -The Benchmarks page lets you view the cloud security posture (CSP) benchmarks for the <> (CSPM) and <> (KSPM) integrations. - -[role="screenshot"] -image::images/benchmark-rules/-cloud-native-security-benchmark-rules.png[Benchmark rules page] - -[discrete] -[id="security-benchmark-rules-what-are-benchmarks{append}"] -== What are benchmarks? - -Each benchmark contains benchmark rules which are used by the CSPM and KSPM integrations to identify configuration risks in your cloud infrastructure. There are different benchmarks for different cloud services, such as AWS, GCP, or Azure. They are based on the Center for Internet Security's (CIS) https://www.cisecurity.org/cis-benchmarks/[secure configuration benchmarks]. - -Each benchmark rule checks to see if a specific type of resource is configured according to a CIS Benchmark. The names of rules describe what they check, for example: - -* `Ensure Kubernetes Secrets are encrypted using Customer Master Keys (CMKs) managed in AWS KMS` -* `Ensure the default namespace is not in use` -* `Ensure IAM policies that allow full "*:*" administrative privileges are not attached` -* `Ensure the default namespace is not in use` - -When benchmark rules are evaluated, the resulting <> data appears on the <>. - -[NOTE] -==== -Benchmark rules are not editable. -==== - -[discrete] -[id="security-benchmark-rules-review-your-benchmarks{append}"] -== Review your benchmarks - -Find **Benchmarks** in the navigation menu or use the global search field. From there, you can click a benchmark's name to view the benchmark rules associated with it. You can click a benchmark rule's name to see details including information about how to remediate it, and related links. - -Benchmark rules are enabled by default, but you can disable some of them — at the benchmark level — to suit your environment. This means for example that if you have two CSPM integrations using the `CIS AWS` benchmark, disabling a rule for that benchmark affects both integrations. To enable or disable a rule, use the **Enabled** toggle on the right of the rules table. - -[NOTE] -==== -Disabling a benchmark rule automatically disables any associated detection rules and alerts. Re-enabling a benchmark rule **does not** automatically re-enable them. -==== - -[discrete] -[id="security-benchmark-rules-how-benchmark-rules-work{append}"] -== How benchmark rules work - -. When a security posture management integration is deployed, and every four hours after that, {agent} fetches relevant cloud resources. -. After resources are fetched, they are evaluated against all applicable enabled benchmark rules. -. Finding values of `pass` or `fail` indicate whether the standards defined by benchmark rules were met. - -// end::content[] - -:append!: diff --git a/docs/serverless/cloud-native-security/cspm-get-started-azure.asciidoc b/docs/serverless/cloud-native-security/cspm-get-started-azure.asciidoc deleted file mode 100644 index 01c42f26e1..0000000000 --- a/docs/serverless/cloud-native-security/cspm-get-started-azure.asciidoc +++ /dev/null @@ -1,198 +0,0 @@ -[[security-cspm-get-started-azure]] -= Get started with CSPM for Azure - -// :description: Start monitoring the security posture of your Azure cloud assets. -// :keywords: serverless, security, overview, cloud security - -preview:[] - -[discrete] -[[cspm-overview-azure]] -== Overview - -This page explains how to get started monitoring the security posture of your cloud assets using the Cloud Security Posture Management (CSPM) feature. - -.Requirements -[NOTE] -==== -* CSPM only works in the `Default` {kib} space. Installing the CSPM integration on a different {kib} space will not work. -* CSPM is supported only on AWS, GCP, and Azure commercial cloud platforms, and AWS GovCloud. Other government cloud platforms are not supported (https://github.com/elastic/kibana/issues/new/choose[request support]). -* To view posture data, you need `read` privileges for the following {es} indices: -+ -** `logs-cloud_security_posture.findings_latest-*` -** `logs-cloud_security_posture.scores-*` -** `logs-cloud_security_posture.findings` -* The user who gives the CSPM integration permissions in Azure must be an Azure subscription `admin`. -==== - -[discrete] -[[cspm-setup-azure]] -== Set up CSPM for Azure - -You can set up CSPM for Azure by by enrolling an Azure organization (management group) containing multiple subscriptions, or by enrolling a single subscription. Either way, first add the CSPM integration, then enable cloud account access. Two deployment technologies are available: agentless, and agent-based. <> allows you to collect cloud posture data without having to manage the deployment of an agent in your cloud. <> requires you to deploy and manage an agent in the cloud account you want to monitor. - -[discrete] -[[cspm-azure-agentless]] -== Agentless deployment - -beta:[] - -. Find **Integrations** in the navigation menu or use the global search field. -. Search for `CSPM`, then click on the result. -. Click **Add Cloud Security Posture Management (CSPM)**. -. Select **Azure**, then either **Azure Organization** to onboard your whole organization, or **Single Subscription** to onboard an individual subscription. -. Give your integration a name that matches the purpose or team of the Azure subscription/organization you want to monitor, for example, `dev-azure-account`. -. Click **Advanced options**, then select **Agentless (BETA)**. -. Next, you'll need to authenticate to Azure by providing a **Client ID**, **Tenant ID**, and **Client Secret**. To learn how to generate them, refer to <>. -. Once you've provided the necessary credentials, click **Save and continue** to finish deployment. Your data should start to appear within a few minutes. - -[discrete] -[[cspm-azure-agent-based]] -== Agent-based deployment - -[discrete] -[[cspm-add-and-name-integration-azure]] -=== Add your CSPM integration - -. Find **Integrations** in the navigation menu or use the global search field. -. Search for `CSPM`, then click on the result. -. Click **Add Cloud Security Posture Management (CSPM)**. -. Under **Configure integration**, select **Azure**, then select either **Azure Organization** or **Single Subscription**, depending on which resources you want to monitor. -. Give your integration a name that matches the purpose or team of the Azure resources you want to monitor, for example, `azure-CSPM-dev-1`. - -[discrete] -[[cspm-set-up-cloud-access-section-azure]] -=== Set up cloud account access - -To set up CSPM for an Azure organization or subscription, you will need admin privileges for that organization or subscription. - -For most users, the simplest option is to use an Azure Resource Manager (ARM) template to automatically provision the necessary resources and permissions in Azure. If you prefer a more hands-on approach or require a specific configuration not supported by the ARM template, you can use one of the manual setup options described below. - -[discrete] -[[cspm-set-up-ARM]] -=== ARM template setup (recommended) - -. Under **Setup Access**, select **ARM Template**. -. Under **Where to add this integration**: -+ -.. Select **New Hosts**. -.. Name the {agent} policy. Use a name that matches the resources you want to monitor, for example, `azure-dev-policy`. Click **Save and continue**. The **ARM Template deployment** window appears. -.. In a new tab, log in to the Azure portal, then return to {kib} and click **Launch ARM Template**. This will open the ARM template in Azure. -.. If you are deploying to an Azure organization, select the management group you want to monitor from the drop-down menu. Next, enter the subscription ID of the subscription where you want to deploy the VM that will scan your resources. -.. Copy the `Fleet URL` and `Enrollment Token` that appear in {kib} to the corresponding fields in the ARM Template, then click **Review + create**. -.. (Optional) Change the `Resource Group Name` parameter. Otherwise, the name of the resource group defaults to a timestamp prefixed with `cloudbeat-`. -. Return to {kib} and wait for the confirmation of data received from your new integration. Then you can click **View Assets** to see your data. - -[discrete] -[[cspm-set-up-manual-azure]] -=== Manual setup - -For manual setup, multiple authentication methods are available: - -. Managed identity (recommended) -. Service principal with client secret -. Service principal with client certificate - -[discrete] -[[cspm-azure-managed-identity-setup]] -=== Option 1: Managed identity (recommended) - -This method involves creating an Azure VM (or using an existing one), giving it read access to the resources you want to monitor with CSPM, and installing {agent} on it. - -. Go to the Azure portal to create a new Azure VM. -. Follow the setup process, and make sure you enable **System assigned managed identity** under the **Management** tab. -. Go to your Azure subscription list and select the subscription or management group you want to monitor with CSPM. -. Go to **Access control (IAM)**, and select **Add Role Assignment**. -. Select the `Reader` role, assign access to **Managed Identity**, then select your VM. - -After assigning the role: - -. Return to the **Add CSPM** page in {kib}. -. Under **Configure integration**, select **Azure**. Under **Setup access**, select **Manual**. -. Under **Where to add this integration**, select **New hosts**. -. Click **Save and continue**, then follow the instructions to install {agent} on your Azure VM. - -Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data. - -[discrete] -[[cspm-azure-client-secret]] -=== Option 2: Service principal with client secret - -Before using this method, you must have set up a https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#get-tenant-and-app-id-values-for-signing-in[Microsoft Entra application and service principal that can access resources]. - -. On the **Add Cloud Security Posture Management (CSPM) integration** page, scroll to the **Setup access** section, then select **Manual**. -. Under **Preferred manual method**, select **Service principal with Client Secret**. -. Go to the **Registered apps** section of https://ms.portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/RegisteredApps[Microsoft Entra ID]. -. Click on **New Registration**, name your app and click **Register**. -. Copy your new app's `Directory (tenant) ID` and `Application (client) ID`. Paste them into the corresponding fields in {kib}. -. Return to the Azure portal. Select **Certificates & secrets**, then go to the **Client secrets** tab. Click **New client secret**. -. Copy the new secret. Paste it into the corresponding field in {kib}. -. Return to Azure. Go to your Azure subscription list and select the subscription or management group you want to monitor with CSPM. -. Go to **Access control (IAM)** and select **Add Role Assignment**. -. Select the `Reader` function role, assign access to **User, group, or service principal**, and select your new app. -. Return to the **Add CSPM** page in {kib}. -. Under **Where to add this integration**, select **New hosts**. -. Click **Save and continue**, then follow the instructions to install {agent} on your selected host. - -Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data. - -[discrete] -[[cspm-azure-client-certificate]] -=== Option 3: Service principal with client certificate - -Before using this method, you must have set up a https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#get-tenant-and-app-id-values-for-signing-in[Microsoft Entra application and service principal that can access resources]. - -. On the **Add Cloud Security Posture Management (CSPM) integration** page, under **Setup access**, select **Manual**. -. Under **Preferred manual method**, select **Service principal with client certificate**. -. Go to the **Registered apps** section of https://ms.portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/RegisteredApps[Microsoft Entra ID]. -. Click on **New Registration**, name your app and click **Register**. -. Copy your new app's `Directory (tenant) ID` and `Application (client) ID`. Paste them into the corresponding fields in {kib}. -. Return to Azure. Go to your Azure subscription list and select the subscription or management group you want to monitor with CSPM. -. Go to **Access control (IAM)** and select **Add Role Assignment**. -. Select the `Reader` function role, assign access to **User, group, or service principal**, and select your new app. - -Next, create a certificate. If you intend to use a password-protected certificate, you must use a pkcs12 certificate. Otherwise, you must use a pem certificate. - -Create a pkcs12 certificate, for example: - -[source,shell] ----- -# Create PEM file -openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes - -# Create pkcs12 bundle using legacy flag (CLI will ask for export password) -openssl pkcs12 -legacy -export -out bundle.p12 -inkey key.pem -in cert.pem ----- - -Create a PEM certificate, for example: - -[source,shell] ----- -# Generate certificate signing request (csr) and key -openssl req -new -newkey rsa:4096 -nodes -keyout cert.key -out cert.csr - -# Generate PEM and self-sign with key -openssl x509 -req -sha256 -days 365 -in cert.csr -signkey cert.key -out signed.pem - -# Create bundle -cat cert.key > bundle.pem -cat signed.pem >> bundle.pem ----- - -. Return to Azure. -. Navigate to the **Certificates & secrets** menu. Select the **Certificates** tab. -. Click **Upload certificate**. -+ -.. If you're using a PEM certificate that was created using the example commands above, upload `signed.pem`. -.. If you're using a pkcs12 certificate that was created using the example commands above, upload `cert.pem`. -. Upload the certificate bundle to the VM where you will deploy {agent}. -+ -.. If you're using a PEM certificate that was created using the example commands above, upload `bundle.pem`. -.. If you're using a pkcs12 certificate that was created using the example commands above, upload `bundle.p12`. -. Return to the **Add CSPM** page in {kib}. -. For **Client Certificate Path**, enter the full path to the certificate that you uploaded to the host where you will install {agent}. -. If you used a pkcs12 certificate, enter its password under **Client Certificate Password**. -. Under **Where to add this integration**, select **New hosts**. -. Click **Save and continue**, then follow the instructions to install {agent} on your selected host. - -Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data. diff --git a/docs/serverless/cloud-native-security/cspm-get-started-gcp.asciidoc b/docs/serverless/cloud-native-security/cspm-get-started-gcp.asciidoc deleted file mode 100644 index 4eea50b7bf..0000000000 --- a/docs/serverless/cloud-native-security/cspm-get-started-gcp.asciidoc +++ /dev/null @@ -1,205 +0,0 @@ -[[security-cspm-get-started-gcp]] -= Get started with CSPM for GCP - -// :description: Start monitoring the security posture of your GCP cloud assets. -// :keywords: serverless, security, overview, cloud security - -preview:[] - -[discrete] -[[cspm-overview-gcp]] -== Overview - -This page explains how to get started monitoring the security posture of your cloud assets using the Cloud Security Posture Management (CSPM) feature. - -.Requirements -[NOTE] -==== -* CSPM only works in the `Default` {kib} space. Installing the CSPM integration on a different {kib} space will not work. -* CSPM is supported only on AWS, GCP, and Azure commercial cloud platforms, and AWS GovCloud. Other government cloud platforms are not supported (https://github.com/elastic/kibana/issues/new/choose[request support]). -* To view posture data, you need the appropriate user role to read the following {es} indices: -+ -** `logs-cloud_security_posture.findings_latest-*` -** `logs-cloud_security_posture.scores-*` -** `Logs-cloud_security_posture.findings` -* The user who gives the CSPM integration GCP permissions must be a GCP project `admin`. -==== - -[discrete] -[[cspm-setup-gcp]] -== Set up CSPM for GCP - -You can set up CSPM for GCP either by enrolling a single project, or by enrolling an organization containing multiple projects. Either way, you need to first add the CSPM integration, then enable cloud account access. Two deployment technologies are available: agentless, and agent-based. <> allows you to collect cloud posture data without having to manage the deployment of an agent in your cloud. <> requires you to deploy and manage an agent in the cloud account you want to monitor. - -[discrete] -[[cspm-gcp-agentless]] -== Agentless deployment - -beta:[] - -. Find **Integrations** in the navigation menu or use the global search field. -. Search for `CSPM`, then click on the result. -. Click **Add Cloud Security Posture Management (CSPM)**. -. Select **GCP**, then either **GCP Organization** to onboard your whole organization, or **Single Account** to onboard an individual account. -. Give your integration a name that matches the purpose or team of the GCP subscription/organization you want to monitor, for example, `dev-gcp-account`. -. Click **Advanced options**, then select **Agentless (BETA)**. -. Next, you'll need to authenticate to GCP. Expand the **Steps to Generate GCP Account Credentials** section, then follow the instructions that appear to automatically create the necessary credentials using Google Cloud Shell. -. Once you've provided the necessary credentials, click **Save and continue** to finish deployment. Your data should start to appear within a few minutes. - -[discrete] -[[cspm-gcp-agent-based]] -== Agent-based deployment - -[discrete] -[[cspm-add-and-name-integration-gcp]] -=== Add your CSPM integration - -. Find **Integrations** in the navigation menu or use the global search field. -. Search for `CSPM`, then click on the result. -. Click **Add Cloud Security Posture Management (CSPM)**. -. Under **Configure integration**, select **GCP**, then either **GCP Organization** (recommended) or **Single Account**. -. Give your integration a name that matches the purpose or team of the GCP account you want to monitor, for example, `dev-gcp-project`. - -[discrete] -[[cspm-set-up-cloud-access-section-gcp]] -=== Set up cloud account access - -To set up CSPM for a GCP project, you need admin privileges for the project. - -For most users, the simplest option is to use a Google Cloud Shell script to automatically provision the necessary resources and permissions in your GCP account. This method, as well as two manual options, are described below. - -[discrete] -[[cspm-set-up-cloudshell]] -== Cloud Shell script setup (recommended) - -. Under **Setup Access**, select **Google Cloud Shell**. Enter your GCP Project ID, and for GCP Organization deployments, your GCP Organization ID. -. Under **Where to add this integration**: -+ -.. Select **New Hosts**. -.. Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`. -.. Click **Save and continue**, then **Add {agent} to your hosts**. The **Add agent** wizard appears and provides {agent} binaries, which you can download and deploy to a VM in your GCP account. -. Click **Save and continue**. -. Copy the command that appears, then click **Launch Google Cloud Shell**. It opens in a new window. -. Check the box to trust Elastic's `cloudbeat` repo, then click **Confirm** -+ -[role="screenshot"] -image::images/cspm-get-started-gcp/-cloud-native-security-cspm-cloudshell-trust.png[The cloud shell confirmation popup] -. In Google Cloud Shell, execute the command you copied. Once it finishes, return to {kib} and wait for the confirmation of data received from your new integration. Then you can click **View Assets** to see your data. - -[NOTE] -==== -During Cloud Shell setup, the CSPM integration adds roles to Google's default service account, which enables custom role creation and attachment of the service account to a compute instance. -After setup, these roles are removed from the service account. If you attempt to delete the deployment but find the deployment manager lacks necessary permissions, consider adding the missing roles to the service account: -https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectIamAdmin[Project IAM Admin], https://cloud.google.com/iam/docs/understanding-roles#iam.roleAdmin[Role Administrator]. -==== - -[discrete] -[[cspm-manual-auth-org]] -== Manual authentication (GCP organization) - -To authenticate manually to monitor a GCP organization, you'll need to create a new GCP service account, assign it the necessary roles, generate credentials, then provide those credentials to the CSPM integration. - -Use the following commands, after replacing `` with the name of your new service account, `` with your GCP organization's ID, and `` with the GCP project ID of the project where you want to provision the compute instance that will run CSPM. - -Create a new service account: - -[source,shell] ----- -gcloud iam service-accounts create \ - --description="Elastic agent service account for CSPM" \ - --display-name="Elastic agent service account for CSPM" \ - --project= ----- - -Assign the necessary roles to the service account: - -[source,shell] ----- -gcloud organizations add-iam-policy-binding \ - --member=serviceAccount:@.iam.gserviceaccount.com \ - --role=roles/cloudasset.viewer - -gcloud organizations add-iam-policy-binding \ - --member=serviceAccount:@.iam.gserviceaccount.com \ - --role=roles/browser ----- - -The `Cloud Asset Viewer` role grants read access to cloud asset metadata. The `Browser` role grants read access to the project hierarchy. - -Download the credentials JSON (first, replace `` with the location where you want to save it): - -[source,shell] ----- -gcloud iam service-accounts keys create \ - --iam-account=@.iam.gserviceaccount.com ----- - -Keep the credentials JSON in a secure location; you will need it later. - -Provide credentials to the CSPM integration: - -. On the CSPM setup screen under **Setup Access**, select **Manual**. -. Enter your GCP **Organization ID**. Enter the GCP **Project ID** of the project where you want to provision the compute instance that will run CSPM. -. Select **Credentials JSON**, and enter the value you generated earlier. -. Under **Where to add this integration**, select **New Hosts**. -. Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`. -. Click **Save and continue**, then follow the instructions to install {agent} in your chosen GCP project. - -Wait for the confirmation that {kib} received data from your new integration. Then you can click **View Assets** to see your data. - -[discrete] -[[cspm-manual-auth-proj]] -== Manual authentication (GCP project) - -To authenticate manually to monitor an individual GCP project, you'll need to create a new GCP service account, assign it the necessary roles, generate credentials, then provide those credentials to the CSPM integration. - -Use the following commands, after replacing `` with the name of your new service account, and `` with your GCP project ID. - -Create a new service account: - -[source,shell] ----- -gcloud iam service-accounts create \ - --description="Elastic agent service account for CSPM" \ - --display-name="Elastic agent service account for CSPM" \ - --project= ----- - -Assign the necessary roles to the service account: - -[source,shell] ----- -gcloud projects add-iam-policy-binding \ - --member=serviceAccount:@.iam.gserviceaccount.com \ - --role=roles/cloudasset.viewer - -gcloud projects add-iam-policy-binding \ - --member=serviceAccount:@.iam.gserviceaccount.com \ - --role=roles/browser ----- - -[NOTE] -==== -The `Cloud Asset Viewer` role grants read access to cloud asset metadata. The `Browser` role grants read access to the project hierarchy. -==== - -Download the credentials JSON (first, replace `` with the location where you want to save it): - -[source,shell] ----- -gcloud iam service-accounts keys create \ - --iam-account=@.iam.gserviceaccount.com ----- - -Keep the credentials JSON in a secure location; you will need it later. - -Provide credentials to the CSPM integration: - -. On the CSPM setup screen under **Setup Access**, select **Manual**. -. Enter your GCP **Project ID**. -. Select **Credentials JSON**, and enter the value you generated earlier. -. Under **Where to add this integration**, select **New Hosts**. -. Name the policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-gcp-account`. -. Click **Save and continue**, then follow the instructions to install the agent in your chosen GCP project. - -Wait for the confirmation that Kibana received data from your new integration. Then you can click **View Assets** to see your data. diff --git a/docs/serverless/cloud-native-security/cspm-get-started.asciidoc b/docs/serverless/cloud-native-security/cspm-get-started.asciidoc deleted file mode 100644 index 5883b97b99..0000000000 --- a/docs/serverless/cloud-native-security/cspm-get-started.asciidoc +++ /dev/null @@ -1,349 +0,0 @@ -[[security-cspm-get-started]] -= Get started with CSPM for AWS - -// :description: Start monitoring the security posture of your AWS cloud assets. -// :keywords: serverless, security, overview, cloud security - -preview:[] - -[discrete] -[[cspm-overview]] -== Overview - -This page explains how to get started monitoring the security posture of your cloud assets using the Cloud Security Posture Management (CSPM) feature. - -.Requirements -[NOTE] -==== -* CSPM only works in the `Default` {kib} space. Installing the CSPM integration on a different {kib} space will not work. -* CSPM is supported only on AWS, GCP, and Azure commercial cloud platforms, and AWS GovCloud. Other government cloud platforms are not supported (https://github.com/elastic/kibana/issues/new/choose[request support]). -* To view posture data, you need the appropriate user role to read the following {es} indices: -+ -** `logs-cloud_security_posture.findings_latest-*` -** `logs-cloud_security_posture.scores-*` -** `Logs-cloud_security_posture.findings` -* The user who gives the CSPM integration AWS permissions must be an AWS account `admin`. -==== - -[discrete] -[[cspm-setup]] -== Set up CSPM for AWS - -You can set up CSPM for AWS either by enrolling a single cloud account, or by enrolling an organization containing multiple accounts. Either way, first you will add the CSPM integration, then enable cloud account access. Two deployment technologies are available: agentless, and agent-based. <> allows you to collect cloud posture data without having to manage the deployment of an {agent} in your cloud. <> requires you to deploy and manage an {agent} in the cloud account you want to monitor. - -[discrete] -[[cspm-aws-agentless]] -== Agentless deployment - -beta:[] - -. Find **Integrations** in the navigation menu or use the global search field. -. Search for `CSPM`, then click on the result. -. Click *Add Cloud Security Posture Management (CSPM)*. -. Select *AWS*, then either *AWS Organization* to onboard multiple accounts, or *Single Account* to onboard an individual account. -. Give your integration a name that matches the purpose or team of the AWS account/organization you want to monitor, for example, `dev-aws-account`. -. Click **Advanced options**, then select **Agentless (BETA)**. -. Next, you'll need to authenticate to AWS. Two methods are available: -.. Option 1: Direct access keys/CloudFormation (Recommended). Under **Preferred method** select **Direct access keys**. Expand the **Steps to Generate AWS Account Credentials** section, then follow the displayed instructions to automatically create the necessary credentials using CloudFormation. -.. Option 2: Temporary keys. To authenticate using temporary keys, refer to the instructions for <>. -. Once you've selected an authentication method and provided all necessary credentials, click **Save and continue** to finish deployment. Your data should start to appear within a few minutes. - -[discrete] -[[cspm-aws-agent-based]] -== Agent-based deployment - -[discrete] -[[cspm-add-and-name-integration]] -=== Add the CSPM integration - -. Find **Integrations** in the navigation menu or use the global search field. -. Search for `CSPM`, then click on the result. -. Click **Add Cloud Security Posture Management (CSPM)**. -. Select **AWS**, then either **AWS Organization** to onboard multiple accounts, or **Single Account** to onboard an individual account. -. Give your integration a name that matches the purpose or team of the AWS account/organization you want to monitor, for example, `dev-aws-account`. - -[discrete] -[[cspm-set-up-cloud-access-section]] -=== Set up cloud account access - -The CSPM integration requires access to AWS's built-in https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_job-functions.html#jf_security-auditor[`SecurityAudit` IAM policy] in order to discover and evaluate resources in your cloud account. There are several ways to provide access. - -For most use cases, the simplest option is to use AWS CloudFormation to automatically provision the necessary resources and permissions in your AWS account. This method, as well as several manual options, are described below. - -[discrete] -[[cspm-set-up-cloudformation]] -=== CloudFormation (recommended) - -. In the **Add Cloud Security Posture Management (CSPM) integration** menu, under **Setup Access**, select **CloudFormation**. -. In a new browser tab or window, log in as an admin to the AWS account or organization you want to onboard. -. Return to your {kib} tab. Click **Save and continue** at the bottom of the page. -. Review the information, then click **Launch CloudFormation**. -. A CloudFormation template appears in a new browser tab. -. For organization-level deployments only, you must enter the ID of the organizational unit where you want to deploy into the `OrganizationalUnitIds` field in the CloudFormation template. You can find it in the AWS console under **AWS Organizations → AWS Accounts** (it appears under the organization name). -. (Optional) Switch to the AWS region where you want to deploy using the controls in the upper right corner. -. Tick the checkbox under **Capabilities** to authorize the creation of necessary resources. -+ -[role="screenshot"] -image::images/cspm-get-started/-cloud-native-security-cspm-cloudformation-template.png[The Add permissions screen in AWS] -. At the bottom of the template, select **Create stack**. - -When you return to {kib}, click **View assets** to review the data being collected by your new integration. - -[discrete] -[[cspm-setup-organization-manual]] -=== Manual authentication for organization-level onboarding - -[NOTE] -==== -If you're onboarding a single account instead of an organization, skip this section. -==== - -When using manual authentication to onboard at the organization level, you need to configure the necessary permissions using the AWS console for the organization where you want to deploy: - -* In the organization's management account (root account), create an IAM role called `cloudbeat-root` (the name is important). The role needs several policies: -+ -** The following inline policy: -+ -.Click to expand policy -[%collapsible] -===== -[source,json] ----- -{ - "Version": "2012-10-17", - "Statement": [ - { - "Action": [ - "organizations:List*", - "organizations:Describe*" - ], - "Resource": "*", - "Effect": "Allow" - }, - { - "Action": [ - "sts:AssumeRole" - ], - "Resource": "*", - "Effect": "Allow" - } - ] -} ----- -===== -+ -** The following trust policy: -+ -.Click to expand policy -[%collapsible] -===== -[source,json] ----- -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Principal": { - "AWS": "arn:aws:iam:::root" - }, - "Action": "sts:AssumeRole" - }, - { - "Effect": "Allow", - "Principal": { - "Service": "ec2.amazonaws.com" - }, - "Action": "sts:AssumeRole" - } - ] -} ----- -===== -+ -** The AWS-managed `SecurityAudit` policy. - -[IMPORTANT] -==== -You must replace `` in the trust policy with your AWS account ID. -==== - -* Next, for each account you want to scan in the organization, create an IAM role named `cloudbeat-securityaudit` with the following policies: -+ -** The AWS-managed `SecurityAudit` policy. -** The following trust policy: -+ -.Click to expand policy -[%collapsible] -===== -[source,json] ----- -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Principal": { - "AWS": "arn:aws:iam:::role/cloudbeat-root" - }, - "Action": "sts:AssumeRole" - } - ] -} ----- -===== - -[IMPORTANT] -==== -You must replace `` in the trust policy with your AWS account ID. -==== - -After creating the necessary roles, authenticate using one of the manual authentication methods. - -[IMPORTANT] -==== -When deploying to an organization using any of the authentication methods below, you need to make sure that the credentials you provide grant permission to assume `cloudbeat-root` privileges. -==== - -[discrete] -[[cspm-set-up-manual]] -=== Manual authentication methods - -* <> -* <> -* <> -* <> -* <> - -[IMPORTANT] -==== -Whichever method you use to authenticate, make sure AWS’s built-in https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_job-functions.html#jf_security-auditor[`SecurityAudit` IAM policy] is attached. -==== - -[discrete] -[[cspm-use-instance-role]] -==== Option 1 - Default instance role - -[NOTE] -==== -If you are deploying to an AWS organization instead of an AWS account, you should already have <>, `cloudbeat-root`. Skip to step 2 "Attach your new IAM role to an EC2 instance", and attach this role. You can use either an existing or new EC2 instance. -==== - -Follow AWS's https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html[IAM roles for Amazon EC2] documentation to create an IAM role using the IAM console, which automatically generates an instance profile. - -. Create an IAM role: -+ -.. In AWS, go to your IAM dashboard. Click **Roles**, then **Create role**. -.. On the **Select trusted entity** page, under **Trusted entity type**, select **AWS service**. -.. Under **Use case**, select **EC2**. Click **Next**. -+ -[role="screenshot"] -image::images/cspm-get-started/-cloud-native-security-cspm-aws-auth-1.png[The Select trusted entity screen in AWS] -.. On the **Add permissions** page, search for and select `SecurityAudit`. Click **Next**. -+ -[role="screenshot"] -image::images/cspm-get-started/-cloud-native-security-cspm-aws-auth-2.png[The Add permissions screen in AWS] -.. On the **Name, review, and create** page, name your role, then click **Create role**. -. Attach your new IAM role to an EC2 instance: -+ -.. In AWS, select an EC2 instance. -.. Select **Actions → Security → Modify IAM role**. -+ -[role="screenshot"] -image::images/cspm-get-started/-cloud-native-security-cspm-aws-auth-3.png[The EC2 page in AWS, showing the Modify IAM role option] -.. On the **Modify IAM role** page, search for and select your new IAM role. -.. Click **Update IAM role**. -.. Return to {kib} and <>. - -[IMPORTANT] -==== -Make sure to deploy the CSPM integration to this EC2 instance. When completing setup in {kib}, in the **Setup Access** section, select **Assume role**. Leave **Role ARN** empty for agentless deployments. For agent-based deployments, leave it empty unless you want to specify a role the {agent} should assume instead of the default role for your EC2 instance. Click **Save and continue**. -==== - -[discrete] -[[cspm-use-keys-directly]] -==== Option 2 - Direct access keys - -Access keys are long-term credentials for an IAM user or AWS account root user. To use access keys as credentials, you must provide the `Access key ID` and the `Secret Access Key`. After you provide credentials, <>. - -For more details, refer to https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html[Access Keys and Secret Access Keys]. - -[IMPORTANT] -==== -You must select **Programmatic access** when creating the IAM user. -==== - -[discrete] -[[cspm-use-temp-credentials]] -==== Option 3 - Temporary security credentials - -You can configure temporary security credentials in AWS to last for a specified duration. They consist of an access key ID, a secret access key, and a session token, which is typically found using `GetSessionToken`. - -Because temporary security credentials are short term, once they expire, you will need to generate new ones and manually update the integration's configuration to continue collecting cloud posture data. Update the credentials before they expire to avoid data loss. - -[NOTE] -==== -IAM users with multi-factor authentication (MFA) enabled need to submit an MFA code when calling `GetSessionToken`. For more details, refer to AWS's https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html[Temporary Security Credentials] documentation. -==== - -You can use the AWS CLI to generate temporary credentials. For example, you could use the following command if you have MFA enabled: - -[source,console] ----- -sts get-session-token --serial-number arn:aws:iam::1234:mfa/your-email@example.com --duration-seconds 129600 --token-code 123456 ----- - -The output from this command includes the following fields, which you should provide when configuring the KSPM integration: - -* `Access key ID`: The first part of the access key. -* `Secret Access Key`: The second part of the access key. -* `Session Token`: The required token when using temporary security credentials. - -After you provide credentials, <>. - -[discrete] -[[cspm-use-a-shared-credentials-file]] -==== Option 4 - Shared credentials file - -If you use different AWS credentials for different tools or applications, you can use profiles to define multiple access keys in the same configuration file. For more details, refer to AWS' https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html[Shared Credentials Files] documentation. - -Instead of providing the `Access key ID` and `Secret Access Key` to the integration, provide the information required to locate the access keys within the shared credentials file: - -* `Credential Profile Name`: The profile name in the shared credentials file. -* `Shared Credential File`: The directory of the shared credentials file. - -If you don't provide values for all configuration fields, the integration will use these defaults: - -* If `Access key ID`, `Secret Access Key`, and `ARN Role` are not provided, then the integration will check for `Credential Profile Name`. -* If there is no `Credential Profile Name`, the default profile will be used. -* If `Shared Credential File` is empty, the default directory will be used. -+ -** For Linux or Unix, the shared credentials file is located at `~/.aws/credentials`. - -After providing credentials, <>. - -[discrete] -[[cspm-use-iam-arn]] -==== Option 5 - IAM role Amazon Resource Name (ARN) - -An IAM role Amazon Resource Name (ARN) is an IAM identity that you can create in your AWS account. You define the role's permissions. Roles do not have standard long-term credentials such as passwords or access keys. Instead, when you assume a role, it provides temporary security credentials for your session. - -To use an IAM role ARN, select **Assume role** under **Preferred manual method**, enter the ARN, and continue to Finish manual setup. - -[discrete] -[[cspm-finish-manual]] -=== Finish manual setup - -Once you’ve provided AWS credentials, under **Where to add this integration**: - -If you want to monitor an AWS account or organization where you have not yet deployed {agent}: - -* Select **New Hosts**. -* Name the {agent} policy. Use a name that matches the purpose or team of the cloud account or accounts you want to monitor. For example, `dev-aws-account`. -* Click **Save and continue**, then **Add {agent} to your hosts**. The **Add agent** wizard appears and provides {agent} binaries, which you can download and deploy to your AWS account. - -If you want to monitor an AWS account or organization where you have already deployed {agent}: - -* Select **Existing hosts**. -* Select an agent policy that applies the AWS account you want to monitor. -* Click **Save and continue**. diff --git a/docs/serverless/cloud-native-security/d4c-get-started.asciidoc b/docs/serverless/cloud-native-security/d4c-get-started.asciidoc deleted file mode 100644 index e23b27fc8d..0000000000 --- a/docs/serverless/cloud-native-security/d4c-get-started.asciidoc +++ /dev/null @@ -1,92 +0,0 @@ -[[security-d4c-get-started]] -= Get started with CWP - -// :description: Secure your containerized workloads and start detecting threats and vulnerabilities. -// :keywords: security, how-to, get-started, cloud security - -preview:[] - -beta:[] - -This page describes how to set up Cloud Workload Protection (CWP) for Kubernetes. - -.Requirements -[NOTE] -==== -* Kubernetes node operating systems must have Linux kernels 5.10.16 or higher. -==== - -[discrete] -[[security-d4c-get-started-initial-setup]] -== Initial setup - -First, you'll need to deploy Elastic's Defend for Containers integration to the Kubernetes clusters you wish to monitor. - -. Find **Container Workload Security** in the navigation menu or use the global search field. Click **Add D4C Integration**. -. Name the integration. The default name, which you can change, is `cloud_defend-1`. -. Optional — make any desired changes to the integration's policy by adjusting the **Selectors** and **Responses** sections. (For more information, refer to the <>). You can also change these later. -. Under **Where to add this integration**, select an existing or new agent policy. -. Click **Save & Continue**, then **Add {agent} to your hosts**. -. On the {agent} policy page, click **Add agent** to open the Add agent flyout. -. In the flyout, go to step 3 (**Install {agent} on your host**) and select the **Kubernetes** tab. -. Download or copy the manifest (`elastic-agent-managed-kubernetes.yml`). -. Open the manifest using your favorite editor, and uncomment the `#capabilities` section: -+ -[source,console] ----- -#capabilities: -# add: -# - BPF # (since Linux 5.8) allows loading of BPF programs, create most map types, load BTF, iterate programs and maps. -# - PERFMON # (since Linux 5.8) allows attaching of BPF programs used for performance metrics and observability operations. -# - SYS_RESOURCE # Allow use of special resources or raising of resource limits. Used by 'Defend for Containers' to modify 'rlimit_memlock' ----- -. From the directory where you saved the manifest, run the command `kubectl apply -f elastic-agent-managed-kubernetes.yml`. -. Wait for the **Confirm agent enrollment** dialogue to show that data has started flowing from your newly-installed agent, then click **Close**. - -[discrete] -[[d4c-get-started-threat]] -== Get started with threat detection - -One of the <> sends process telemetry events (`fork` and `exec`) to {es}. - -In order to detect threats using this data, you'll need active <>. Elastic has prebuilt detection rules designed for this data. (You can also create your own <>.) - -To install and enable the prebuilt rules: - -. Find **Detection rules (SIEM)** in the navigation menu or use the global search field, then click **Add Elastic rules**. -. Click the **Tags** filter next to the search bar, and search for the `Data Source: Elastic Defend for Containers` tag. -. Select all the displayed rules, then click **Install _x_ selected rule(s)**. -. Return to the **Rules** page. Click the **Tags** filter next to the search bar, and search for the `Data Source: Elastic Defend for Containers` tag. -. Select all the rules with the tag, and then click **Bulk actions → Enable**. - -[discrete] -[[d4c-get-started-drift]] -== Get started with drift detection and prevention - -{elastic-sec} defines container drift as the creation or modification of an executable within a container. Blocking drift restricts the number of attack vectors available to bad actors by prohibiting them from using external tools. - -To enable drift detection, you can use the default D4C policy: - -. Make sure the <> is active. -. Make sure you enabled at least the "Container Workload Protection" rule, by following the steps to install prebuilt rules, above. - -To enable drift prevention, create a new policy: - -. Find **Container Workload Security** in the navigation menu or use the global search field, then select your integration. -. Under **Selectors**, click **Add selector → File Selector**. By default, it selects the operations `createExecutable` and `modifyExecutable`. -. Name the selector, for example: `blockDrift`. -. Scroll down to the **Responses** section and click **Add response → File Response**. -. Under **Match selectors**, add the name of your new selector, for example: `blockDrift`. -. Select the **Alert** and **Block** actions. -. Click **Save integration**. - -[IMPORTANT] -==== -Before you enable blocking, we strongly recommend you observe a production workload that's using the default D4C policy to ensure that the workload does not create or modify executables as part of its normal operation. -==== - -[discrete] -[[d4c-get-started-validation]] -== Policy validation - -To ensure the stability of your production workloads, you should test policy changes before implementing them in production workloads. We also recommend you test policy changes on a simulated environment with workloads similar to production. This approach allows you to test that policy changes prevent undesirable behavior without disrupting your production workloads. diff --git a/docs/serverless/cloud-native-security/environment-variable-capture.asciidoc b/docs/serverless/cloud-native-security/environment-variable-capture.asciidoc deleted file mode 100644 index 311796c7a3..0000000000 --- a/docs/serverless/cloud-native-security/environment-variable-capture.asciidoc +++ /dev/null @@ -1,42 +0,0 @@ -[[security-environment-variable-capture]] -= Capture environment variables - -// :description: Capture environment variables from monitored Linux sessions. -// :keywords: serverless, security, overview, cloud security - -preview:[] - -You can configure an {agent} policy to capture up to five environment variables (`env vars`). - -[NOTE] -==== -* Env var names must be no more than 63 characters, and env var values must be no more than 1023 characters. Values outside these limits are silently ignored. -* Env var names are case sensitive. -==== - -To set up environment variable capture for an {agent} policy: - -. Find **Policies** in the navigation menu or use the global search field. -. Select an {agent} policy. -. Click **Show advanced settings**. -. Scroll down or search for `linux.advanced.capture_env_vars`, or `mac.advanced.capture_env_vars`. -. Enter the names of env vars you want to capture, separated by commas. For example: `PATH,USER` -. Click **Save**. - -[role="screenshot"] -image::images/environment-variable-capture/-cloud-native-security-env-var-capture.png[The "linux.advanced.capture_env_vars" advanced agent policy setting] - -[discrete] -[[find-cap-env-vars]] -== Find captured environment variables - -Captured environment variables are associated with process events, and appear in each event's `process.env_vars` field. - -To view environment variables in the **Events** table: - -. Click the **Events** tab on the **Hosts**, **Network**, or **Users** pages, then click **Fields** in the Events table. -. Search for the `process.env_vars` field, select it, and click **Close**. -A new column appears containing captured environment variable data. - -[role="screenshot"] -image::images/environment-variable-capture/-cloud-native-security-env-var-capture-detail.png[The Events table with the "process.env_vars" column highlighted] diff --git a/docs/serverless/cloud-native-security/get-started-with-kspm.asciidoc b/docs/serverless/cloud-native-security/get-started-with-kspm.asciidoc deleted file mode 100644 index 2380fb9fef..0000000000 --- a/docs/serverless/cloud-native-security/get-started-with-kspm.asciidoc +++ /dev/null @@ -1,446 +0,0 @@ -[[security-get-started-with-kspm]] -= Get started with KSPM - -// :keywords: serverless, security, overview, cloud security - -preview:[] - -This page explains how to configure the Kubernetes Security Posture Management (KSPM) integration. - -.Requirements -[NOTE] -==== -* KSPM only works in the `Default` {kib} space. Installing the KSPM integration on a different {kib} space will not work. -* KSPM is not supported on EKS clusters in AWS GovCloud (https://github.com/elastic/kibana/issues/new/choose[request support]). -* To view posture data, ensure you have the appropriate user role to read the following {es} indices: - -* `logs-cloud_security_posture.findings_latest-*` -* `logs-cloud_security_posture.scores-*` -* `logs-cloud_security_posture.findings` -==== - -The instructions differ depending on whether you're installing on EKS or on unmanaged clusters. - -* Install on EKS-managed clusters: -+ -.. <> -.. <> -.. <> -.. <> -* Install on unmanaged clusters: -+ -.. <> -.. <> - -[discrete] -[[kspm-setup-eks-start]] -== Set up KSPM for Amazon EKS clusters - -[discrete] -[[security-get-started-with-kspm-name-your-integration-and-select-a-kubernetes-deployment-type]] -=== Name your integration and select a Kubernetes Deployment type - -. Find **Cloud Security Posture** in the navigation menu or use the global search field. -. Click **Add a KSPM integration**. -. Read the integration's description to understand how it works. Then, click {integrations-docs}/cloud_security_posture[_Add Kubernetes Security Posture Management_]. -. Name your integration. Use a name that matches the purpose or team of the cluster(s) you want to monitor, for example, `IT-dev-k8s-clusters`. -. Select **EKS** from the **Kubernetes Deployment** menu. A new section for AWS credentials will appear. - -[discrete] -[[kspm-setup-eks-auth]] -=== Authenticate to AWS - -There are several options for how to provide AWS credentials: - -* <> -* <> -* <> -* <> -* <> -* <> - -Regardless of which option you use, you'll need to grant the following permissions: - -[source,console] ----- -ecr:GetRegistryPolicy, -eks:ListTagsForResource -elasticloadbalancing:DescribeTags -ecr-public:DescribeRegistries -ecr:DescribeRegistry -elasticloadbalancing:DescribeLoadBalancerPolicyTypes -ecr:ListImages -ecr-public:GetRepositoryPolicy -elasticloadbalancing:DescribeLoadBalancerAttributes -elasticloadbalancing:DescribeLoadBalancers -ecr-public:DescribeRepositories -eks:DescribeNodegroup -ecr:DescribeImages -elasticloadbalancing:DescribeLoadBalancerPolicies -ecr:DescribeRepositories -eks:DescribeCluster -eks:ListClusters -elasticloadbalancing:DescribeInstanceHealth -ecr:GetRepositoryPolicy ----- - -If you are using the AWS visual editor to create and modify your IAM Policies, you can copy and paste this IAM policy JSON object: - -.Click to view JSON object -[%collapsible] -===== -[source,json] ----- -{ - "Version": "2012-10-17", - "Statement": [ - { - "Sid": "VisualEditor0", - "Effect": "Allow", - "Action": [ - "ecr:GetRegistryPolicy", - "eks:ListTagsForResource", - "elasticloadbalancing:DescribeTags", - "ecr-public:DescribeRegistries", - "ecr:DescribeRegistry", - "elasticloadbalancing:DescribeLoadBalancerPolicyTypes", - "ecr:ListImages", - "ecr-public:GetRepositoryPolicy", - "elasticloadbalancing:DescribeLoadBalancerAttributes", - "elasticloadbalancing:DescribeLoadBalancers", - "ecr-public:DescribeRepositories", - "eks:DescribeNodegroup", - "ecr:DescribeImages", - "elasticloadbalancing:DescribeLoadBalancerPolicies", - "ecr:DescribeRepositories", - "eks:DescribeCluster", - "eks:ListClusters", - "elasticloadbalancing:DescribeInstanceHealth", - "ecr:GetRepositoryPolicy" - ], - "Resource": "*" - } - ] -} ----- -===== - -[discrete] -[[kspm-use-irsa]] -==== Option 1 - [Recommended] Use Kubernetes Service Account to assume IAM role - -Follow AWS's https://aws.github.io/aws-eks-best-practices/security/docs/iam/#iam-roles-for-service-accounts-irsa[EKS Best Practices] documentation to use the https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html[IAM Role to Kubernetes Service-Account] (IRSA) feature to get temporary credentials and scoped permissions. - -[IMPORTANT] -==== -During setup, do not fill in any option in the "Setup Access" section. Click **Save and continue**. -==== - -[discrete] -[[kspm-use-instance-role]] -==== Option 2 - Use default instance role - -Follow AWS's https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html[IAM roles for Amazon EC2] documentation to create an IAM role using the IAM console, which automatically generates an instance profile. - -[IMPORTANT] -==== -During setup, do not fill in any option in the "Setup Access" section. Click **Save and continue**. -==== - -[discrete] -[[kspm-use-keys-directly]] -==== Option 3 - Use access keys directly - -Access keys are long-term credentials for an IAM user or AWS account root user. To use access keys as credentials, you must provide the `Access key ID` and the `Secret Access Key`. - -For more details, refer to AWS' https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html[Access Keys and Secret Access Keys] documentation. - -[IMPORTANT] -==== -You must select "Programmatic access" when creating the IAM user. -==== - -[discrete] -[[kspm-use-temp-credentials]] -==== Option 4 - Use temporary security credentials - -You can configure temporary security credentials in AWS to last for a specified duration. They consist of an access key ID, a secret access key, and a security token, which is typically found using `GetSessionToken`. - -Because temporary security credentials are short term, once they expire, you will need to generate new ones and manually update the integration's configuration to continue collecting cloud posture data. Update the credentials before they expire to avoid data loss. - -[NOTE] -==== -IAM users with multi-factor authentication (MFA) enabled need to submit an MFA code when calling `GetSessionToken`. For more details, refer to AWS' https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html[Temporary Security Credentials] documentation. -==== - -You can use the AWS CLI to generate temporary credentials. For example, you could use the following command if you have MFA enabled: - -[source,console] ----- -`sts get-session-token --serial-number arn:aws:iam::1234:mfa/your-email@example.com --duration-seconds 129600 --token-code 123456` ----- - -The output from this command includes the following fields, which you should provide when configuring the KSPM integration: - -* `Access key ID`: The first part of the access key. -* `Secret Access Key`: The second part of the access key. -* `Session Token`: A token required when using temporary security credentials. - -[discrete] -[[kspm-use-a-shared-credentials-file]] -==== Option 5 - Use a shared credentials file - -If you use different AWS credentials for different tools or applications, you can use profiles to define multiple access keys in the same configuration file. For more details, refer to AWS' https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html[Shared Credentials Files] documentation. - -Instead of providing the `Access key ID` and `Secret Access Key` to the integration, provide the information required to locate the access keys within the shared credentials file: - -* `Credential Profile Name`: The profile name in the shared credentials file. -* `Shared Credential File`: The directory of the shared credentials file. - -If you don't provide values for all configuration fields, the integration will use these defaults: - -* If `Access key ID`, `Secret Access Key`, and `ARN Role` are not provided, then the integration will check for `Credential Profile Name`. -* If there is no `Credential Profile Name`, the default profile will be used. -* If `Shared Credential File` is empty, the default directory will be used. -+ -** For Linux or Unix, the shared credentials file is located at `~/.aws/credentials`. - -[discrete] -[[kspm-use-iam-arn]] -==== Option 6 - Use an IAM role Amazon Resource Name (ARN) - -An IAM role Amazon Resource Name (ARN) is an IAM identity that you can create in your AWS account. You define the role's permissions. -Roles do not have standard long-term credentials such as passwords or access keys. -Instead, when you assume a role, it provides temporary security credentials for your session. -An IAM role's ARN can be used to specify which AWS IAM role to use to generate temporary credentials. - -For more details, refer to AWS' https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html[AssumeRole API] documentation. -Follow AWS' instructions to https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html[create an IAM user], and define the IAM role's permissions using the JSON permissions policy above. - -To use an IAM role's ARN, you need to provide either a <> or <> along with the `ARN role`. -The `ARN Role` value specifies which AWS IAM role to use for generating temporary credentials. - -[NOTE] -==== -If `ARN Role` is present, the integration will check if `Access key ID` and `Secret Access Key` are present. -If not, the package will check for a `Credential Profile Name`. -If a `Credential Profile Name` is not present, the default credential profile will be used. -==== - -[discrete] -[[kspm-setup-eks-finish]] -=== Finish configuring the KSPM integration for EKS - -Once you've provided AWS credentials, finish configuring the KSPM integration: - -. If you want to monitor Kubernetes clusters that aren’t yet enrolled in {fleet}, select **New Hosts** under “where to add this integration”. -. Name the {agent} policy. Use a name that matches the purpose or team of the cluster(s) you want to monitor. For example, `IT-dev-k8s-clusters`. -. Click **Save and continue**, then **Add agent to your hosts**. The **Add agent** wizard appears and provides a DaemonSet manifest `.yaml` file with pre-populated configuration information, such as the `Fleet ID` and `Fleet URL`. - -[discrete] -[[kspm-setup-eks-modify-deploy]] -=== Deploy the KSPM integration to EKS clusters - -The **Add agent** wizard helps you deploy the KSPM integration on the Kubernetes clusters you wish to monitor. For each cluster: - -. Download the manifest and make any necessary revisions to its configuration to suit the needs of your environment. -. Apply the manifest using the `kubectl apply -f` command. For example: `kubectl apply -f elastic-agent-managed-kubernetes.yaml` - -After a few minutes, a message confirming the {agent} enrollment appears, followed by a message confirming that data is incoming. You can then click **View assets** to see where the newly-collected configuration information appears, including the <> and the <>. - -[discrete] -[[kspm-setup-unmanaged]] -== Set up KSPM for unmanaged Kubernetes clusters - -Follow these steps to deploy the KSPM integration to unmanaged clusters. Keep in mind credentials are NOT required for unmanaged deployments. - -[discrete] -[[security-get-started-with-kspm-configure-the-kspm-integration]] -=== Configure the KSPM integration - -To install the integration on unmanaged clusters: - -. Find **Connectors** in the navigation menu or use the global search field. -. Click **Add a KSPM integration**. -. Read the integration's description to understand how it works. Then, click {integrations-docs}/cloud_security_posture[_Add Kubernetes Security Posture Management_]. -. Name your integration. Use a name that matches the purpose or team of the cluster(s) you want to monitor, for example, `IT-dev-k8s-clusters`. -. Select **Unmanaged Kubernetes** from the **Kubernetes Deployment** menu. -. If you want to monitor Kubernetes clusters that aren’t yet enrolled in {fleet}, select **New Hosts** when choosing the {agent} policy. -. Select the {agent} policy where you want to add the integration. -. Click **Save and continue**, then **Add agent to your hosts**. The **Add agent** wizard appears and provides a DaemonSet manifest `.yaml` file with pre-populated configuration information, such as the `Fleet ID` and `Fleet URL`. - -[role="screenshot"] -image::images/get-started-with-kspm/-cloud-native-security-kspm-add-agent-wizard.png[The KSPM integration's Add agent wizard] - -[discrete] -[[kspm-setup-unmanaged-modify-deploy]] -=== Deploy the KSPM integration to unmanaged clusters - -The **Add agent** wizard helps you deploy the KSPM integration on the Kubernetes clusters you wish to monitor. To do this, for each cluster: - -. Download the manifest and make any necessary revisions to its configuration to suit the needs of your environment. -. Apply the manifest using the `kubectl apply -f` command. For example: `kubectl apply -f elastic-agent-managed-kubernetes.yaml` - -After a few minutes, a message confirming the {agent} enrollment appears, followed by a message confirming that data is incoming. You can then click **View assets** to see where the newly-collected configuration information appears, including the <> and the <>. - -[discrete] -[[kspm-eck]] -=== Set up KSPM on ECK deployments - -To run KSPM on an https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html[ECK] deployment, -you must edit the https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-configuration.html[Elastic Agent CRD] and https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-configuration.html#k8s-elastic-agent-role-based-access-control[Elastic Agent Cluster-Role] `.yaml` files. - -.Patch Elastic Agent -[%collapsible] -===== -Add `volumes` and `volumeMounts` to `podTemplate`: - -[source,yaml] ----- -podTemplate: - spec: - containers: - - name: agent - volumeMounts: - - name: proc - mountPath: /hostfs/proc - readOnly: true - - name: cgroup - mountPath: /hostfs/sys/fs/cgroup - readOnly: true - - name: varlibdockercontainers - mountPath: /var/lib/docker/containers - readOnly: true - - name: varlog - mountPath: /var/log - readOnly: true - - name: etc-full - mountPath: /hostfs/etc - readOnly: true - - name: var-lib - mountPath: /hostfs/var/lib - readOnly: true - - name: etc-mid - mountPath: /etc/machine-id - readOnly: true - volumes: - - name: proc - hostPath: - path: /proc - - name: cgroup - hostPath: - path: /sys/fs/cgroup - - name: varlibdockercontainers - hostPath: - path: /var/lib/docker/containers - - name: varlog - hostPath: - path: /var/log - - name: etc-full - hostPath: - path: /etc - - name: var-lib - hostPath: - path: /var/lib - # Mount /etc/machine-id from the host to determine host ID - # Needed for Elastic Security integration - - name: etc-mid - hostPath: - path: /etc/machine-id - type: File ----- -===== - -.Patch RBAC -[%collapsible] -===== -Make sure that the `elastic-agent` service-account has the following Role and ClusterRole: - -[source,yaml] ----- -apiVersion: rbac.authorization.k8s.io/v1 -kind: RoleBinding -metadata: - namespace: default - name: elastic-agent -subjects: -- kind: ServiceAccount - name: elastic-agent - namespace: default -roleRef: - kind: Role - name: elastic-agent - apiGroup: rbac.authorization.k8s.io ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: elastic-agent - labels: - k8s-app: elastic-agent -rules: -- apiGroups: [""] - resources: - - nodes - - namespaces - - events - - pods - - services - - configmaps - - serviceaccounts - - persistentvolumes - - persistentvolumeclaims - verbs: ["get", "list", "watch"] -- apiGroups: ["extensions"] - resources: - - replicasets - verbs: ["get", "list", "watch"] -- apiGroups: ["apps"] - resources: - - statefulsets - - deployments - - replicasets - - daemonsets - verbs: ["get", "list", "watch"] -- apiGroups: - - "" - resources: - - nodes/stats - verbs: - - get -- apiGroups: [ "batch" ] - resources: - - jobs - - cronjobs - verbs: [ "get", "list", "watch" ] -- nonResourceURLs: - - "/metrics" - verbs: - - get -- apiGroups: ["rbac.authorization.k8s.io"] - resources: - - clusterrolebindings - - clusterroles - - rolebindings - - roles - verbs: ["get", "list", "watch"] -- apiGroups: ["policy"] - resources: - - podsecuritypolicies - verbs: ["get", "list", "watch"] ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: Role -metadata: - name: elastic-agent - namespace: default - labels: - k8s-app: elastic-agent -rules: - - apiGroups: - - coordination.k8s.io - resources: - - leases - verbs: ["get", "create", "update"] ----- -===== diff --git a/docs/serverless/cloud-native-security/kspm.asciidoc b/docs/serverless/cloud-native-security/kspm.asciidoc deleted file mode 100644 index a5a9e8b200..0000000000 --- a/docs/serverless/cloud-native-security/kspm.asciidoc +++ /dev/null @@ -1,86 +0,0 @@ -[[security-kspm]] -= Kubernetes security posture management - -// :description: Identify configuration risks in your Kubernetes clusters. -// :keywords: serverless, security, overview, cloud security - -preview:[] - -[discrete] -[[kspm-overview]] -== Overview - -The Kubernetes Security Posture Management (KSPM) integration allows you to identify configuration risks in the various components that make up your Kubernetes cluster. -It does this by evaluating your Kubernetes clusters against secure configuration guidelines defined by the Center for Internet Security (CIS) and generating findings with step-by-step instructions for remediating potential security risks. - -This integration supports Amazon EKS and unmanaged Kubernetes clusters. For setup instructions, refer to <>. - -.Requirements -[NOTE] -==== -* KSPM only works in the `Default` {kib} space. Installing the KSPM integration on a different {kib} space will not work. -* KSPM is not supported on EKS clusters in AWS GovCloud (https://github.com/elastic/kibana/issues/new/choose[request support]). -* To view posture data, ensure you have the appropriate user role to read the following {es} indices: - -* `logs-cloud_security_posture.findings_latest-*` -* `logs-cloud_security_posture.scores-*` -* `logs-cloud_security_posture.findings` -==== - -[discrete] -[[kspm-how-kspm-works]] -== How KSPM works - -. When you add a KSPM integration, it generates a Kubernetes manifest. When applied to a cluster, the manifest deploys an {agent} as a https://kubernetes.io/docs/concepts/workloads/controllers/daemonset[DaemonSet] to ensure all nodes are evaluated. -. Upon deployment, the integration immediately assesses the security posture of your Kubernetes resources. The evaluation process repeats every four hours. -. After each evaluation, the integration sends findings to {es}. Findings appear on the <> and the <> page. - -[discrete] -[[kspm-use-cases]] -== Use cases - -The KSPM integration helps you to: - -* Identify and remediate `failed` findings -* Identify the most misconfigured resources -* Identify risks in particular CIS benchmark sections - -[discrete] -[[kspm-remediate-failed-findings]] -=== Identify and remediate failed findings - -To identify and remediate failed failed findings: - -. Go to the <>. -. Click **View all failed findings**, either for an individual cluster or for all monitored clusters. -. Click a failed finding. The findings flyout opens. -. Follow the steps under **Remediation** to correct the misconfiguration. -+ -[NOTE] -==== -Remediation steps typically include commands for you to execute. These sometimes contain placeholder values that you must replace before execution. -==== - -[discrete] -[[kspm-identify-misconfigured-resources]] -=== Identify the most misconfigured Kubernetes resources - -To identify the Kubernetes resources generating the most failed findings: - -. Go to the <> page. -. Click the **Group by** menu near the search box and select **Resource** to view a list of resources sorted by their total number of failed findings. -. Click a resource ID to view the findings associated with that resource. - -[discrete] -[[kspm-identify-config-risks-by-section]] -=== Identify configuration risks by CIS section - -To identify risks in particular CIS sections: - -. Go to the <>. -. In the Failed findings by CIS section widget, click the name of a CIS section to view all failed findings for that section. - -Alternatively: - -. Go to the Findings page. -. Filter by the `rule.section` field. For example, search for `rule.section : API Server` to view findings for benchmark rules in the API Server category. diff --git a/docs/serverless/cloud-native-security/security-posture-management.asciidoc b/docs/serverless/cloud-native-security/security-posture-management.asciidoc deleted file mode 100644 index 0975975752..0000000000 --- a/docs/serverless/cloud-native-security/security-posture-management.asciidoc +++ /dev/null @@ -1,50 +0,0 @@ -[[security-posture-management]] -= Security posture management overview - -// :description: Discovers and evaluates your cloud services and resources against security best practices. -// :keywords: serverless, security, overview, cloud security - -preview:[] - -[discrete] -== Overview - -Elastic's <> (CSPM) and <> (KSPM) features help you discover and evaluate the services and resources in your cloud environment — like storage, compute, IAM, and more — against security guidelines defined by the Center for Internet Security (CIS). They help you identify and remediate configuration risks that could undermine the confidentiality, integrity, and availability of your cloud assets, such as publicly exposed storage buckets or overly permissive networking objects. - -The KSPM feature assesses the security of your Kubernetes assets, while the CSPM feature assesses the security of your AWS resources such as storage, compute, IAM, and more. - -[discrete] -[[security-posture-management-get-started]] -== Getting started - -For setup instructions, refer to: - -* <> -* <> - -[discrete] -[[security-posture-use-cases]] -== Use cases - -Using the data generated by these features, you can: - -**Identify and secure misconfigured infrastructure:** - -. Find **Cloud Security Posture** in the navigation menu or use the global search field. -. Click **View all failed findings**, either for an individual resource or a group of resources. -. Click a failed finding to open the Findings flyout. -. Follow the steps under Remediation to fix the misconfiguration. - -**Identify the CIS Sections (security best practice categories) with which your resources are least compliant:** - -. Find **Cloud Security Posture** in the navigation menu or use the global search field. -. Do one of the following: -+ -.. Under Failed findings by CIS section, click the name of a CIS section to view all failed findings from that section. -.. Go to the **Findings** page and filter by the `rule.section` field. For example, search for `rule.section : API Server` to view findings from the API Server category. - -**Identify your least compliant cloud resources** - -. Go to the **Findings** page. -. Click the **Group by** menu near the search box, and select **Resource** to sort resources by their number of failed findings. -. Click a resource ID to view associated findings. diff --git a/docs/serverless/cloud-native-security/vuln-management-get-started.asciidoc b/docs/serverless/cloud-native-security/vuln-management-get-started.asciidoc deleted file mode 100644 index b2a2157621..0000000000 --- a/docs/serverless/cloud-native-security/vuln-management-get-started.asciidoc +++ /dev/null @@ -1,77 +0,0 @@ -[[security-vuln-management-get-started]] -= Get started with CNVM - -// :description: Set up cloud native vulnerability management. -// :keywords: serverless, security, overview, cloud security - -preview:[] - -This page explains how to set up Cloud Native Vulnerability Management (CNVM). - -.Requirements -[NOTE] -==== -* CNVM only works in the `Default` {kib} space. Installing the CNVM integration on a different {kib} space will not work. -* Requires {agent} version 8.8 or higher. -* CNVM can only be deployed on ARM-based VMs. -* To view vulnerability scan findings, you need the appropriate user role to read the following indices: -+ -** `logs-cloud_security_posture.vulnerabilities-*` -** `logs-cloud_security_posture.vulnerabilities_latest-*` -* You need an AWS user account with permissions to perform the following actions: run CloudFormation templates, create IAM Roles and InstanceProfiles, and create EC2 SecurityGroups and Instances. -==== - -[NOTE] -==== -CNVM currently only supports AWS EC2 Linux workloads. -==== - -[discrete] -[[vuln-management-setup]] -== Set up CNVM for AWS - -To set up the CNVM integration for AWS, install the integration on a new {agent} policy, sign into the AWS account you want to scan, and run the https://docs.aws.amazon.com/cloudformation/index.html[CloudFormation] template. - -[IMPORTANT] -==== -Do not add the integration to an existing {agent} policy. It should always be added to a new policy since it should not run on VMs with existing workloads. For more information, refer to <>. -==== - -[discrete] -[[vuln-management-setup-step-1]] -=== Step 1: Add the CNVM integration - -. Find **Integrations** in the navigation menu or use the global search field. -. Search for **Cloud Native Vulnerability Management**, then click on the result. -. Click **Add Cloud Native Vulnerability Management**. -. Give your integration a name that matches its purpose or the AWS account region you want to scan for vulnerabilities (for example, `uswest2-aws-account`.) -+ -[role="screenshot"] -image::images/vuln-management-get-started/-dashboards-cnvm-setup-1.png[The CNVM integration setup page] -. Click **Save and continue**. The integration will create a new {agent} policy. -. Click **Add {agent} to your hosts**. - -[discrete] -[[vuln-management-setup-step-2]] -=== Step 2: Sign in to the AWS management console - -. Open a new browser tab and use it to sign into your AWS management console. -. Switch to the cloud region with the workloads that you want to scan for vulnerabilities. - -[IMPORTANT] -==== -The integration will only scan VMs in the region you select. To scan multiple regions, repeat this setup process for each region. -==== - -[discrete] -[[vuln-management-setup-step-3]] -=== Step 3: Run the CloudFormation template - -. Switch back to the tab with Elastic Security. -. Click **Launch CloudFormation**. The CloudFormation page appears. -+ -[role="screenshot"] -image::images/vuln-management-get-started/-dashboards-cnvm-cloudformation.png[The cloud formation template] -. Click **Create stack**. To avoid authentication problems, you can only make configuration changes to the VM InstanceType, which you could make larger to increase scanning speed. -. Wait for the confirmation that {agent} was enrolled. -. Your data will start to appear on the **Vulnerabilities** tab of the <>.