diff --git a/articles/sentinel/aws-cloudformation/aws-cloudformation-overview.md b/articles/sentinel/aws-cloudformation/aws-cloudformation-overview.md new file mode 100644 index 0000000000000..42194c6dddabc --- /dev/null +++ b/articles/sentinel/aws-cloudformation/aws-cloudformation-overview.md @@ -0,0 +1,223 @@ +--- +title: Automate AWS log collection for Microsoft Sentinel with AWS CloudFormation +description: Use AWS CloudFormation templates to deploy the Amazon Web Services S3-based data connector for Microsoft Sentinel and automatically create the required AWS resources. +author: KanenasCS +ms.author: bagol +ms.service: microsoft-sentinel +ms.topic: how-to +ms.collection: microsoft-sentinel +ms.date: 12/01/2025 +--- + +# Automate AWS log collection for Microsoft Sentinel Amazon Web Services S3 connector with AWS CloudFormation + +## Overview + +This repository contains infrastructure and automation to onboard AWS security logs into **Microsoft Sentinel** using the **AWS S3 data connector**. + +Supported log sources (per stack): + +- **VPC Flow Logs** +- **CloudTrail** (management and data events) +- **GuardDuty** findings +- **CloudWatch** (via Lambda export to S3) + +All stacks follow the same high-level pattern: + +1. The AWS service writes logs to an **S3 bucket**. +2. **S3 → SQS notifications** are configured for new log objects. +3. Microsoft Sentinel assumes an **OIDC IAM role** in the AWS account and: + - Reads messages from **SQS**. + - Fetches the log files from **S3** and ingests them into the Sentinel workspace. + +--- + +## One-time OIDC prerequisite (applies to all stacks) + +> **Important:** All security stacks in this repo rely on a common **OIDC trust** between AWS and Microsoft Entra ID (Azure AD). + +- You deploy the **OIDC + Sentinel role** stack **once** per AWS account/region. +- After that, every log-specific stack (VPC Flow Logs, CloudTrail, GuardDuty, CloudWatch) **reuses the same OIDC provider and Sentinel IAM role**. +- When you onboard a new log type later, you do **not** redeploy OIDC – you only deploy the new log-specific stack and reference the existing role. + +In short: +**Deploy OIDC once → reuse it for all AWS security log stacks.** + +--- + +## Repository structure (high level) + +onAwsPolicies.ps1`, `HelperFunctions.ps1`, `AwsPoliciesUpdate.ps1`, `AwsResourceCreator.ps1`, `AwsSentinelTag.ps1`, `EnviornmentConstants.ps1` – shared helper functions and policy definitions. + +- **CloudFormation templates** + + (Example names – align them with your repo): + + - `aws-s3-vpcflowlogs.yaml` – VPC Flow Logs S3 + SQS + resource policies. + - `aws-s3-cloudtrail.yaml` – CloudTrail S3 + SQS (+ optional KMS) + resource policies. + - `aws-s3-guardduty.yaml` – GuardDuty S3 + KMS + SQS + resource policies. + - `aws-s3-cloudwatch-part1.yaml` – **CloudWatch Part 1:** OIDC Sentinel role, CloudWatch S3 bucket, SQS queue, bucket policy, queue policy, and S3 notification. + - `aws-s3-cloudwatch-part2-lambda.yaml` – **CloudWatch Part 2:** Lambda exporter, IAM role, and EventBridge schedule. + +- **Lambda functions** (for CloudWatch) + - `CloudWatchLambdaFunction.py`, `CloudWatchLambdaFunction_V2.py`, `CloudWatchLambdaFunction_V3.py` etc. – reference implementations for CloudWatch → S3 export. + The Part 2 CloudFormation stack embeds the latest, validated version inline. + +- **Supporting content** + - `README.md` (this document). + - Per-log-type onboarding guides (WAF, CloudTrail, CloudWatch, etc). + +--- + +## Prerequisites + +### Azure side + +- An **Azure subscription** with: + - A **Log Analytics workspace**. + - **Microsoft Sentinel** enabled on that workspace. +- Permissions in Azure to: + - Configure **data connectors** for the Sentinel workspace. + - View and copy the **Workspace ID (External ID)** from the AWS S3 connectors. + - Create and manage identities used for OIDC / role assumption (if required by your governance). +- RBAC: **Microsoft Sentinel Contributor** or equivalent on the workspace is recommended. + +### AWS side + +- An **AWS account** where the stacks will be deployed. +- IAM permissions (for the deploying identity) to: + - Deploy and manage **CloudFormation** stacks: + - `cloudformation:CreateStack`, `UpdateStack`, `DescribeStacks`, `ListStacks`, `DeleteStack`. + - Create and manage: + - **IAM roles and policies** (Sentinel OIDC role, Lambda role, EventBridge role). + - **S3 buckets** and bucket policies. + - **SQS queues** and queue policies. + - **KMS keys** and key policies (for encrypted services such as GuardDuty and CloudTrail, where used). + - **Lambda functions** and **EventBridge** rules (for CloudWatch exporter). + +### Admin workstation + +- **PowerShell** (latest recommended version). +- **AWS CLI**, configured with credentials that can deploy the resources: + - Run `aws configure` and ensure access to the target AWS account(s) and Region(s). +- Optionally, **Azure CLI** or the Azure portal to validate connector status and Workspace ID. + +--- + +## Security stacks per log type + +Each log type has its own CloudFormation-based “security stack”. At a high level, each stack: + +1. Creates or uses an **S3 bucket** for that log type. +2. Creates an **SQS queue** for S3 notifications. +3. Configures **S3 → SQS** notifications (filtering by prefix/suffix, as required). +4. Grants the **Sentinel OIDC IAM role** permissions to: + - `sqs:ReceiveMessage`, `sqs:DeleteMessage`, `sqs:ChangeMessageVisibility`, `sqs:GetQueueAttributes`. + - `s3:GetObject` and related read actions on the log bucket. +5. If encryption is used, adds **KMS key policies** for: + - The AWS service principal (for example `cloudtrail.amazonaws.com`, `guardduty.amazonaws.com`). + - The Sentinel OIDC role so it can decrypt encrypted log files. + +Below is a summary for each log type. + +### VPC Flow Logs stack + +**Template:** `aws-s3-vpcflowlogs.yaml` + +This stack: + +- Creates or reuses an **S3 bucket** dedicated to VPC Flow Logs. +- Configures **VPC Flow Logs** to deliver to that bucket using the required IAM delivery role. +- Creates an **SQS queue** for the Sentinel connector (for example, `omicron-sentinel-vpcflowlogs-queue`). +- Configures **S3 event notifications** for the VPC Flow Logs prefix (e.g. `AWSLogs//vpcflowlogs/`). +- Adds: + - An S3 **bucket policy** allowing the Sentinel OIDC role to list and read log objects. + - An SQS **queue policy** allowing the bucket to send messages and Sentinel to read from the queue. + +### CloudTrail stack + +**Template:** `aws-s3-cloudtrail.yaml` + +This stack: + +- Creates or uses the **CloudTrail S3 bucket**. +- Optionally creates or updates a **CloudTrail trail** that delivers management and data events to that bucket. +- Creates an **SQS queue** for CloudTrail notifications. +- Configures **S3 → SQS** notifications for the CloudTrail log prefix. +- If CloudTrail uses **SSE-KMS**, the template configures: + - A **KMS key** (or references an existing one). + - Key policy entries for `cloudtrail.amazonaws.com` and the Sentinel OIDC role. +- Adds: + - S3 bucket policy entries for Sentinel read access. + - SQS queue policy allowing S3 to send and Sentinel to consume messages. + +### GuardDuty stack + +**Template:** `aws-s3-guardduty.yaml` + +This stack: + +- Creates or uses a dedicated **S3 bucket** for GuardDuty findings. +- Creates a **KMS key** for encrypting GuardDuty exports (if not already present). +- Configures **GuardDuty export** to the S3 bucket with KMS encryption. +- Creates an **SQS queue** for GuardDuty connector messages. +- Configures **S3 event notifications** for GuardDuty prefixes. +- Sets: + - A bucket policy that grants GuardDuty the right to write, and the Sentinel role the right to read. + - A KMS key policy that allows GuardDuty to encrypt and the Sentinel role to decrypt. + - An SQS queue policy that allows S3 to send and Sentinel to receive messages. + +### CloudWatch (two-part deployment) + +CloudWatch logs are handled in **two stacks**: + +#### CloudWatch Part 1 – Sentinel role, S3, and SQS + +**Template:** `aws-s3-cloudwatch-part1.yaml` + +This stack: + +- Creates or reuses the **CloudWatch export S3 bucket** + (for example `azure-sentinel-cloudwatch--`). +- Creates the **CloudWatch connector SQS queue** + (for example `omicron-sentinel-cloudwatch-queue`). +- Creates the **Sentinel OIDC IAM role** (if not already deployed globally) or can reference an existing one. +- Adds: + - An S3 **bucket policy** that allows: + - The Sentinel role to `ListBucket` and `GetObject`. + - An SQS **queue policy** that allows: + - The S3 bucket to call `SQS:SendMessage`. + - The Sentinel role to receive and delete messages. +- Configures an **S3 event notification** on the bucket that sends all new `.gz` objects in the CloudWatch export prefix to the SQS queue. + +#### CloudWatch Part 2 – Lambda exporter and schedule + +**Template:** `aws-s3-cloudwatch-part2-lambda.yaml` + +This stack: + +- Creates an **IAM role for the CloudWatch exporter Lambda** with: + - `logs:DescribeLogGroups`, `DescribeLogStreams`, `GetLogEvents`, `FilterLogEvents` on CloudWatch Logs. + - `s3:PutObject` and `s3:AbortMultipartUpload` on the CloudWatch export bucket prefix `AWSLogs//CloudWatch/`. + - Basic Lambda logging permissions. +- Deploys the **Lambda function** that: + - Runs every **N minutes** (default 15) using an **EventBridge schedule**. + - Queries CloudWatch Logs for the last N minutes across all log groups. + - Writes gzipped NDJSON files to the S3 prefix `AWSLogs//CloudWatch//…`. +- Creates an **EventBridge rule** with a `rate(N minutes)` expression that invokes the Lambda function on the configured schedule. + +Sentinel then ingests these CloudWatch logs via the same **S3 → SQS** mechanism as the other stacks. + +--- + +## Next steps + +1. Ensure the **OIDC prerequisite stack** is deployed in the target AWS account/region. +2. Choose the log type(s) you want to onboard (VPC Flow Logs, CloudTrail, GuardDuty, CloudWatch). +3. Deploy the corresponding **CloudFormation stack(s)** from this repo. +4. In **Microsoft Sentinel**, open the matching **AWS S3 data connector**: + - Supply the **role ARN** and **External ID (Workspace ID)** where required. + - Validate that the connector status becomes **Connected**. +5. Confirm that the expected tables (for example `AWSVPCFlow`, `AWSCloudTrail`, `AWSGuardDuty`, `CloudWatch` ) start populating with events. + +This model gives you a repeatable, infrastructure-as-code approach for onboarding multiple AWS log sources into Microsoft Sentinel using a common, secure OIDC foundation. diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/aws-cloudformation-cloudtrail.md b/articles/sentinel/aws-cloudformation/cloudtrail/aws-cloudformation-cloudtrail.md new file mode 100644 index 0000000000000..bcaea16ad4615 --- /dev/null +++ b/articles/sentinel/aws-cloudformation/cloudtrail/aws-cloudformation-cloudtrail.md @@ -0,0 +1,128 @@ +--- +title: Deploy AWS CloudTrail log collection for Microsoft Sentinel with CloudFormation +description: Use an AWS CloudFormation stack to configure AWS CloudTrail to send logs to an S3 bucket and onboard them to Microsoft Sentinel via the AWS S3 connector. +author: KanenasCS +ms.author: bagol +ms.service: microsoft-sentinel +ms.topic: how-to +ms.collection: microsoft-sentinel +ms.date: 12/01/2025 +--- + +# Deploy AWS CloudTrail log collection for Microsoft Sentinel with CloudFormation + +### 1. Microsoft Sentinel configuration + +1. Sign in to the **Azure portal**. +2. Navigate to **Microsoft Sentinel** and then to **Content Hub**. +3. Install the connector **Amazon Web Services S3**. + +![Azure Sentinel Security hub](images/cloudtrail-step-1.png) + +4. Navigate to **Data connectors** and open the **Amazon Web Services S3** connector page. + +--- + +### 2. Create the CloudFormation stack + +1. Sign in to the **AWS Management Console**. +2. In the search bar, search for **CloudFormation** and open the **CloudFormation** service. + +![CloudFormation Service](images/cloudtrail-step-2.png) + +3. Select **Create stack** → **With new resources (standard)**. + +![Create stack](images/cloudtrail-step-3.png) + +--- + +#### 2.1 Step 1 – Specify template + +1. Under **Prepare template**, select **Choose an existing template**. +2. Under **Template source**, select **Upload a template file**, then choose and upload the provided template file. +3. Click **Next**. + +![Step 1](images/cloudtrail-step-4.png) + +--- + +#### 2.2 Step 2 – Specify stack details + +Fill in the following parameters: + +- **Stack name**: Enter a name for the stack. +- **AWSRoleName**: Enter the IAM role name (the name must start with `OIDC_XXXXX`). +- **GuardDutyBucketName**: Enter the name of the S3 bucket to be used. + - If you already have a generic S3 bucket or want to use an existing bucket, enter its name here. +- **BucketName**: Set to `false` if you are using an existing S3 bucket (leave as `true` if a new bucket should be created). +- **CloudTrail-TrailName**: Name of the existing CloudTrail trail that delivers logs to the S3 bucket. +- **SentinelSQSQueueName**: Enter the name of the Amazon SQS queue. +- **LogFileSuffix**: S3 object key suffix for CloudTrail exported findings used in the notification filter (must be `.gz` as the default). +- **SentinelWorkspaceId**: Enter the Workspace ID from the Azure Log Analytics workspace page: + - In the Azure portal, go to **Log Analytics workspace → Overview** and copy the **Workspace ID**. + +![Step 2](images/cloudtrail-step-5.png) + +After filling all required fields, click **Next**. + +![Step 2.1](images/cloudtrail-step-6.png) + +--- + +#### 2.3 Step 3 – Configure stack options + +1. Leave the default options unchanged. +2. Acknowledge that AWS CloudFormation might create IAM resources with custom names by selecting the required checkbox. + +![Step 3](images/cloudtrail-step-7.png) + +3. Click **Next**. + +--- + +#### 2.4 Step 4 – Review + +1. Review all settings and confirm that all required fields are correctly populated. + +![Step 4](images/cloudtrail-step-8.png) + +2. Click **Submit** to create the stack. + +Monitor the stack creation: + +1. In **CloudFormation → Stacks → Events**, monitor the progress status. +2. When the status indicates completion, verify in the left panel that the stack has been successfully created. + +![Progress of creation](images/cloudtrail-step-9.png) + +--- + +### 3. Export CloudTrail logs + +1. Go to the **CloudTrail** dashboard and create a trail (if one does not already exist). + +![CloudTrail Service](images/cloudtrail-step-14.png) + +2. Configure the trail: + + - **Trail name**: Enter the trail name that you used in the CloudFormation stack in the previous steps (for example, `management-events`). + - **Enable for all accounts in organization**: Optional, if there are other accounts in the organization. + - **Storage location**: Choose **Use existing S3 bucket** and enter the **ARN** of the S3 bucket that was created by CloudFormation. + - **Log file SSE-KMS encryption**: Disable. + + Click **Next**. + +![Create Trail](images/cloudtrail-step-15.png) + +3. **Choose log events** (minimum configuration): + + - **Management events**: Enabled. + - **API activity**: **Read** and **Write**. + + Click **Next**. + +![Events logs](images/cloudtrail-step-16.png) + +4. Click **Create trail**. + +![logging events](images/cloudtrail-step-17.png) diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-1.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-1.png new file mode 100644 index 0000000000000..1063630076f13 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-1.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-14.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-14.png new file mode 100644 index 0000000000000..a3d98d7c9e97f Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-14.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-15.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-15.png new file mode 100644 index 0000000000000..82422a2d0f939 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-15.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-16.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-16.png new file mode 100644 index 0000000000000..22d5d05bfaee6 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-16.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-17.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-17.png new file mode 100644 index 0000000000000..30669358cb33b Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-17.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-2.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-2.png new file mode 100644 index 0000000000000..d179c4271e450 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-2.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-3.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-3.png new file mode 100644 index 0000000000000..1d63b2dab74a0 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-3.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-4.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-4.png new file mode 100644 index 0000000000000..443d141c7619f Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-4.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-5.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-5.png new file mode 100644 index 0000000000000..7eb01b73150fb Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-5.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-6.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-6.png new file mode 100644 index 0000000000000..3287784d95a83 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-6.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-7.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-7.png new file mode 100644 index 0000000000000..c6ca36923f717 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-7.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-8.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-8.png new file mode 100644 index 0000000000000..7c10517233f2b Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-8.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-9.png b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-9.png new file mode 100644 index 0000000000000..4f375d63ca53f Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudtrail/images/cloudtrail-step-9.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudtrail/template 2_AWS CloudTrail resources deployment.yaml b/articles/sentinel/aws-cloudformation/cloudtrail/template 2_AWS CloudTrail resources deployment.yaml new file mode 100644 index 0000000000000..5bd8c5d5d30d2 --- /dev/null +++ b/articles/sentinel/aws-cloudformation/cloudtrail/template 2_AWS CloudTrail resources deployment.yaml @@ -0,0 +1,184 @@ +AWSTemplateFormatVersion: "2010-09-09" +Description: > + Microsoft Sentinel integration for AWS CloudTrail logs. + Creates an OIDC IAM role, S3 bucket (optional), SQS queue and attaches S3/SQS + policies so Sentinel can read CloudTrail logs from S3 through SQS. + +Parameters: + AwsRoleName: + Type: String + AllowedPattern: OIDC_[-_a-zA-Z0-9]+ + Default: OIDC_MicrosoftSentinelS3CloudTrailRole + Description: Name of the OIDC role for Microsoft Sentinel. + + SentinelWorkspaceId: + Type: String + Description: Microsoft Sentinel Workspace ID (used in RoleSessionName condition). + + LogBucketName: + Type: String + Description: > + Name of the S3 bucket where CloudTrail will write logs and from which + Microsoft Sentinel will read. + + SentinelSQSQueueName: + Type: String + Description: > + Name of the SQS queue that will receive S3 event notifications for + CloudTrail log objects. + + CloudTrailTrailName: + Type: String + Default: "management-events" + Description: > + Name of the existing CloudTrail trail that delivers logs to the S3 bucket. + + LogFileSuffix: + Type: String + Default: ".gz" + Description: > + S3 object key suffix for CloudTrail log files used in the notification + filter (for example ".gz"). + + CreateNewBucket: + Type: String + AllowedValues: ["true", "false"] + Default: "true" + Description: > + Set to "true" to let this stack create and configure the S3 bucket. + Set to "false" if the bucket already exists and you will configure + bucket policy and event notifications manually. + +Conditions: + CreateNewBucketCondition: !Equals [ !Ref CreateNewBucket, "true" ] + +Resources: + + # IAM role assumed by Microsoft Sentinel via OIDC, with AWSCloudTrail_ReadOnlyAccess attached + SentinelWebIdentityBasedRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Ref AwsRoleName + AssumeRolePolicyDocument: !Sub > + {"Version":"2012-10-17","Statement":[{"Effect":"Allow", + "Principal":{"Federated":"arn:aws:iam::${AWS::AccountId}:oidc-provider/sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d/"}, + "Action":"sts:AssumeRoleWithWebIdentity", + "Condition":{"StringEquals":{ + "sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d/:aud":"api://1462b192-27f7-4cb9-8523-0f4ecb54b47e", + "sts:RoleSessionName":"MicrosoftSentinel_${SentinelWorkspaceId}"}}}]} + ManagedPolicyArns: + - !Sub arn:${AWS::Partition}:iam::aws:policy/AWSCloudTrail_ReadOnlyAccess + + # SQS queue that will receive notifications for new CloudTrail log objects + SentinelSQSQueue: + Type: AWS::SQS::Queue + Properties: + QueueName: !Ref SentinelSQSQueueName + + # SQS queue policy allowing S3 to send messages and Sentinel role to read them + SentinelSQSQueuePolicy: + Type: AWS::SQS::QueuePolicy + Properties: + Queues: + - !Ref SentinelSQSQueue + PolicyDocument: + Version: "2012-10-17" + Id: "SentinelSQSPolicy" + Statement: + - Sid: "AllowS3ToSendMessages" + Effect: Allow + Principal: "*" + Action: "SQS:SendMessage" + Resource: !GetAtt SentinelSQSQueue.Arn + Condition: + ArnLike: + aws:SourceArn: !Sub "arn:${AWS::Partition}:s3:::${LogBucketName}" + + - Sid: "AllowSentinelRoleToAccessQueue" + Effect: Allow + Principal: + AWS: !GetAtt SentinelWebIdentityBasedRole.Arn + Action: + - "SQS:ReceiveMessage" + - "SQS:DeleteMessage" + - "SQS:GetQueueAttributes" + - "SQS:GetQueueUrl" + - "SQS:ChangeMessageVisibility" + Resource: !GetAtt SentinelSQSQueue.Arn + + # S3 bucket to store CloudTrail logs (created only when CreateNewBucketCondition is true) + CloudTrailLogBucket: + Type: AWS::S3::Bucket + Condition: CreateNewBucketCondition + DependsOn: + - SentinelSQSQueuePolicy + Properties: + BucketName: !Ref LogBucketName + NotificationConfiguration: + QueueConfigurations: + - Event: s3:ObjectCreated:* + Queue: !GetAtt SentinelSQSQueue.Arn + Filter: + S3Key: + Rules: + - Name: suffix + Value: !Ref LogFileSuffix + + # S3 bucket policy for CloudTrail delivery and Sentinel read access + # (only created when the bucket is created by this stack) + CloudTrailAndSentinelBucketPolicy: + Type: AWS::S3::BucketPolicy + Condition: CreateNewBucketCondition + Properties: + Bucket: !Ref CloudTrailLogBucket + PolicyDocument: + Version: "2012-10-17" + Statement: + - Sid: "AWSCloudTrailAclCheck20150319-871196ad-f7d8-4b62-b73b-37787bb24763" + Effect: Allow + Principal: + Service: cloudtrail.amazonaws.com + Action: "s3:GetBucketAcl" + Resource: !Sub "arn:${AWS::Partition}:s3:::${LogBucketName}" + Condition: + StringEquals: + "AWS:SourceArn": !Sub "arn:${AWS::Partition}:cloudtrail:${AWS::Region}:${AWS::AccountId}:trail/${CloudTrailTrailName}" + + - Sid: "AWSCloudTrailWrite20150319-a5ca8e2f-ecf6-4ef1-8c54-d3caa01f7654" + Effect: Allow + Principal: + Service: cloudtrail.amazonaws.com + Action: "s3:PutObject" + Resource: !Sub "arn:${AWS::Partition}:s3:::${LogBucketName}/AWSLogs/${AWS::AccountId}/*" + Condition: + StringEquals: + "s3:x-amz-acl": "bucket-owner-full-control" + "AWS:SourceArn": !Sub "arn:${AWS::Partition}:cloudtrail:${AWS::Region}:${AWS::AccountId}:trail/${CloudTrailTrailName}" + + - Sid: "AllowSentinelRead" + Effect: Allow + Principal: + AWS: !GetAtt SentinelWebIdentityBasedRole.Arn + Action: + - "s3:GetObject" + - "s3:ListBucket" + Resource: + - !Sub "arn:${AWS::Partition}:s3:::${LogBucketName}" + - !Sub "arn:${AWS::Partition}:s3:::${LogBucketName}/*" + +Outputs: + SentinelRoleArn: + Description: ARN of the IAM role that Microsoft Sentinel should use. + Value: !GetAtt SentinelWebIdentityBasedRole.Arn + + LogBucketNameOutput: + Description: Name of the S3 bucket used for CloudTrail logs. + Value: !Ref LogBucketName + + SentinelSQSQueueURL: + Description: URL of the SQS queue to configure in the Microsoft Sentinel CloudTrail connector. + Value: !Ref SentinelSQSQueue + + SentinelSQSQueueArn: + Description: ARN of the SQS queue (for verification / troubleshooting). + Value: !GetAtt SentinelSQSQueue.Arn diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/aws-cloudformation-cloudwatch.md b/articles/sentinel/aws-cloudformation/cloudwatch/aws-cloudformation-cloudwatch.md new file mode 100644 index 0000000000000..dd4cf3b113f92 --- /dev/null +++ b/articles/sentinel/aws-cloudformation/cloudwatch/aws-cloudformation-cloudwatch.md @@ -0,0 +1,111 @@ +--- +title: Deploy AWS CloudWatch log collection for Microsoft Sentinel with CloudFormation +description: Use an AWS CloudFormation stack to export AWS CloudWatch logs to S3 and ingest them into Microsoft Sentinel with the AWS S3 connector. +author: KanenasCS +ms.author: bagol +ms.service: microsoft-sentinel +ms.topic: how-to +ms.collection: microsoft-sentinel +ms.date: 12/01/2025 +--- + +# Deploy AWS CloudWatch log collection for Microsoft Sentinel with CloudFormation + +### 1. Microsoft Sentinel configuration + +1. Sign in to the **Azure portal**. +2. Navigate to **Microsoft Sentinel** and then to **Content Hub**. +3. Install the connector **Amazon Web Services S3**. + +![Sentinel Security hub](images/cloudwatch-step-1.png) + +4. Navigate to **Data connectors** and open the **Amazon Web Services S3** connector page. + +--- + +### 2. Create the CloudFormation stack + +1. Sign in to the **AWS Management Console**. +2. In the search bar, search for **CloudFormation** and open the **CloudFormation** service. + +![CloudFormation Service](images/cloudwatch-step-2.png) + +3. Select **Create stack** → **With new resources (standard)**. + +![Create stack](images/cloudwatch-step-3.png) + +--- + +#### 2.1 Step 1 – Specify template + +1. Under **Prepare template**, select **Choose an existing template**. +2. Under **Template source**, select **Upload a template file**, then choose and upload the provided template file. +3. Click **Next**. + +![Step 1](images/cloudwatch-step-4.png) + +--- + +#### 2.2 Step 2 – Specify stack details + +##### 2.2.1 Template 2 + +Fill in the following parameters: + +- **Stack name**: Enter a name for the stack. +- **SentinelRoleName**: Enter the IAM role name (the name must start with `OIDC_XXXXX`). +- **CloudWatchBucketName**: Enter the name of the S3 bucket to be used. + - If you already have a generic S3 bucket or wish to use another existing bucket, enter its name here. +- **CloudWatchSQSQueueName**: Enter the name of the Amazon SQS queue. +- **SentinelWorkspaceId**: Enter the **Workspace ID** from the Azure Log Analytics workspace page: + - In the Azure portal, go to **Log Analytics workspace → Overview** and copy the **Workspace ID**. + +![Step 2](images/cloudwatch-step-5.png) + +After filling all required fields, click **Next**. + +![Step 2.1](images/cloudwatch-step-6.png) + +##### 2.2.2 Template 3 + +Fill in the following parameters: + +- **Stack name**: Enter a name for the stack. +- **EnvironmentName**: Enter the environment name used in resource names. +- **LambdaRoleName**: Enter the name of the IAM role used by the CloudWatch exporter Lambda. +- **ExporterLambdaName**: Enter the name of the CloudWatch export Lambda function. +- **LamdaScheduleMinutes**: How often (in minutes) to run the exporter Lambda (leave the default **15** minutes). +- **LambdaWindowMinutes**: How many minutes of logs to export each run (rolling window) (leave the default **15** minutes). + +![Step 2 from Template 3](images/cloudwatch-step-7.png) + +--- + +#### 2.3 Step 3 – Configure stack options + +1. Leave the default options unchanged. +2. Acknowledge that AWS CloudFormation might create IAM resources with custom names by selecting the required checkbox. + +![Step 3](images/cloudwatch-step-8.png) + +3. Click **Next**. + +--- + +#### 2.4 Step 4 – Review + +1. Review all settings and confirm that all required fields are correctly populated. + +![Step 4](images/cloudwatch-step-9.png) + +2. Click **Submit** to create the stack. + +Monitor the stack creation: + +1. In **CloudFormation → Stacks → Events**, monitor the progress status. + +![Progress stack](images/cloudwatch-step-10.png) + +2. When the status indicates completion, verify in the left panel that the stack has been successfully created. + +![Completed stack](images/cloudwatch-step-11.png) diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-1.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-1.png new file mode 100644 index 0000000000000..1063630076f13 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-1.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-10.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-10.png new file mode 100644 index 0000000000000..ba46fd0369c91 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-10.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-11.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-11.png new file mode 100644 index 0000000000000..e4fef41f919d5 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-11.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-2.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-2.png new file mode 100644 index 0000000000000..d179c4271e450 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-2.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-3.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-3.png new file mode 100644 index 0000000000000..1d63b2dab74a0 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-3.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-4.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-4.png new file mode 100644 index 0000000000000..443d141c7619f Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-4.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-5.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-5.png new file mode 100644 index 0000000000000..7eb01b73150fb Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-5.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-6.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-6.png new file mode 100644 index 0000000000000..e0b40096a83a4 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-6.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-7.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-7.png new file mode 100644 index 0000000000000..5acefbdbbbcd9 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-7.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-8.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-8.png new file mode 100644 index 0000000000000..c6ca36923f717 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-8.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-9.png b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-9.png new file mode 100644 index 0000000000000..7c10517233f2b Binary files /dev/null and b/articles/sentinel/aws-cloudformation/cloudwatch/images/cloudwatch-step-9.png differ diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/template 2_AWS CloudWatch resources deployment.yaml b/articles/sentinel/aws-cloudformation/cloudwatch/template 2_AWS CloudWatch resources deployment.yaml new file mode 100644 index 0000000000000..8d20363538b46 --- /dev/null +++ b/articles/sentinel/aws-cloudformation/cloudwatch/template 2_AWS CloudWatch resources deployment.yaml @@ -0,0 +1,146 @@ +AWSTemplateFormatVersion: '2010-09-09' +Description: > + Step 1 - Microsoft Sentinel CloudWatch integration: + OIDC IAM role, S3 bucket (+ policy + notification), and SQS queue (+ policy). + +Parameters: + CloudWatchBucketName: + Type: String + Description: Name of the S3 bucket where CloudWatch export .gz files for Sentinel will be stored. + + CloudWatchSQSQueueName: + Type: String + Description: Name of the SQS queue used by the Sentinel CloudWatch connector. + + SentinelRoleName: + Type: String + Description: Name of the IAM role to be used by Microsoft Sentinel (OIDC role). + + SentinelWorkspaceId: + Type: String + Description: Microsoft Sentinel Workspace ID (also shown as External ID in the connector). + +Resources: + + ####################################### + # OIDC IAM Role for Sentinel + ####################################### + + SentinelCloudWatchRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Ref SentinelRoleName + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Federated: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:oidc-provider/sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d/" + Action: "sts:AssumeRoleWithWebIdentity" + Condition: + StringEquals: + "sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d/:aud": "api://1462b192-27f7-4cb9-8523-0f4ecb54b47e" + "sts:RoleSessionName": !Sub "MicrosoftSentinel_${SentinelWorkspaceId}" + # Attach AWS managed CloudWatchReadOnlyAccess + ManagedPolicyArns: + - arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess + + ####################################### + # SQS queue + policy + ####################################### + + SentinelCloudWatchQueue: + Type: AWS::SQS::Queue + Properties: + QueueName: !Ref CloudWatchSQSQueueName + + SentinelCloudWatchQueuePolicy: + Type: AWS::SQS::QueuePolicy + Properties: + PolicyDocument: + Version: '2012-10-17' + Id: SentinelCloudWatchSQSPolicy + Statement: + # Allow S3 bucket to send messages (matches your example) + - Sid: AllowS3ToSendMessages + Effect: Allow + Principal: + Service: "s3.amazonaws.com" + Action: "SQS:SendMessage" + Resource: !GetAtt SentinelCloudWatchQueue.Arn + Condition: + ArnLike: + aws:SourceArn: !Sub "arn:${AWS::Partition}:s3:::${CloudWatchBucketName}" + # Allow Sentinel role to read from queue (matches your example) + - Sid: AllowSentinelRoleToAccessQueue + Effect: Allow + Principal: + AWS: !GetAtt SentinelCloudWatchRole.Arn + Action: + - "SQS:ReceiveMessage" + - "SQS:DeleteMessage" + - "SQS:ChangeMessageVisibility" + - "SQS:GetQueueAttributes" + Resource: !GetAtt SentinelCloudWatchQueue.Arn + Queues: + - !Ref SentinelCloudWatchQueue + + ####################################### + # S3 bucket + bucket policy + notification + ####################################### + + CloudWatchExportBucket: + Type: AWS::S3::Bucket + Properties: + BucketName: !Ref CloudWatchBucketName + NotificationConfiguration: + QueueConfigurations: + - Event: "s3:ObjectCreated:*" + Queue: !GetAtt SentinelCloudWatchQueue.Arn + # Like your console config: filter only on suffix ".gz" + Filter: + S3Key: + Rules: + - Name: suffix + Value: ".gz" + DependsOn: + - SentinelCloudWatchQueuePolicy + + CloudWatchExportBucketPolicy: + Type: AWS::S3::BucketPolicy + Properties: + Bucket: !Ref CloudWatchExportBucket + PolicyDocument: + Version: '2012-10-17' + Id: AWSLogDeliveryAndSentinelAccess + Statement: + # Same as your bucket policy example, parameterised + - Sid: AllowSentinelCloudWatchRoleList + Effect: Allow + Principal: + AWS: !GetAtt SentinelCloudWatchRole.Arn + Action: "s3:ListBucket" + Resource: !Sub "arn:${AWS::Partition}:s3:::${CloudWatchBucketName}" + - Sid: AllowSentinelCloudWatchRoleReadObjects + Effect: Allow + Principal: + AWS: !GetAtt SentinelCloudWatchRole.Arn + Action: "s3:GetObject" + Resource: !Sub "arn:${AWS::Partition}:s3:::${CloudWatchBucketName}/*" + +Outputs: + CloudWatchBucketNameOut: + Description: S3 bucket used for Sentinel CloudWatch logs + Value: !Ref CloudWatchBucketName + + CloudWatchQueueUrl: + Description: SQS queue URL used by Sentinel CloudWatch connector + Value: !Ref SentinelCloudWatchQueue + + CloudWatchQueueArn: + Description: ARN of the SQS queue + Value: !GetAtt SentinelCloudWatchQueue.Arn + + SentinelRoleArn: + Description: ARN of the IAM role that Sentinel must assume + Value: !GetAtt SentinelCloudWatchRole.Arn diff --git a/articles/sentinel/aws-cloudformation/cloudwatch/template 3_AWS CloudWatch resources deployment.yaml b/articles/sentinel/aws-cloudformation/cloudwatch/template 3_AWS CloudWatch resources deployment.yaml new file mode 100644 index 0000000000000..0ac65c42bba6f --- /dev/null +++ b/articles/sentinel/aws-cloudformation/cloudwatch/template 3_AWS CloudWatch resources deployment.yaml @@ -0,0 +1,290 @@ +AWSTemplateFormatVersion: '2010-09-09' +Description: > + Step 2 - Microsoft Sentinel CloudWatch integration: + Lambda exporter (CloudWatch -> S3) and EventBridge schedule. + +Parameters: + EnvironmentName: + Type: String + Description: Short environment name used in resource names (e.g. Omicron). + + CloudWatchBucketName: + Type: String + Description: Name of the S3 bucket where CloudWatch export .gz files for Sentinel are stored. + + LambdaRoleName: + Type: String + Description: Name of the IAM role used by the CloudWatch exporter Lambda. + + ExporterLambdaName: + Type: String + Description: Name of the CloudWatch export Lambda function. + + LambdaWindowMinutes: + Type: Number + Default: 15 + Description: How many minutes of logs to export each run (rolling window). + + LambdaScheduleMinutes: + Type: Number + Default: 15 + Description: How often (in minutes) to run the exporter Lambda. + +Resources: + + ####################################### + # Lambda execution role + ####################################### + + CloudWatchExporterLambdaRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Ref LambdaRoleName + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: "lambda.amazonaws.com" + Action: "sts:AssumeRole" + Policies: + - PolicyName: CloudWatchExporterPolicy + PolicyDocument: + Version: '2012-10-17' + Statement: + - Sid: CloudWatchRead + Effect: Allow + Action: + - "logs:DescribeLogGroups" + - "logs:DescribeLogStreams" + - "logs:GetLogEvents" + - "logs:FilterLogEvents" + Resource: "*" + - Sid: WriteToSentinelBucket + Effect: Allow + Action: + - "s3:PutObject" + - "s3:AbortMultipartUpload" + Resource: !Sub "arn:${AWS::Partition}:s3:::${CloudWatchBucketName}/AWSLogs/${AWS::AccountId}/CloudWatch/*" + - Sid: BasicLambdaLogging + Effect: Allow + Action: + - "logs:CreateLogGroup" + - "logs:CreateLogStream" + - "logs:PutLogEvents" + Resource: "*" + + ####################################### + # Lambda function (CloudWatch exporter) + ####################################### + + CloudWatchExporterLambda: + Type: AWS::Lambda::Function + Properties: + FunctionName: !Ref ExporterLambdaName + Runtime: python3.12 + Handler: index.lambda_handler + Role: !GetAtt CloudWatchExporterLambdaRole.Arn + Timeout: 120 + MemorySize: 256 + Environment: + Variables: + BUCKET_NAME: !Ref CloudWatchBucketName + BUCKET_PREFIX: !Sub "AWSLogs/${AWS::AccountId}/CloudWatch/" + WINDOW_MINUTES: !Ref LambdaWindowMinutes + Code: + ZipFile: | + import boto3 + import json + import os + from datetime import datetime, timedelta + import gzip + import io + import uuid + + # AWS clients + logs_client = boto3.client("logs") + s3_client = boto3.client("s3") + + # Environment variables (configure these in the Lambda console) + BUCKET_NAME = os.environ["BUCKET_NAME"] # e.g. azure-sentinel-cloudwatch-omicron-8134 + BUCKET_PREFIX = os.environ["BUCKET_PREFIX"] # e.g. AWSLogs/718905963082/CloudWatch/ + WINDOW_MINUTES = int(os.environ.get("WINDOW_MINUTES", "15")) # default: last 15 minutes + + + def _ms_to_iso8601(ms: int | None) -> str | None: + """ + Convert milliseconds since epoch to ISO8601 Zulu string, or None if ms is None. + Example: 2025-11-27T10:15:30.123Z + """ + if ms is None: + return None + dt = datetime.utcfromtimestamp(ms / 1000.0) + return dt.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z" + + + def lambda_handler(event, context): + # Rolling time window: last WINDOW_MINUTES minutes in UTC + end_dt = datetime.utcnow() + start_dt = end_dt - timedelta(minutes=WINDOW_MINUTES) + + start_ms = int(start_dt.timestamp() * 1000) + end_ms = int(end_dt.timestamp() * 1000) + + print( + f"Exporting CloudWatch logs from {start_dt} to {end_dt} UTC " + f"({start_ms}?{end_ms} ms)" + ) + + exported_objects = 0 + + try: + # List all log groups + lg_paginator = logs_client.get_paginator("describe_log_groups") + + for lg_page in lg_paginator.paginate(): + for lg in lg_page.get("logGroups", []): + log_group = lg["logGroupName"] + print(f"Processing log group: {log_group}") + + # List log streams in this group + ls_paginator = logs_client.get_paginator("describe_log_streams") + + for ls_page in ls_paginator.paginate( + logGroupName=log_group, + orderBy="LastEventTime", + descending=True, + ): + for ls in ls_page.get("logStreams", []): + log_stream = ls["logStreamName"] + + # Pull all events for this stream in the time window + next_token = None + all_events = [] + + while True: + kwargs = { + "logGroupName": log_group, + "logStreamName": log_stream, + "startTime": start_ms, + "endTime": end_ms, + "startFromHead": True, + } + if next_token: + kwargs["nextToken"] = next_token + + resp = logs_client.get_log_events(**kwargs) + events = resp.get("events", []) + all_events.extend(events) + + new_token = resp.get("nextForwardToken") + if not new_token or new_token == next_token: + break + next_token = new_token + + if not all_events: + # No events in this stream for the window + continue + + # Build gzipped NDJSON payload (one JSON object per line) + buf = io.BytesIO() + with gzip.GzipFile(fileobj=buf, mode="w") as gz: + for e in all_events: + ts_ms = e.get("timestamp") + ing_ms = e.get("ingestionTime") + + record = { + "timestamp": _ms_to_iso8601(ts_ms), + "ingestionTime": _ms_to_iso8601(ing_ms), + "message": e.get("message"), + "logGroup": log_group, + "logStream": log_stream, + } + gz.write((json.dumps(record) + "\n").encode("utf-8")) + + # Derive a safe filename from the log stream + file_name = log_stream + closing_bracket_index = file_name.find("]") + if closing_bracket_index != -1: + output_name = file_name[closing_bracket_index + 1 :] + else: + output_name = str(uuid.uuid4()).replace("-", "") + + # Clean log group name for folder use + safe_group = log_group.strip("/").replace("/", "_") + + # Final S3 key + key = ( + f"{BUCKET_PREFIX}" + f"{safe_group}/" + f"{output_name}_{start_ms}_{end_ms}.json.gz" + ) + + print( + f"Uploading {len(all_events)} events to " + f"s3://{BUCKET_NAME}/{key}" + ) + + s3_client.put_object( + Bucket=BUCKET_NAME, + Key=key, + Body=buf.getvalue(), + ContentType="application/json", + ContentEncoding="gzip", + ) + + exported_objects += 1 + + except Exception as e: + print(f"ERROR exporting logs: {e}") + raise + + print(f"Exported {exported_objects} S3 objects") + return { + "statusCode": 200, + "body": json.dumps({"exported_objects": exported_objects}) + } + + ####################################### + # EventBridge role + schedule + ####################################### + + EventInvokeLambdaRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Sub "${EnvironmentName}-CloudWatchExporterEventsRole" + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: "events.amazonaws.com" + Action: "sts:AssumeRole" + Policies: + - PolicyName: AllowEventsToInvokeLambda + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: "lambda:InvokeFunction" + Resource: !GetAtt CloudWatchExporterLambda.Arn + + CloudWatchExporterSchedule: + Type: AWS::Events::Rule + Properties: + Name: !Sub "${EnvironmentName}-CloudWatchExporter-Rate" + ScheduleExpression: !Sub "rate(${LambdaScheduleMinutes} minutes)" + State: ENABLED + Targets: + - Id: CloudWatchExporterLambdaTarget + Arn: !GetAtt CloudWatchExporterLambda.Arn + RoleArn: !GetAtt EventInvokeLambdaRole.Arn + +Outputs: + ExporterLambdaArn: + Description: ARN of the CloudWatch exporter Lambda + Value: !GetAtt CloudWatchExporterLambda.Arn + + LambdaRoleArn: + Description: ARN of the Lambda execution role + Value: !GetAtt CloudWatchExporterLambdaRole.Arn diff --git a/articles/sentinel/aws-cloudformation/flow-logs/aws-cloudformation-vpc-flow-logs.md b/articles/sentinel/aws-cloudformation/flow-logs/aws-cloudformation-vpc-flow-logs.md new file mode 100644 index 0000000000000..142a8fbe5cfa7 --- /dev/null +++ b/articles/sentinel/aws-cloudformation/flow-logs/aws-cloudformation-vpc-flow-logs.md @@ -0,0 +1,135 @@ +--- +title: Deploy AWS VPC Flow Logs collection for Microsoft Sentinel with CloudFormation +description: Use an AWS CloudFormation stack to configure AWS VPC Flow Logs to S3 and ingest them into Microsoft Sentinel with the AWS S3 connector. +author: KanenasCS +ms.author: bagol +ms.service: microsoft-sentinel +ms.topic: how-to +ms.collection: microsoft-sentinel +ms.date: 12/01/2025 +--- + +# Deploy AWS VPC Flow Logs collection for Microsoft Sentinel with CloudFormation + +### 1. Create the CloudFormation stack + +1. Sign in to the **AWS Management Console**. +2. In the search bar, search for **CloudFormation** and open the **CloudFormation** service. + +![ClouFormation service](images/flowlogs-step-1.png) + +3. Select **Create stack** → **With new resources (standard)**. + +(images/flowlogs-step-2.png) + +--- + +#### 1.1 Step 1 – Specify template + +1. Under **Prepare template**, select **Choose an existing template**. +2. Under **Template source**, select **Upload a template file**, then choose and upload the provided template file. +3. Click **Next**. + +![Step 1](images/flowlogs-step-3.png) + +--- + +#### 1.2 Step 2 – Specify stack details + +Fill in the following parameters: + +- **Stack name**: Enter a name for the stack. +- **AWSRoleName**: Enter the IAM role name (the name must start with `OIDC_XXXXX`). +- **BucketName**: Enter the name of the S3 bucket to be used. + - If you already have a generic S3 bucket or wish to use another existing bucket, enter its name here. +- **CreateNewBucket**: Set to `false` if you are using an existing S3 bucket (leave as `true` if a new bucket should be created). +- **FlowLogsPrefix (Optional)**: Optionally specify an S3 prefix for Flow Logs (must end with `/`). + - If left empty, the default `AWSLogs/${AWS::AccountId}/vpcflowlogs/` will be used. +- **SentinelSQSQueueName**: Enter the name of the Amazon SQS queue. +- **SentinelWorkspaceId**: Enter the **External ID** from the Azure connector page: + - In the Azure portal, go to **Microsoft Sentinel → Data connectors → open the relevant connector → expand _Setup with PowerShell script_** and copy the **External ID**. + +![Step 2](images/flowlogs-step-4.png) +![Step 2.1](images/flowlogs-step-5.png) + +After filling all required fields, click **Next**. + +![Step 2.2](images/flowlogs-step-6.png) + +--- + +#### 1.3 Step 3 – Configure stack options + +1. Leave the default options unchanged. +2. Acknowledge that AWS CloudFormation might create IAM resources with custom names by selecting the required checkbox. + +![Step 3](images/flowlogs-step-7.png) + +3. Click **Next**. + +--- + +#### 1.4 Step 4 – Review + +1. Review all settings and confirm that all required fields are correctly populated. + +![Step 4](images/flowlogs-step-8.png) +![Step 4.1](images/flowlogs-step-9.png) + +2. Click **Submit** to create the stack. + +Monitor the stack creation: + +1. In **CloudFormation → Stacks → Events**, monitor the progress status. + +![In progress stack](images/flowlogs-step-10.png) + +2. When the status indicates completion, verify in the left panel that the stack has been successfully created. + +![Completed stack](images/flowlogs-step-11.png) + +--- + +### 2. Create VPC Flow Logs + +1. In the AWS Management Console, search for and open the **VPC (Virtual Private Cloud)** service. + +![VPC service](images/flowlogs-step-12.png) + +2. In the left pane, select **Your VPCs**. + +![VPC flow logs ](images/flowlogs-step-13.png) + +3. Select the VPC for which you want to enable Flow Logs and open its details. + +![Selected VPC](images/flowlogs-step-14.png) + +4. Navigate to the **Flow logs** tab. + +![Flow logs tab](images/flowlogs-step-15.png) + +5. Click **Create flow log**. + +![create flow log](images/flowlogs-step-16.png) + +6. Under **Flow log settings**, configure the following: + + - **Name**: Enter a name for the flow log. + - **Filter**: Select one of the following (recommended: **All**): + - All + - Accept + - Reject + - **Maximum aggregation interval**: Choose **1 minute** or **10 minutes** (recommended: **1 minute**). + - **Destination**: Select **Send to an Amazon S3 bucket**. + - **S3 bucket ARN**: Enter the ARN of the S3 bucket created or specified in the previous steps. + - To find it: open the **S3** service → select the bucket → **Properties** tab → copy the **ARN**. + +![setup flow logs](images/flowlogs-step-17.png) + + - **Partition logs by time**: Choose **Every 1 hour** or **Every 24 hours** (recommended: **Every 1 hour**). + - Leave all other settings at their default values. + +![validate the settings](images/flowlogs-step-18.png) +![confirm the flow logs](images/flowlogs-step-19.png) + +7. Click **Create flow log**. diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-1.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-1.png new file mode 100644 index 0000000000000..d179c4271e450 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-1.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-10.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-10.png new file mode 100644 index 0000000000000..48df11f9bfb59 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-10.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-11.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-11.png new file mode 100644 index 0000000000000..824ce96e2bcbf Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-11.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-12.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-12.png new file mode 100644 index 0000000000000..dbc617310a776 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-12.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-13.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-13.png new file mode 100644 index 0000000000000..ce827058c3e4f Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-13.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-14.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-14.png new file mode 100644 index 0000000000000..68d88f7bc74f3 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-14.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-15.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-15.png new file mode 100644 index 0000000000000..7da553e6e93b6 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-15.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-16.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-16.png new file mode 100644 index 0000000000000..e1222a572c249 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-16.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-17.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-17.png new file mode 100644 index 0000000000000..071e8b6463f09 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-17.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-18.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-18.png new file mode 100644 index 0000000000000..704886ff07b3e Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-18.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-19.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-19.png new file mode 100644 index 0000000000000..e16989b9e07c6 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-19.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-2.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-2.png new file mode 100644 index 0000000000000..1d63b2dab74a0 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-2.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-3.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-3.png new file mode 100644 index 0000000000000..443d141c7619f Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-3.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-4.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-4.png new file mode 100644 index 0000000000000..53953613b2430 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-4.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-5.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-5.png new file mode 100644 index 0000000000000..a4c6575452907 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-5.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-6.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-6.png new file mode 100644 index 0000000000000..e4a52232dd780 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-6.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-7.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-7.png new file mode 100644 index 0000000000000..032753471a077 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-7.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-8.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-8.png new file mode 100644 index 0000000000000..b95bea55a393c Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-8.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-9.png b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-9.png new file mode 100644 index 0000000000000..cbf980008a8a2 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/flow-logs/images/flowlogs-step-9.png differ diff --git a/articles/sentinel/aws-cloudformation/flow-logs/template 2_AWS VPC Flow Logs resources deployment.yaml b/articles/sentinel/aws-cloudformation/flow-logs/template 2_AWS VPC Flow Logs resources deployment.yaml new file mode 100644 index 0000000000000..5987a6dee467a --- /dev/null +++ b/articles/sentinel/aws-cloudformation/flow-logs/template 2_AWS VPC Flow Logs resources deployment.yaml @@ -0,0 +1,161 @@ +AWSTemplateFormatVersion: "2010-09-09" +Description: > + Microsoft Sentinel integration for AWS VPC Flow Logs (manual activation). + Creates OIDC role, S3 bucket, SQS queue, and S3?SQS notifications. + +Parameters: + AwsRoleName: + Type: String + AllowedPattern: OIDC_[-_a-zA-Z0-9]+ + Default: OIDC_MicrosoftSentinelRole + Description: Role name for Sentinel OIDC + + BucketName: + Type: String + AllowedPattern: '^[a-z0-9][a-z0-9-.]{1,61}[a-z0-9]$' + Default: azure-sentinel-vpc-logs-omicron-8134 + Description: S3 bucket to store VPC Flow Logs + + SentinelSQSQueueName: + Type: String + Default: MicrosoftSentinelRnDFlowLogsSqs + Description: Exact SQS queue name (no suffix will be added) + + SentinelWorkspaceId: + Type: String + Description: Microsoft Sentinel Workspace ID (External ID) + + CreateNewBucket: + Type: String + AllowedValues: ["true", "false"] + Default: "true" + Description: Set to false if you do NOT want CloudFormation to create the bucket + + FlowLogsPrefix: + Type: String + Default: "" + Description: > + Optional S3 prefix for Flow Logs (must end with '/'). + If empty, AWSLogs/${AWS::AccountId}/vpcflowlogs/ will be used. + +Conditions: + CreateNewBucketCondition: !Equals [ !Ref CreateNewBucket, "true" ] + UseCustomPrefix: !Not [ !Equals [ !Ref FlowLogsPrefix, "" ] ] + +Resources: + + SentinelWebIdentityBasedRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Ref AwsRoleName + AssumeRolePolicyDocument: !Sub > + {"Version":"2012-10-17","Statement":[{"Effect":"Allow", + "Principal":{"Federated":"arn:aws:iam::${AWS::AccountId}:oidc-provider/sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d/"}, + "Action":"sts:AssumeRoleWithWebIdentity", + "Condition":{"StringEquals":{ + "sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d/:aud":"api://1462b192-27f7-4cb9-8523-0f4ecb54b47e", + "sts:RoleSessionName":"MicrosoftSentinel_${SentinelWorkspaceId}"}}}]} + # No inline policies ? you?ll grant access via S3/SQS policies + + SentinelSQSQueue: + Type: AWS::SQS::Queue + Properties: + QueueName: !Ref SentinelSQSQueueName + + SentinelSQSQueuePolicyForS3: + Type: AWS::SQS::QueuePolicy + Properties: + Queues: + - !Ref SentinelSQSQueue + PolicyDocument: + Version: "2012-10-17" + Statement: + - Sid: AllowS3Send + Effect: Allow + Principal: + Service: s3.amazonaws.com + Action: SQS:SendMessage + Resource: !GetAtt SentinelSQSQueue.Arn + Condition: + ArnLike: + aws:SourceArn: !Sub arn:${AWS::Partition}:s3:::${BucketName} + - Sid: AllowSentinel + Effect: Allow + Principal: + AWS: !GetAtt SentinelWebIdentityBasedRole.Arn + Action: + - sqs:ReceiveMessage + - sqs:DeleteMessage + - sqs:ChangeMessageVisibility + - sqs:GetQueueUrl + Resource: !GetAtt SentinelSQSQueue.Arn + + S3Bucket: + Type: AWS::S3::Bucket + Condition: CreateNewBucketCondition + DependsOn: + - SentinelSQSQueuePolicyForS3 + Properties: + BucketName: !Ref BucketName + NotificationConfiguration: + QueueConfigurations: + - Event: s3:ObjectCreated:* + Queue: !GetAtt SentinelSQSQueue.Arn + Filter: + S3Key: + Rules: + - Name: suffix + Value: ".gz" + + SampleBucketPolicy: + Type: AWS::S3::BucketPolicy + Condition: CreateNewBucketCondition + Properties: + Bucket: !Ref BucketName + PolicyDocument: + Version: "2012-10-17" + Id: AWSLogDeliveryAndSentinelAccess + Statement: + - Sid: AWSLogDeliveryWrite + Effect: Allow + Principal: + Service: delivery.logs.amazonaws.com + Action: s3:PutObject + Resource: !Sub arn:${AWS::Partition}:s3:::${BucketName}/AWSLogs/${AWS::AccountId}/* + Condition: + StringEquals: + s3:x-amz-acl: bucket-owner-full-control + aws:SourceAccount: !Ref AWS::AccountId + ArnLike: + aws:SourceArn: !Sub arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:* + - Sid: AWSLogDeliveryAclCheck + Effect: Allow + Principal: + Service: delivery.logs.amazonaws.com + Action: s3:GetBucketAcl + Resource: !Sub arn:${AWS::Partition}:s3:::${BucketName} + Condition: + StringEquals: + aws:SourceAccount: !Ref AWS::AccountId + ArnLike: + aws:SourceArn: !Sub arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:* + - Sid: AllowSentinelRead + Effect: Allow + Principal: + AWS: !GetAtt SentinelWebIdentityBasedRole.Arn + Action: + - s3:GetObject + - s3:ListBucket + Resource: + - !Sub arn:${AWS::Partition}:s3:::${BucketName} + - !Sub arn:${AWS::Partition}:s3:::${BucketName}/* + +Outputs: + SentinelRoleArn: + Value: !GetAtt SentinelWebIdentityBasedRole.Arn + SentinelSQSQueueURL: + Value: !Ref SentinelSQSQueue + SentinelSQSQueueArn: + Value: !GetAtt SentinelSQSQueue.Arn + LogBucketName: + Value: !Ref BucketName diff --git a/articles/sentinel/aws-cloudformation/guardduty/aws-cloudformation-guardduty.md b/articles/sentinel/aws-cloudformation/guardduty/aws-cloudformation-guardduty.md new file mode 100644 index 0000000000000..b084fe35c5b5f --- /dev/null +++ b/articles/sentinel/aws-cloudformation/guardduty/aws-cloudformation-guardduty.md @@ -0,0 +1,100 @@ +--- +title: Deploy AWS GuardDuty log collection for Microsoft Sentinel with CloudFormation +description: Use an AWS CloudFormation stack to send encrypted AWS GuardDuty findings to S3 and ingest them into Microsoft Sentinel with the AWS S3 connector. +author: KanenasCS +ms.author: bagol +ms.service: microsoft-sentinel +ms.topic: how-to +ms.collection: microsoft-sentinel +ms.date: 12/01/2025 +--- + +# Deploy AWS GuardDuty log collection for Microsoft Sentinel with CloudFormation + +### 1. Create the CloudFormation stack + +1. Sign in to the **AWS Management Console**. +2. In the search bar, search for **CloudFormation** and open the **CloudFormation** service. + +![CloudFormation Service](images/guardduty-step-1.png) + +3. Select **Create stack** → **With new resources (standard)**. + +![Create stack](images/guardduty-step-2.png) + +--- + +#### 1.1 Step 1 – Specify template + +1. Under **Prepare template**, select **Choose an existing template**. +2. Under **Template source**, select **Upload a template file**, then choose and upload the provided template file. +3. Click **Next**. + +![Step 1](images/guardduty-step-3.png) + +--- + +#### 1.2 Step 2 – Specify stack details + +Fill in the following parameters: + +- **Stack name**: Enter a name for the stack. +- **AWSRoleName**: Enter the IAM role name (the name must start with `OIDC_XXXXX`). +- **GuardDutyBucketName**: Enter the name of the S3 bucket to be used. + - If you already have a generic S3 bucket or wish to use another existing bucket, enter its name here. +- **BucketName**: Set to `false` if you are using an existing S3 bucket (leave as `true` if a new bucket should be created). +- **GuardDutyKmsAliasName**: Alias name (without the `alias/` prefix) for the new KMS key that will encrypt GuardDuty findings. +- **SentinelSQSQueueName**: Enter the name of the Amazon SQS queue. +- **LogFileSuffix**: S3 object key suffix for GuardDuty exported findings used in the notification filter (must be `.gz` by default). +- **SentinelWorkspaceId**: Enter the **Workspace ID** from the Azure Log Analytics workspace page: + - In the Azure portal, go to **Log Analytics workspace → Overview** and copy the **Workspace ID**. + +![Step 2](images/guardduty-step-4.png) + +After filling all required fields, click **Next**. + +![Step 2.1](images/guardduty-step-5.png) + +--- + +#### 1.3 Step 3 – Configure stack options + +1. Leave the default options unchanged. +2. Acknowledge that AWS CloudFormation might create IAM resources with custom names by selecting the required checkbox. + +![Step 3](images/guardduty-step-6.png) + +3. Click **Next**. + +--- + +#### 1.4 Step 4 – Review + +1. Review all settings and confirm that all required fields are correctly populated. + +![Step 4](images/guardduty-step-7.png) + +2. Click **Submit** to create the stack. + +Monitor the stack creation: + +1. In **CloudFormation → Stacks → Events**, monitor the progress status. +2. When the status indicates completion, verify in the left panel that the stack has been successfully created. + +![Progress of creation](images/guardduty-step-8.png) + +--- + +### 2. Export GuardDuty logs + +1. Go to the **GuardDuty** console and open **Settings**. + +![GuarDuty Configuration](images/guardduty-step-9.png) + +2. Under **Findings export options**, choose **Configure now** (or **Edit** if already configured). + +![Export logs](images/guardduty-step-10.png) + +3. Enter the **KMS key ARN** and **S3 bucket ARN**, then click **Save**. + +![Destination logs](images/guardduty-step-11.png) diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-1.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-1.png new file mode 100644 index 0000000000000..1063630076f13 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-1.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-10.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-10.png new file mode 100644 index 0000000000000..63c94be5ebe99 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-10.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-11.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-11.png new file mode 100644 index 0000000000000..02f3232d7ac2d Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-11.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-2.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-2.png new file mode 100644 index 0000000000000..d179c4271e450 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-2.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-3.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-3.png new file mode 100644 index 0000000000000..1d63b2dab74a0 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-3.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-4.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-4.png new file mode 100644 index 0000000000000..443d141c7619f Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-4.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-5.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-5.png new file mode 100644 index 0000000000000..7eb01b73150fb Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-5.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-6.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-6.png new file mode 100644 index 0000000000000..f75341f094f56 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-6.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-7.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-7.png new file mode 100644 index 0000000000000..c6ca36923f717 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-7.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-8.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-8.png new file mode 100644 index 0000000000000..7c10517233f2b Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-8.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-9.png b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-9.png new file mode 100644 index 0000000000000..b2bdf4cebf192 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/guardduty/images/guardduty-step-9.png differ diff --git a/articles/sentinel/aws-cloudformation/guardduty/template 2_AWS GuardDuty resources deployment.yaml b/articles/sentinel/aws-cloudformation/guardduty/template 2_AWS GuardDuty resources deployment.yaml new file mode 100644 index 0000000000000..3070435f59163 --- /dev/null +++ b/articles/sentinel/aws-cloudformation/guardduty/template 2_AWS GuardDuty resources deployment.yaml @@ -0,0 +1,243 @@ +AWSTemplateFormatVersion: '2010-09-09' +Description: | + Microsoft Sentinel integration for AWS GuardDuty findings. Creates an OIDC IAM role, KMS key + alias, S3 bucket, SQS queue and attaches S3/SQS policies so Sentinel can read GuardDuty findings from S3 through SQS. + +Parameters: + AwsRoleName: + Type: String + AllowedPattern: OIDC_[-_a-zA-Z0-9]+ + Default: OIDC_MicrosoftSentinelS3GuardDutyRole + Description: Name of the OIDC role for Microsoft Sentinel. + + SentinelWorkspaceId: + Type: String + Description: Microsoft Sentinel Workspace ID (used in RoleSessionName condition). + + GuardDutyBucketName: + Type: String + Description: | + Name of the S3 bucket where GuardDuty will write findings and from which Microsoft Sentinel will read. + + SentinelSQSQueueName: + Type: String + Description: | + Name of the SQS queue that will receive S3 event notifications for GuardDuty finding objects. + + LogFileSuffix: + Type: String + Default: .gz + Description: | + S3 object key suffix for GuardDuty exported findings used in the notification filter (for example ".gz" or ".json.gz"). + + GuardDutyKmsAliasName: + Type: String + Default: guardduty + Description: | + Alias name (without the 'alias/' prefix) for the new KMS key that will encrypt GuardDuty findings, for example "guardduty". + + CreateNewBucket: + Type: String + AllowedValues: + - 'true' + - 'false' + Default: 'true' + Description: | + Set to "true" to let this stack create and configure the S3 bucket. Set to "false" if the bucket already exists and you will configure bucket policy, encryption and event notifications manually. + +Conditions: + CreateNewBucketCondition: !Equals + - !Ref CreateNewBucket + - 'true' + +Resources: + + # IAM role assumed by Microsoft Sentinel via OIDC, + # with AmazonGuardDutyReadOnlyAccess attached + SentinelWebIdentityBasedRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Ref AwsRoleName + AssumeRolePolicyDocument: !Sub | + {"Version":"2012-10-17","Statement":[{"Effect":"Allow", "Principal":{"Federated":"arn:aws:iam::${AWS::AccountId}:oidc-provider/sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d/"}, "Action":"sts:AssumeRoleWithWebIdentity", "Condition":{"StringEquals":{ "sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d/:aud":"api://1462b192-27f7-4cb9-8523-0f4ecb54b47e", "sts:RoleSessionName":"MicrosoftSentinel_${SentinelWorkspaceId}"}}}]} + ManagedPolicyArns: + - !Sub arn:${AWS::Partition}:iam::aws:policy/AmazonGuardDutyReadOnlyAccess + + # KMS key used to encrypt GuardDuty findings in S3 + GuardDutyKmsKey: + Type: AWS::KMS::Key + Properties: + Description: KMS key for encrypting GuardDuty findings exported to S3 for + Microsoft Sentinel. + Enabled: true + KeyUsage: ENCRYPT_DECRYPT + KeySpec: SYMMETRIC_DEFAULT + MultiRegion: false + KeyPolicy: + Version: '2012-10-17' + Id: key-default-1 + Statement: + # Root account full permissions + - Sid: Enable IAM User Permissions + Effect: Allow + Principal: + AWS: !Sub arn:${AWS::Partition}:iam::${AWS::AccountId}:root + Action: kms:* + Resource: '*' + # GuardDuty Encrypt / GenerateDataKey / DescribeKey + - Sid: AllowGuardDutyEncryptAndGenerateDataKey + Effect: Allow + Principal: + Service: guardduty.amazonaws.com + Action: + - kms:Encrypt + - kms:GenerateDataKey* + - kms:DescribeKey + Resource: '*' + # Sentinel role Decrypt / DescribeKey + - Sid: AllowSentinelGuardDutyRoleDecrypt + Effect: Allow + Principal: + AWS: !GetAtt SentinelWebIdentityBasedRole.Arn + Action: + - kms:Decrypt + - kms:DescribeKey + Resource: '*' + + # Alias for the GuardDuty KMS key + GuardDutyKmsAlias: + Type: AWS::KMS::Alias + Properties: + AliasName: !Sub alias/${GuardDutyKmsAliasName} + TargetKeyId: !Ref GuardDutyKmsKey + + # SQS queue that will receive notifications for new GuardDuty S3 objects + SentinelSQSQueue: + Type: AWS::SQS::Queue + Properties: + QueueName: !Ref SentinelSQSQueueName + + # SQS queue policy allowing S3 to send messages and Sentinel role to read them + SentinelSQSQueuePolicy: + Type: AWS::SQS::QueuePolicy + Properties: + Queues: + - !Ref SentinelSQSQueue + PolicyDocument: + Version: '2012-10-17' + Id: SentinelSQSPolicy + Statement: + - Sid: AllowS3ToSendMessages + Effect: Allow + Principal: '*' + Action: SQS:SendMessage + Resource: !GetAtt SentinelSQSQueue.Arn + Condition: + ArnLike: + aws:SourceArn: !Sub arn:${AWS::Partition}:s3:::${GuardDutyBucketName} + + - Sid: AllowSentinelRoleToAccessQueue + Effect: Allow + Principal: + AWS: !GetAtt SentinelWebIdentityBasedRole.Arn + Action: + - SQS:ReceiveMessage + - SQS:DeleteMessage + - SQS:GetQueueAttributes + - SQS:GetQueueUrl + - SQS:ChangeMessageVisibility + Resource: !GetAtt SentinelSQSQueue.Arn + + # S3 bucket to store GuardDuty findings (created only when CreateNewBucketCondition is true) + GuardDutyBucket: + Type: AWS::S3::Bucket + Condition: CreateNewBucketCondition + DependsOn: + - SentinelSQSQueuePolicy + - GuardDutyKmsKey + Properties: + BucketName: !Ref GuardDutyBucketName + BucketEncryption: + ServerSideEncryptionConfiguration: + - ServerSideEncryptionByDefault: + SSEAlgorithm: aws:kms + KMSMasterKeyID: !Ref GuardDutyKmsKey + NotificationConfiguration: + QueueConfigurations: + - Event: s3:ObjectCreated:* + Queue: !GetAtt SentinelSQSQueue.Arn + Filter: + S3Key: + Rules: + - Name: suffix + Value: !Ref LogFileSuffix + + # S3 bucket policy for GuardDuty delivery and Sentinel read access + # (only created when the bucket is created by this stack) + GuardDutyBucketPolicy: + Type: AWS::S3::BucketPolicy + Condition: CreateNewBucketCondition + Properties: + Bucket: !Ref GuardDutyBucket + PolicyDocument: + Version: '2012-10-17' + Statement: + - Sid: AllowGuardDutyBucketLocation + Effect: Allow + Principal: + Service: guardduty.amazonaws.com + Action: s3:GetBucketLocation + Resource: !Sub arn:${AWS::Partition}:s3:::${GuardDutyBucketName} + Condition: + StringEquals: + aws:SourceAccount: !Ref AWS::AccountId + + - Sid: AllowGuardDutyWrite + Effect: Allow + Principal: + Service: guardduty.amazonaws.com + Action: s3:PutObject + Resource: !Sub arn:${AWS::Partition}:s3:::${GuardDutyBucketName}/* + Condition: + StringEquals: + s3:x-amz-acl: bucket-owner-full-control + aws:SourceAccount: !Ref AWS::AccountId + + - Sid: AllowSentinelListGuardDuty + Effect: Allow + Principal: + AWS: !GetAtt SentinelWebIdentityBasedRole.Arn + Action: s3:ListBucket + Resource: !Sub arn:${AWS::Partition}:s3:::${GuardDutyBucketName} + + - Sid: AllowSentinelReadGuardDuty + Effect: Allow + Principal: + AWS: !GetAtt SentinelWebIdentityBasedRole.Arn + Action: s3:GetObject + Resource: !Sub arn:${AWS::Partition}:s3:::${GuardDutyBucketName}/* + +Outputs: + SentinelRoleArn: + Description: ARN of the IAM role that Microsoft Sentinel should use. + Value: !GetAtt SentinelWebIdentityBasedRole.Arn + + GuardDutyBucketNameOutput: + Description: Name of the S3 bucket used for GuardDuty findings. + Value: !Ref GuardDutyBucketName + + GuardDutyKmsKeyArn: + Description: ARN of the KMS key used to encrypt GuardDuty findings. + Value: !GetAtt GuardDutyKmsKey.Arn + + GuardDutyKmsAliasNameOutput: + Description: Alias of the KMS key used to encrypt GuardDuty findings. + Value: !Sub alias/${GuardDutyKmsAliasName} + + SentinelSQSQueueURL: + Description: URL of the SQS queue to configure in the Microsoft Sentinel + GuardDuty connector. + Value: !Ref SentinelSQSQueue + + SentinelSQSQueueArn: + Description: ARN of the SQS queue (for verification / troubleshooting). + Value: !GetAtt SentinelSQSQueue.Arn \ No newline at end of file diff --git a/articles/sentinel/aws-cloudformation/openid/Template 1_ OpenID connect authentication deployment.json b/articles/sentinel/aws-cloudformation/openid/Template 1_ OpenID connect authentication deployment.json new file mode 100644 index 0000000000000..cc972649ec4e0 --- /dev/null +++ b/articles/sentinel/aws-cloudformation/openid/Template 1_ OpenID connect authentication deployment.json @@ -0,0 +1,19 @@ +{ + "AWSTemplateFormatVersion": "2010-09-09", + "Description": "This Stack creates an Open ID Connect (OIDC) web identity provider and an AWS assumed role.", + "Resources": { + "OIDCIdentityProvider": { + "Type": "AWS::IAM::OIDCProvider", + "DeletionPolicy": "Retain", + "Properties": { + "ClientIdList": [ + "api://1462b192-27f7-4cb9-8523-0f4ecb54b47e" + ], + "ThumbprintList": [ + "626D44E704D1CEABE3BF0D53397464AC8080142C" + ], + "Url": "https://sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d/" + } + } + } +} \ No newline at end of file diff --git a/articles/sentinel/aws-cloudformation/openid/aws-cloudformation-openid-connect.md b/articles/sentinel/aws-cloudformation/openid/aws-cloudformation-openid-connect.md new file mode 100644 index 0000000000000..edf5c6529022e --- /dev/null +++ b/articles/sentinel/aws-cloudformation/openid/aws-cloudformation-openid-connect.md @@ -0,0 +1,61 @@ +--- +title: Configure AWS OpenID Connect trust for the Microsoft Sentinel AWS S3 connector +description: Use an AWS CloudFormation stack to create the OpenID Connect identity provider and IAM role that let Microsoft Sentinel securely access AWS resources. +author: KanenasCS +ms.author: bagol +ms.service: microsoft-sentinel +ms.topic: how-to +ms.collection: microsoft-sentinel +ms.date: 12/01/2025 +--- + +# Configure AWS OpenID Connect trust for the Microsoft Sentinel AWS S3 connector + +### 3. Create the CloudFormation stack + +1. Sign in to the **AWS Management Console**. +2. In the search bar, type **CloudFormation** and open the **CloudFormation** service. +![CloudFormation service](images/openid-step-1.png) + +3. Select **Create stack** → **With new resources (standard)**. +![Create stack](images/openid-step-2.png) + +--- + +#### 3.1 Step 1 – Specify template + +1. Under **Prepare template**, select **Choose an existing template**. +2. Under **Template source**, select **Upload a template file**. +3. Click **Choose file** and upload the provided template file. +4. Click **Next**. +![Step 1](images/openid-step-3.png) +--- + +#### 3.2 Step 2 – Specify stack details + +1. **Stack name**: Enter a name for the stack (for example, `OpenID-Authentication`). +2. Leave all other parameters at their **default values**, unless your environment requires changes. +3. Click **Next**. +![Step 2](images/openid-step-4.png) +--- + +#### 3.3 Step 3 – Configure stack options + +1. Leave the default options unchanged. +2. In the **Capabilities** section, acknowledge that AWS CloudFormation might create IAM resources with custom names by selecting the required checkbox. +3. Click **Next**. +![Step 3](images/openid-step-5.png) +--- + +#### 3.4 Step 4 – Review + +1. Review all settings and confirm that all required fields are correctly populated. +2. Click **Submit** to create the stack. +![Step 4](images/openid-step-6.png) + + +After submission: + +1. Go to **CloudFormation → Stacks → Events** and monitor the stack creation progress. +2. When the status shows **CREATE_COMPLETE**, verify in the **Stacks** list that the stack has been successfully created. +![Completed the stack](images/openid-step-7.png) diff --git a/articles/sentinel/aws-cloudformation/openid/images/openid-step-1.png b/articles/sentinel/aws-cloudformation/openid/images/openid-step-1.png new file mode 100644 index 0000000000000..d179c4271e450 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/openid/images/openid-step-1.png differ diff --git a/articles/sentinel/aws-cloudformation/openid/images/openid-step-2.png b/articles/sentinel/aws-cloudformation/openid/images/openid-step-2.png new file mode 100644 index 0000000000000..443d141c7619f Binary files /dev/null and b/articles/sentinel/aws-cloudformation/openid/images/openid-step-2.png differ diff --git a/articles/sentinel/aws-cloudformation/openid/images/openid-step-3.png b/articles/sentinel/aws-cloudformation/openid/images/openid-step-3.png new file mode 100644 index 0000000000000..1076dba1fea2e Binary files /dev/null and b/articles/sentinel/aws-cloudformation/openid/images/openid-step-3.png differ diff --git a/articles/sentinel/aws-cloudformation/openid/images/openid-step-4.png b/articles/sentinel/aws-cloudformation/openid/images/openid-step-4.png new file mode 100644 index 0000000000000..b53ff2b1ec6f6 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/openid/images/openid-step-4.png differ diff --git a/articles/sentinel/aws-cloudformation/openid/images/openid-step-5.png b/articles/sentinel/aws-cloudformation/openid/images/openid-step-5.png new file mode 100644 index 0000000000000..c6ca36923f717 Binary files /dev/null and b/articles/sentinel/aws-cloudformation/openid/images/openid-step-5.png differ diff --git a/articles/sentinel/aws-cloudformation/openid/images/openid-step-6.png b/articles/sentinel/aws-cloudformation/openid/images/openid-step-6.png new file mode 100644 index 0000000000000..7c10517233f2b Binary files /dev/null and b/articles/sentinel/aws-cloudformation/openid/images/openid-step-6.png differ diff --git a/articles/sentinel/aws-cloudformation/openid/images/openid-step-7.png b/articles/sentinel/aws-cloudformation/openid/images/openid-step-7.png new file mode 100644 index 0000000000000..cea2ece99112f Binary files /dev/null and b/articles/sentinel/aws-cloudformation/openid/images/openid-step-7.png differ