Skip to content

Add sizing matrices and sar deployment option #501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 1, 2025
Merged

Add sizing matrices and sar deployment option #501

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 26 additions & 9 deletions docs/reference/edot-cloud-forwarder/aws.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
---

Check notice on line 1 in docs/reference/edot-cloud-forwarder/aws.md

View workflow job for this annotation

GitHub Actions / docs-preview / build 

The 'planned' lifecycle is deprecated and will be removed in a future release.
navigation_title: AWS
description: Set up the EDOT Cloud Forwarder for AWS to bring your AWS logs to Elastic Observability.
applies_to:
Expand All @@ -17,7 +17,7 @@

# EDOT Cloud Forwarder for AWS

{{edot-cf}} (CF) for AWS provides the EDOT Collector as a Lambda function that collects and forwards logs to {{product.observability}} on {{serverless-full}}.
{{edot-cf}} (CF) for AWS provides the EDOT Collector as a Lambda function that collects and forwards logs to {{motlp}}.

{{edot-cf}} for AWS supports the following log sources:

Expand Down Expand Up @@ -104,9 +104,7 @@
| --- | --- |
| CloudFormation (AWS CLI) | Deploy using AWS CLI commands with CloudFormation templates. |
| CloudFormation (AWS Console) | Deploy using the AWS Management Console UI. |
<!-- Not Public yet
| **AWS Serverless Application Repository (SAR)** | Deploy directly from the AWS Serverless Application Repository. |
-->
| AWS Serverless Application Repository (SAR) | Deploy directly from the AWS Serverless Application Repository. |

Each method achieves the same result and uses CloudFormation templates. Choose the method that best adapts to your workflow.

Expand Down Expand Up @@ -199,6 +197,27 @@
Adjust these parameters only if you notice performance issues such as Lambda timeouts, throttling, high memory usage or dropped data. If you need assistance tuning these parameters for your specific workload, refer to [Contact support](docs-content://troubleshoot/ingest/opentelemetry/contact-support.md).
:::

## Sizing and performance tuning

Use the following sizing matrices to select appropriate Lambda memory and concurrency settings for your traffic volume. Proper tuning helps maximize performance and prevent throttling in high‑volume log sources.

### VPC Flow Logs sizing

Check notice on line 204 in docs/reference/edot-cloud-forwarder/aws.md

View workflow job for this annotation

GitHub Actions / vale 

Elastic.Capitalization: 'VPC Flow Logs sizing' should use sentence-style capitalization.

| Throughput | Log Rate | Recommended Memory | Recommended Concurrency |
| :--- | :--- | :--- | :--- |
| **< 5 MB/s** | < 50,000 events/s | 512 MB | 5 |
| **5 - 10 MB/s** | 50,000 - 100,000 events/s | 512 MB | 10 |
| **> 10 MB/s** | > 100,000 events/s | 1024 MB | 20+ |

### ELB Access Logs sizing

Check notice on line 212 in docs/reference/edot-cloud-forwarder/aws.md

View workflow job for this annotation

GitHub Actions / vale 

Elastic.Capitalization: 'ELB Access Logs sizing' should use sentence-style capitalization.

| Throughput | Log Rate | Recommended Memory | Recommended Concurrency |
| :--- | :--- | :--- | :--- |
| **< 10 MB/s** | < 25,000 events/s | 512 MB | 5 |
| **10 - 40 MB/s** | 25,000 - 100,000 events/s | 1024 MB | 10 |
| **> 40 MB/s** | > 100,000 events/s | 2048 MB | 20+ |


## Deploy using CloudFormation (AWS CLI)

Use the AWS CLI to deploy the EDOT Cloud Forwarder with CloudFormation templates. This method is ideal for automation and infrastructure-as-code workflows.
Expand Down Expand Up @@ -462,7 +481,6 @@
7. Select **Submit** to apply the updates. In case of a change set, **Execute changeset** .
8. Monitor the stack update progress in the console.

<!-- Not public yet
## Deploy using AWS Serverless Application Repository

In addition to deploying through CloudFormation templates, you can deploy the EDOT Cloud Forwarder application directly from the AWS Serverless Application Repository (SAR).
Expand All @@ -472,9 +490,9 @@
To deploy from SAR, follow these steps:

1. Navigate to **AWS Serverless Application Repository** in the AWS Management Console.
2. Search for `edot-cloud-forwarder-s3-logs` and select the application.
3. Select **Deploy**.
4. **Configure the application settings**: You will be prompted to fill in the same parameters described in the [Configure the template](#configure-the-template) section. Refer to that section for details on each parameter.
2. Select **Available applications** and check the box **Show apps that create custom IAM roles or resource policies**.
3. Search for `edot-cloud-forwarder-s3-logs` and select the application.
4. **Configure the application settings**: Under **Application settings**, fill in the parameters described in the [Configure the template](#configure-the-template) section. Refer to that section for details on each parameter.
5. **Acknowledge IAM role creation**: At the bottom of the page, check the box to acknowledge that the application will create custom IAM roles. This is required for the forwarder to access your S3 bucket and send data to Elastic Observability.
6. Select **Deploy**.

Expand All @@ -483,7 +501,6 @@
:::{note}
The same [deployment considerations](#deployment-considerations) apply to SAR deployments, including the requirement to deploy separate serverless applications for each log type and ensure the deployment region matches your S3 bucket region.
:::
-->

## CloudFormation stack resources

Expand Down
Loading