Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 68 additions & 25 deletions src/content/docs/aws/tutorials/simulating-outages.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -55,28 +55,58 @@ The following diagram shows the architecture that this application builds and de

![Architecture](/images/aws/arch-1.png)

### Preflight checks

Before starting any outages, it's important to verify that our application is functioning correctly.
Start by creating an entity and saving it.
To do this, use curl to call the API Gateway endpoint for the POST method:
## Testing the application

Before simulating outages, verify that the application is working as expected:

```bash
curl --location 'http://12345.execute-api.localhost.localstack.cloud:4566/dev/productApi' \
--header 'Content-Type: application/json' \
--data '{
"id": "prod-2004",
"name": "Ultimate Gadget",
"price": "49.99",
"description": "The Ultimate Gadget is the perfect tool for tech enthusiasts looking for the next level in gadgetry.
Compact, powerful, and loaded with features."
}'
--header 'Content-Type: application/json' \
--data '{
"id": "prod-2004",
"name": "Ultimate Gadget",
"price": "49.99",
"description": "The Ultimate Gadget is the perfect tool for tech enthusiasts looking for the next level in gadgetry. Compact, powerful, and loaded with features."
}'
```

Expected output:

```bash title="Output"
Product added/updated successfully.
```

After ending the outage, confirm that previously failed items are stored successfully:

```bash
awslocal dynamodb scan --table-name Products
```

Expected output:

```json
{
"Items": [
{
"name": { "S": "Super Widget" },
"description": { "S": "A versatile widget that can be used for a variety of purposes. Durable, reliable, and affordable." },
"id": { "S": "prod-1003" },
"price": { "N": "29.99" }
},
{
"name": { "S": "Ultimate Gadget" },
"description": { "S": "The Ultimate Gadget is the perfect tool for tech enthusiasts looking for the next level in gadgetry. Compact, powerful, and loaded with features." },
"id": { "S": "prod-2004" },
"price": { "N": "49.99" }
}
],
"Count": 2,
"ScannedCount": 2,
"ConsumedCapacity": null
}
```

### Simulating the outage

Next, we will configure the Chaos API to target all DynamoDB operations.
Expand Down Expand Up @@ -197,32 +227,45 @@ Compact, powerful, and loaded with features."
}
```


## Conclusion

Simulating outages with the Chaos API helps uncover weaknesses in application error handling and data durability. To mitigate outages:

- Implement retry logic for failed operations
- Use queues (e.g., SQS) to buffer writes during downtime
- Employ Lambda functions to process and retry queued items
- Monitor system health and automate recovery actions

By proactively testing and designing for failure, you can build resilient cloud applications that gracefully handle disruptions and minimize data loss.

---

### Introducing network latency

The LocalStack Chaos API can also introduce a network latency for all connections.
This can be done with the following configuration:

```bash
curl --location --request POST 'http://localhost.localstack.cloud:4566/_localstack/chaos/effects' \
--header 'Content-Type: application/json' \
--data '{
"latency": 5000
}'
--header 'Content-Type: application/json' \
--data '{
"latency": 5000
}'
```

With this configured, you can use the same sample stack to observe and understand the effects of a 5-second delay on each service call.

```bash
curl --location 'http://12345.execute-api.localhost.localstack.cloud:4566/dev/productApi' \
--max-time 2 \
--header 'Content-Type: application/json' \
--data '{
"id": "prod-1088",
"name": "Super Widget",
"price": "29.99",
"description": "A versatile widget that can be used for a variety of purposes.
Durable, reliable, and affordable."
}'
--max-time 2 \
--header 'Content-Type: application/json' \
--data '{
"id": "prod-1088",
"name": "Super Widget",
"price": "29.99",
"description": "A versatile widget that can be used for a variety of purposes. Durable, reliable, and affordable."
}'
```

```bash title="Output"
Expand Down