v1.3
Solution Overview
Monitoring and Observability
Datadog is a SaaS-based monitoring and analytics platform for large-scale applications and infrastructure. Combining real-time logs, metrics from servers, containers, databases, and applications with end-to-end tracing, Datadog delivers actionable alerts and powerful visualizations to provide full-stack observability. Datadog includes over 400 vendor-supported integrations and APM libraries for several languages.
Customers running services on Outposts will have the need to not only monitor the health and performance of those services (eg. EKS clusters, EC2 instances, RDS databases etc.) but also the applications they run on them. Datadog can monitor technologies on Outposts, as well as customer's own on-prem infrastructure, and provide a single consolidated place to view and analyze the data from every part of their technology stack.
If yes provide a link to the marketplace offering
Yes, Datadog products can be found on the AWS Marketplace here
Yes, Datadog has an official integration with EKS. More information can be found on:
For EKS:
- EKS
- EC2
- Any other services being used, eg. ELB, AppMesh
- IAM
- CloudWatch
If Yes. Attach image to test plan.
Yes, EKS integration architecture diagram here
If so provide a brief description why and refer to test plan.
No, only one should be required.
Objectives:
- Create a basic EKS cluster
- Launch one EC2 instance as a bastion host used to install/configure the Datadog Agents
- Install Datadog Container Agent on EKS cluster containers
- Install Datadog node-based Agent on EC2 instances
Acceptance Criteria:
- Success is achieved when metrics and logs about the EKS environment are visible on Datadog's default dashboards
- 2 EC2 instances
- 1 IAM Role with policy that has the required permissions to query the CloudWatch API for metrics
- This is required for Datadog to query CloudWatch metrics
- 1 new VPC for the test EKS cluster
Create a basic EKS cluster, setup the Datadog Cluster Agent and datadog node-based Agent. Finally, verify metrics and logs are visible in Datadog.
1. Create a basic EKS cluster using eksctl
Use the included eks_cluster.yaml
file to create an EKS cluster.
eksctl create cluster -f eks_cluster.yaml
2. Configure RBAC permissions for the Cluster Agent and node-based Agents
Use the included cluster-agent-rbac.yaml
, datadog-rbac.yaml
to deploy
$ kubectl apply -f /path/to/cluster-agent-rbac.yaml
$ kubectl apply -f /path/to/datadog-rbac.yaml
3. Secure communication between node-based Agents and the Cluster Agent
Generate a secret token to use in the included dca-secret.yaml
file (replace <TOKEN>
value):
echo -n '<32_CHARACTER_LONG_STRING>' | base64
vi /path/to/dca-secret.yaml
Create the secret:
$ kubectl apply -f /path/to/dca-secret.yaml
4. Create and deploy the Cluster Agent manifest
Use the included datadog-cluster-agent.yaml
file (replace <YOUR_API_KEY>
with Datadog API key)
$ kubectl apply -f /path/to/datadog-cluster-agent.yaml
- Deploy the node-based Agent DaemonSet
Use the included
datadog-agent.yaml
file (replace<YOUR_API_KEY>
with Datadog API key)
$ kubectl apply -f /path/to/datadog-agent.yaml
5. Setup CloudWatch
-
Create a new role in the AWS IAM Console and attach a policy that has the required permissions to query the CloudWatch API for metrics
-
In Datadog, configure the AWS Integration Tile with:
- AWS Account ID
- Role Name
- Tick the boxes for: "EC2", "EC2 API", "EC2 Spot Fleet" and "EBS"
- Navigate to Datadog and verify dashboard contains EKS metrics