Observability as Code – Datadog on AWS + AWX

This repo is a SANITIZED starter skeleton to deploy Datadog resources with Terraform, install agents with Ansible via AWX, and cover ECS EC2 and Fargate (sidecar + FireLens) patterns. Tuned for us-east-1, AWX org yondu.

I wrote / generated this for use in a corproate entrprise enviroment. Lots of details ahve been stripped out. I am just stashing here to save / reference for future endeavors

So, if you are going to use this - apply all the normal think before you act and replace the hardcoded stuff! -- gitobic

Contents

• terraform/ – Datadog monitors and ECS agent modules
• ansible/ – Playbooks for Linux/Windows agents, inventories, and a thin role wrapper
• awx/ – AWX org artifacts (Project, Inventories, Credentials, Job Templates)
• .gitlab-ci.yml – Example GitLab pipeline (Terraform + AWX canary/rollout)

Quick Start

1. Secrets
   • Store Datadog API/App keys in AWS Secrets Manager:
     • prod/observability/datadog_api_key
     • prod/observability/datadog_app_key
2. Terraform

   cd terraform/stacks/datadog
   terraform init -backend-config=../../envs/prod/backend.hcl
   terraform plan -var="env=prod" -var="aws_region=us-east-1"        -var="datadog_api_key=$(aws secretsmanager get-secret-value --region us-east-1 --secret-id prod/observability/datadog_api_key --query SecretString --output text)"        -var="datadog_app_key=$(aws secretsmanager get-secret-value --region us-east-1 --secret-id prod/observability/datadog_app_key --query SecretString --output text)"

3. AWX
   • Sync the Observability Project.
   • Import YAMLs under awx/ or create equivalent resources in UI.
   • Launch Datadog – Linux canary with a limit (tag_Canary_true) before full rollout.

Tree Layout

.
├── ansible
│   ├── collections
│   │   └── collections.txt
│   ├── inventories
│   │   ├── aws.yaml
│   │   ├── group_vars
│   │   │   ├── all.yaml
│   │   │   ├── linux.yaml
│   │   │   └── windows.yaml
│   │   └── vmware.ini
│   ├── playbooks
│   │   ├── linux_datadog.yaml
│   │   ├── templates
│   │   │   └── windows
│   │   │       └── datadog.yaml.j2
│   │   └── windows_datadog.yaml
│   └── roles
│       └── datadog_agent
│           └── dd_wrapper
├── awx
│   ├── credentials.yaml
│   ├── inventories.yaml
│   ├── job_templates.yaml
│   └── projects.yaml
├── README.md
└── terraform
    ├── envs
    │   └── prod
    │       ├── backend.hcl
    │       └── tfvars.auto.tfvars
    ├── examples
    │   └── ecs-fargate-task.json
    ├── modules
    │   └── ecs_dd_agent_daemon
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    └── stacks
        └── datadog
            ├── monitors.tf
            ├── providers.tf
            └── vars.tf