Agentic AI for industrial equipment monitoring and maintenance — built on AWS.
Smart Machines is a full-stack demo that shows how AI agents, real-time telemetry, computer vision, and human-in-the-loop workflows combine to manage industrial fleets. It demonstrates how AWS services can be used to build intelligent monitoring and autonomous maintenance systems for physical equipment.
⚠️ This is a demo, not production-ready software. It is intended as a reference architecture to show how smart machines can be built and managed on AWS. Currently, only excavators are fully functional with simulated telemetry. Electric vehicles and robot arms are work in progress. Use this as a starting point and adapt it to your own use case.
A fleet of simulated excavators generates live telemetry. AI agents monitor the data, detect anomalies, diagnose root causes, and take action — from creating maintenance tickets to adjusting machine parameters autonomously. Operators can escalate to humans via voice/video, query a knowledge base for troubleshooting guidance, and watch live video feeds analyzed by AI.
| Section | Description |
|---|---|
| Demo Setup | Configure which machines are active, select a city for GPS simulation, start/stop telemetry |
| Dashboard | Live map with machine locations, real-time status indicators, click-to-inspect |
| Machines | Fleet table with dynamic telemetry columns, status filtering |
| Analytics | Time-series charts, fleet heatmap |
| AI Assistant | Multi-agent chat — diagnoses issues, reads telemetry, searches docs, creates tickets |
| Scenario Builder | Natural language telemetry overrides ("make Excavator-001 overheat") |
| Knowledge Base | Q&A over equipment manuals + document upload with sync |
| Alerts | AI-generated and human-reported alerts with threshold rules and ticket creation |
| Predictive ML | Reference architecture for classical ML on SageMaker (simulated) |
| Cloud Monitoring (KVS) | Live webcam → Bedrock AI analysis every 6 seconds, SNS alert rules |
| Edge Monitoring (NVIDIA) | Real-time YOLOWorld + VLM on NVIDIA Jetson, natural language detection rules |
| Ticketing | Maintenance work order management with AI-populated tickets |
| Amazon Connect | Human escalation via voice/chat when AI resolution fails (work in progress — not included in current release) |
| Excavator Control | Digital twin REST API for Unreal Engine integration |
L100 — Overview:
L300 — Deep Dive:
┌─────────────────────────────────────────────────────────────────────┐
│ CloudFront + S3 │
│ (React Frontend) │
└───────────────────────────────┬─────────────────────────────────────┘
│ WebSocket
┌───────────────────────────────▼─────────────────────────────────────┐
│ API Gateway (WebSocket) │
│ 9 routes: machine_data, assistant, scenario, ... │
└──┬──────────┬──────────┬──────────┬──────────┬──────────┬───────────┘
│ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼
Lambda Lambda Lambda Lambda Lambda
(SiteWise) (Bedrock) (Scenario) (KB) (ChatHist)
│ │ │ │
▼ ▼ ▼ ▼
SiteWise Bedrock S3 Overrides OpenSearch
(Telemetry) (Claude) + Step Fns Serverless
- Frontend: React, Cloudscape Design System, MapLibre GL
- Infrastructure: AWS CDK (TypeScript), 18+ stacks
- AI: Amazon Bedrock (Claude), Strands SDK multi-agent orchestration
- IoT: AWS IoT SiteWise (excavators + EVs)
- Video: Amazon KVS (WebRTC + PutMedia), NVIDIA Jetson (YOLOWorld + Moondream)
- Data: DynamoDB, S3, OpenSearch Serverless
# 1. Clone
git clone https://github.com/aws-samples/sample-smart-machines-physical-hybrid-ai.git
cd sample-smart-machines-physical-hybrid-ai
# 2. Deploy (requires AWS credentials + Docker/Finch)
cd infra/cdk && npm install
npx cdk deploy "dev/*" -c pipeline="" --method=direct
# 3. Create a user
./scripts/setup/create-user.sh --email you@example.com
# 4. Open the CloudFront URL from the deployment outputSee DEPLOYMENT.md for detailed instructions.
├── infra/cdk/ # All CDK infrastructure (18 stacks)
├── core-agents/ # Multi-agent AI platform (Strands + Bedrock)
├── kvs/ # KVS video streaming + AI analysis
├── edge/ # NVIDIA Jetson edge inference
├── knowledge-base/ # Equipment manuals + Bedrock KB
├── simulator/ # Telemetry simulation (Step Functions)
├── frontend/ # React app (Cloudscape UI)
├── scripts/setup/ # Post-deployment setup scripts
├── data/ # Sample data (support cases)
└── docs/ # Architecture docs and references
See CONTRIBUTING.md for reporting security issues.
This project is licensed under the MIT-0 License. See LICENSE.
See CONTRIBUTING.md.