CloudSense - Cloud-Based Real-Time IoT Monitoring System

CloudSense is a complete IoT monitoring system that collects sensor data from multiple ESP32 devices via MQTT, stores it in MongoDB Atlas (cloud database), and displays real-time updates on a React dashboard. The backend and MQTT broker are containerized with Docker, while the frontend is built and served directly on the host using Nginx. This project can be deployed to any cloud service (AWS, GCP, Azure, etc.) and is designed for college demonstration of cloud services, including the use of external cloud database services (MongoDB Atlas) and hybrid deployment patterns (containerized services + host-based services).

🏗️ Architecture

ESP32 Devices (NTP) → Mosquitto MQTT Broker (Container) → FastAPI Backend (Container) → MongoDB Atlas (Cloud Database)
                                                      ↓
                                              WebSocket → React Dashboard → Nginx (Host)

Deployment Model:

• Containerized Services: Mosquitto MQTT broker and FastAPI backend run in Docker containers
• Host-based Service: React frontend is built and served directly on the host using Nginx
• External Cloud Service: MongoDB Atlas provides the database as a managed cloud service, eliminating the need to run and maintain a local database instance

This hybrid approach demonstrates both containerization benefits and host-based service deployment patterns.

🚀 Tech Stack

• Backend: FastAPI (Python) - Containerized
• Frontend: React + Vite - Built and served on host via Nginx
• Database: MongoDB Atlas (Cloud Database Service)
• MQTT Broker: Mosquitto - Containerized
• Web Server: Nginx (installed on host)
• Containerization: Docker + Docker Compose (for backend and MQTT broker)
• Cloud: Any cloud provider (AWS EC2, GCP VM, Azure VM, etc.) with Ubuntu 22.04

📋 Prerequisites

• Docker and Docker Compose installed
• Nginx installed on host (for serving frontend)
• Node.js and npm installed (for building React frontend)
• Cloud VM instance (AWS EC2, GCP VM, Azure VM, etc.) with Ubuntu 22.04, or local machine for testing
• MongoDB Atlas account - Free tier available at mongodb.com/cloud/atlas
• ESP32 development boards with DHT22/DHT11 sensors
• Arduino IDE (for ESP32 firmware)
• For local development: uv package manager (optional but recommended) or Python 3.11+

🛠️ Setup Instructions

1. Clone and Navigate

cd /home/luffy/work/cc_project

2. Local Development Setup (Optional)

For local backend development with uv:

cd backend
./setup.sh  # Or manually: uv venv .venv --python 3.11 && source .venv/bin/activate && uv pip install -r requirements.txt
source .venv/bin/activate
uvicorn main:app --host 0.0.0.0 --port 8000 --reload

See backend/README.md for more details.

3. Set Up MongoDB Atlas (Cloud Database)

1. Create MongoDB Atlas Account:

   • Go to mongodb.com/cloud/atlas
   • Sign up for a free account (M0 Free Tier available)

2. Create a Cluster:

   • Choose your preferred cloud provider and region
   • Select M0 (Free) tier
   • Wait for cluster creation (2-3 minutes)

3. Configure Database Access:

   • Go to "Database Access" → "Add New Database User"
   • Create a username and password (save these securely)
   • Set user privileges to "Read and write to any database"

4. Configure Network Access:

   • Go to "Network Access" → "Add IP Address"
   • For development: Add 0.0.0.0/0 (allows from anywhere)
   • For production: Add your cloud VM's IP address only

5. Get Connection String:

   • Go to "Database" → "Connect" → "Connect your application"
   • Copy the connection string (format: mongodb+srv://<username>:<password>@cluster0.xxxxx.mongodb.net/?retryWrites=true&w=majority)

4. Configure Environment

Create a .env file in the backend/ directory:

MQTT_BROKER=mosquitto
MQTT_PORT=1883
MONGODB_URL=mongodb+srv://<username>:<password>@cluster0.xxxxx.mongodb.net/?retryWrites=true&w=majority
MONGODB_DB=iot_sensors

Important: Replace <username>, <password>, and the cluster URL with your actual MongoDB Atlas credentials.

5. Build Frontend

Build the React frontend on the host:

cd frontend
npm install
npm run build

This creates a dist/ directory with the production-ready frontend files.

6. Configure Nginx on Host

1. Install Nginx (if not already installed):

   sudo apt update
   sudo apt install nginx -y

2. Copy Nginx configuration:

   sudo cp nginx/nginx.conf /etc/nginx/sites-available/cloudsense
   sudo ln -s /etc/nginx/sites-available/cloudsense /etc/nginx/sites-enabled/

3. Update Nginx config to point to your frontend build:

   • Edit /etc/nginx/sites-available/cloudsense
   • Update the root directive to point to your frontend dist/ directory (e.g., /home/luffy/work/cc_project/frontend/dist)
   • Update the proxy_pass for backend API to http://localhost:8000

4. Test and reload Nginx:

   sudo nginx -t
   sudo systemctl reload nginx

7. Start Containerized Services

docker-compose up --build -d

This will start:

• Mosquitto MQTT broker (port 1883) - Containerized
• FastAPI backend (port 8000) - Containerized, connects to MongoDB Atlas

Note:

• MongoDB Atlas runs as a managed cloud service, so no local MongoDB container is needed
• Frontend is served directly by host Nginx (not containerized)

8. Access the Dashboard

Open your browser and navigate to:

• Local: http://localhost
• Cloud VM: http://<your-vm-ip> (replace with your cloud instance IP)

9. Configure ESP32 Devices

1. Open esp32_firmware/esp32_sensor.ino in Arduino IDE
2. Update WiFi credentials:

   const char* ssid = "YOUR_WIFI_SSID";
   const char* password = "YOUR_WIFI_PASSWORD";

3. Update MQTT broker IP (your cloud VM IP):

   const char* mqtt_server = "YOUR_CLOUD_VM_IP";

4. Set unique device ID for each ESP32:

   const char* device_id = "esp_01";  // Change for each device

5. Upload to ESP32

See esp32_firmware/README.md for detailed instructions.

📡 MQTT Topics

• Subscribe: sensors/+/data (wildcard for all devices)
• Publish: sensors/<device_id>/data

🔌 API Endpoints

REST APIs

• GET /api/data/latest?device_id=<device_id> - Get latest reading
• GET /api/data/history?device_id=<device_id>&limit=20 - Get historical data
• GET /api/devices - Get list of all devices

WebSocket

• WS /ws - Real-time sensor data stream

📊 Features

• ✅ Multi-device support
• ✅ Real-time data visualization with charts
• ✅ Historical data table (last 20 readings)
• ✅ Online/offline device status
• ✅ Auto-detection of new devices
• ✅ WebSocket reconnection with exponential backoff
• ✅ Responsive dashboard UI

🐳 Docker Services (Containerized)

Service	Container Name	Port	Description
Mosquitto	iot_mosquitto	1883	MQTT broker
FastAPI	iot_backend	8000	Backend API

🖥️ Host Services

Service	Location	Port	Description
Nginx	Host	80	Web server serving React frontend and proxying API requests

External Cloud Service:

• MongoDB Atlas - Managed cloud database (no local container needed)

📁 Project Structure

cc_project/
├── backend/              # FastAPI backend
│   ├── main.py          # Main application
│   ├── requirements.txt # Python dependencies
│   └── Dockerfile       # Backend container
├── frontend/            # React frontend
│   ├── src/
│   │   ├── components/  # React components
│   │   ├── App.jsx      # Main app component
│   │   └── main.jsx     # Entry point
│   ├── package.json     # Node dependencies
│   └── dist/            # Built frontend (generated after npm run build)
├── nginx/               # Nginx configuration for host
│   └── nginx.conf       # Nginx config (to be copied to /etc/nginx/sites-available/)
├── mosquitto/           # MQTT broker config
│   └── config/
│       └── mosquitto.conf
├── esp32_firmware/      # ESP32 Arduino code
│   ├── esp32_sensor.ino # Main firmware
│   └── README.md        # Firmware instructions
├── docker-compose.yml   # Docker orchestration
└── README.md           # This file

🔧 Troubleshooting

Containerized services not starting

# Check logs
docker-compose logs

# Restart services
docker-compose restart

# Rebuild containers
docker-compose up --build -d

Frontend not loading

1. Verify frontend is built: ls frontend/dist/
2. Check Nginx is running: sudo systemctl status nginx
3. Verify Nginx config: sudo nginx -t
4. Check Nginx error logs: sudo tail -f /var/log/nginx/error.log
5. Ensure Nginx config points to correct frontend dist/ directory
6. Rebuild frontend if needed: cd frontend && npm run build

ESP32 not connecting to MQTT

1. Verify MQTT broker IP is correct
2. Check firewall rules (port 1883 should be open)
3. Verify WiFi connection on ESP32
4. Check Serial Monitor for error messages

WebSocket connection issues

1. Verify host Nginx configuration includes WebSocket upgrade headers (check /etc/nginx/sites-available/cloudsense)
2. Check backend logs: docker-compose logs backend
3. Ensure frontend is using correct WebSocket URL
4. Reload Nginx after config changes: sudo systemctl reload nginx

Database connection errors

1. Verify MongoDB Atlas connection string in .env file (check username, password, and cluster URL)
2. Ensure MongoDB Atlas network access allows your IP address (check Network Access in Atlas dashboard)
3. Verify database user credentials are correct
4. Check backend logs: docker-compose logs backend
5. Test connection string format: mongodb+srv://username:password@cluster.mongodb.net/?retryWrites=true&w=majority

🌐 Cloud Deployment

This project can be deployed to any cloud service provider (AWS, GCP, Azure, DigitalOcean, etc.). The following example uses GCP, but similar steps apply to other providers:

1. Create Cloud VM:

   • Ubuntu 22.04
   • Allow HTTP traffic (port 80)
   • Allow custom TCP port 1883 (for MQTT)

2. SSH into VM:

   # GCP example:
   gcloud compute ssh <instance-name> --zone=<zone>
   
   # AWS example:
   ssh -i your-key.pem ubuntu@<your-ec2-ip>
   
   # Azure example:
   ssh azureuser@<your-vm-ip>

3. Install required software:

   sudo apt update
   sudo apt install docker.io docker-compose nginx nodejs npm -y
   sudo usermod -aG docker $USER
   # Log out and back in

4. Set up MongoDB Atlas:

   • Follow steps in "Set Up MongoDB Atlas (Cloud Database)" section above
   • Create .env file in backend/ directory with your MongoDB Atlas connection string
   • Ensure Network Access in Atlas allows your cloud VM's IP address

5. Clone repository:

   git clone <your-repo>
   cd cc_project

6. Build frontend:

   cd frontend
   npm install
   npm run build
   cd ..

7. Configure Nginx:

   sudo cp nginx/nginx.conf /etc/nginx/sites-available/cloudsense
   # Edit the config file to update paths:
   sudo nano /etc/nginx/sites-available/cloudsense
   # Update root path to: /home/<user>/cc_project/frontend/dist (or your actual path)
   sudo ln -s /etc/nginx/sites-available/cloudsense /etc/nginx/sites-enabled/
   sudo nginx -t
   sudo systemctl reload nginx

8. Start containerized services:

   docker-compose up --build -d

9. Configure firewall:

   • GCP: Use gcloud compute firewall-rules or Cloud Console
   • AWS: Configure Security Groups to allow ports 80 and 1883
   • Azure: Configure Network Security Groups (NSG) to allow ports 80 and 1883
   • Other providers: Follow their respective firewall/security group configuration

📝 Notes

• MQTT broker allows anonymous connections (for MVP)
• MongoDB Atlas provides managed cloud database - no local database container needed
• Data is stored in MongoDB Atlas cloud service, demonstrating external cloud service integration
• Hybrid deployment model: Backend and MQTT broker are containerized, while frontend is served directly on host via Nginx
• Frontend must be built manually using npm run build before deployment
• Nginx runs on the host (not in a container) and serves the built frontend static files
• ESP32 devices publish data every 5 seconds
• Device online status is based on last 10 seconds of activity

🎓 For College Project

This project is designed for college demonstration of cloud services and showcases:

• ✅ Cloud Deployment - Deployable to any cloud provider (AWS, GCP, Azure, etc.)
• ✅ External Cloud Database Service - MongoDB Atlas as a managed cloud database (demonstrates Database-as-a-Service)
• ✅ Hybrid Deployment Model - Combination of containerized services (backend, MQTT) and host-based services (frontend via Nginx)
• ✅ Containerization - Docker and Docker Compose for backend and MQTT broker orchestration
• ✅ Microservices Architecture - Modular, scalable service design
• ✅ Real-time Communication - WebSocket for live data streaming
• ✅ IoT Integration - MQTT protocol for sensor data collection
• ✅ Modern Web Stack - FastAPI backend with React frontend
• ✅ Reverse Proxy Pattern - Nginx for load balancing and routing
• ✅ Cloud Infrastructure - Demonstrates practical cloud service utilization and integration of multiple cloud services

📄 License

This project is for educational/demonstration purposes.

🤝 Contributing

This is a college project. Feel free to use and modify as needed.