🇬🇧 English | 🇹🇼 繁體中文

Server-API | FCM Notification Management Server| Violation Records Server| Data Management Server| Streaming-Web | Data-Augmentation | Evaluation | Train

"Construction-Hazard-Detection" is an AI-powered tool designed to enhance safety on construction sites. By leveraging the YOLO model for object detection, it identifies potential hazards such as:

• Workers without helmets
• Workers without safety vests
• Workers near machinery or vehicles
• Workers in restricted areas, restricted areas will be automatically generated by computing and clustering the coordinates of safety cones.

Post-processing algorithms further improve detection accuracy. The system is built for real-time deployment, offering instant analysis and alerts for identified hazards.

Additionally, the system integrates AI recognition results in real-time via a web interface. It can send notifications and real-time on-site images through messaging apps like LINE, Messenger, WeChat, and Telegram for prompt alerts and reminders. The system also supports multiple languages, enabling users to receive notifications and interact with the interface in their preferred language. Supported languages include:

• 🇹🇼 Traditional Chinese (Taiwan)
• 🇨🇳 Simplified Chinese (Mainland China)
• 🇫🇷 French
• 🇬🇧 English
• 🇹🇭 Thai
• 🇻🇳 Vietnamese
• 🇮🇩 Indonesian

This multi-language support makes the system accessible to a global audience, improving usability across different regions.

Contents

• Hazard Detection Examples
• Usage
• Database Configuration and Management
• Environment Variables
• Additional Information
• Dataset Information
• Contributing
• Development Roadmap
• License

Hazard Detection Examples

Below are examples of real-time hazard detection by the system:

Workers without Helmets or Safety Vests

Workers near Machinery or Vehicles

Workers in Restricted Areas

Usage

Before running the application, you need to configure the system by specifying the details of the video streams and other parameters in a JSON configuration file. An example configuration file config/configuration.json should look like this:

[
  {
    "video_url": "https://cctv1.kctmc.nat.gov.tw/6e559e58/",
    "site": "Kaohsiung",
    "stream_name": "Test",
    "model_key": "yolo11n",
    "notifications": {
      "line_token_1": "language_1",
      "line_token_2": "language_2"
    },
    "detect_with_server": true,
    "expire_date": "2024-12-31T23:59:59",
    "detection_items": {
      "detect_no_safety_vest_or_helmet": true,
      "detect_near_machinery_or_vehicle": true,
      "detect_in_restricted_area": true
    },
    "work_start_hour": 7,
    "work_end_hour": 18,
    "store_in_redis": true
  },
  {
    "video_url": "streaming URL",
    "site": "Factory_1",
    "stream_name": "camera_1",
    "model_key": "yolo11n",
    "notifications": {
      "line_token_3": "language_3",
      "line_token_4": "language_4"
    },
    "detect_with_server": false,
    "expire_date": "No Expire Date",
    "detection_items": {
      "detect_no_safety_vest_or_helmet": true,
      "detect_near_machinery_or_vehicle": false,
      "detect_in_restricted_area": true
    },
    "work_start_hour": 0,
    "work_end_hour": 24,
    "store_in_redis": true
  }
]

Each object in the array represents a video stream configuration with the following fields:

• video_url: The URL of the live video stream. This can include:

  • Surveillance streams
  • RTSP streams
  • Secondary streams
  • YouTube videos or live streams
  • Discord streams

• site: The location of the monitoring system (e.g., construction site, factory).

• stream_name: The name assigned to the camera or stream (e.g., "Front Gate", "Camera 1").

• model_key: The key identifier for the machine learning model to use (e.g., "yolo11n").

• notifications: A list of LINE messaging API tokens and corresponding languages for sending notifications.

  • line_token_1, line_token_2, etc.: These are the LINE API tokens.
  • language_1, language_2, etc.: The languages for the notifications (e.g., "en" for English, "zh-TW" for Traditional Chinese).

  Supported languages for notifications include:

  • zh-TW: Traditional Chinese
  • zh-CN: Simplified Chinese
  • en: English
  • fr: French
  • vi: Vietnamese
  • id: Indonesian
  • th: Thai

  For information on how to obtain a LINE token, please refer to line_notify_guide_en.

• detect_with_server: Boolean value indicating whether to run object detection using a server API. If True, the system will use the server for object detection. If False, object detection will run locally on the machine.

• expire_date: Expire date for the video stream configuration in ISO 8601 format (e.g., "2024-12-31T23:59:59"). If there is no expiration date, a string like "No Expire Date" can be used.

• detection_items: Specifies the safety detection items for monitoring specific scenarios. Each item can be set to True to enable or False to disable. The available detection items are:

  • detect_no_safety_vest_or_helmet: Detects if a person is not wearing a safety vest or helmet. This is essential for monitoring compliance with safety gear requirements on sites where such equipment is mandatory for personnel protection.
  • detect_near_machinery_or_vehicle: Detects if a person is dangerously close to machinery or vehicles. This helps prevent accidents caused by close proximity to heavy equipment or moving vehicles, often encountered in construction sites or industrial areas.
  • detect_in_restricted_area: Detects if a person has entered a restricted or controlled area. Restricted areas may be dangerous for untrained personnel or may contain sensitive equipment, so this setting aids in controlling access to such zones.

• work_start_hour: Specifies the hour (in 24-hour format) when the system should start monitoring the video stream. This allows monitoring to be restricted to active work hours, reducing unnecessary processing outside the defined timeframe (e.g., 7 for 7:00 AM).

• work_end_hour: Specifies the hour (in 24-hour format) when the system should stop monitoring the video stream. Monitoring will cease after this time to optimise resource usage (e.g., 18 for 6:00 PM).

  Together, work_start_hour and work_end_hour define the monitoring period during a day. For 24-hour monitoring, set work_start_hour to 0 and work_end_hour to 24.

• store_in_redis: A boolean value that determines whether to store processed frames and associated detection data in Redis. If True, the system will save the data to a Redis database for further use, such as real-time monitoring or integration with other services. If False, no data will be saved in Redis.

Database Configuration and Management

The Construction Hazard Detection system supports two operational modes:

1. JSON Configuration Mode: Uses a static JSON file for stream configurations
2. Database Mode: Dynamically loads and manages stream configurations from a MySQL database

Database Schema Overview

The system utilises a comprehensive database schema designed to manage users, sites, stream configurations, and violation records. Key tables include:

Core Tables

• sites: Manages construction site information
• stream_configs: Stores video stream configurations and detection settings
• violations: Records safety violation incidents with images and detection data
• users and user_profiles: User management and authentication
• group_info: Organises users into groups with permission controls
• features: Defines available system features

Stream Configuration Table Structure

The stream_configs table contains the following key fields:

CREATE TABLE stream_configs (
    id INT PRIMARY KEY AUTO_INCREMENT,
    group_id INT NOT NULL,
    site_id INT NOT NULL,
    stream_name VARCHAR(80) NOT NULL,
    video_url VARCHAR(255) NOT NULL,
    model_key VARCHAR(80) NOT NULL,
    detect_no_safety_vest_or_helmet BOOLEAN DEFAULT FALSE,
    detect_near_machinery_or_vehicle BOOLEAN DEFAULT FALSE,
    detect_in_restricted_area BOOLEAN DEFAULT FALSE,
    detect_in_utility_pole_restricted_area BOOLEAN DEFAULT FALSE,
    detect_machinery_close_to_pole BOOLEAN DEFAULT FALSE,
    detect_with_server BOOLEAN DEFAULT TRUE,
    expire_date DATETIME,
    work_start_hour INT,
    work_end_hour INT,
    store_in_redis BOOLEAN DEFAULT FALSE,
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

Running with Database Mode

When no --config parameter is provided, the system automatically enters database mode:

python main.py --poll 10

Database Mode Features

• Dynamic Configuration Loading: The system polls the database every --poll seconds (default: 10) for configuration changes
• Automatic Process Management: Automatically starts new stream processes when configurations are added
• Configuration Change Detection: Restarts stream processes when configurations are modified (based on updated_at timestamp)
• Expired Configuration Cleanup: Automatically stops processes for expired or deleted configurations
• Process Lifecycle Management: Graceful startup and shutdown of child processes

Database Connection Requirements

Ensure your .env file contains the correct database connection URL:

DATABASE_URL='mysql+asyncmy://username:password@localhost/construction_hazard_detection'

The system uses:

• asyncmy: Asynchronous MySQL driver for Python
• Connection Pooling: Maintains 1-5 concurrent database connections
• Automatic Reconnection: Handles database connection failures gracefully

JSON Configuration Mode vs Database Mode

Feature	JSON Mode	Database Mode
Configuration Source	Static JSON file	MySQL database
Dynamic Updates	Requires restart	Automatic polling
Process Management	Manual	Automatic
Multi-user Support	No	Yes (with user groups)
Violation Tracking	Limited	Full database logging
Configuration History	No	Yes (with timestamps)
Scalability	Limited	High

Database Initialisation

To set up the database schema, run the provided SQL script:

mysql -u root -p < scripts/init.sql

This script creates all necessary tables with proper relationships and constraints.

Working with Database Configurations

When using database mode, stream configurations are managed through the database rather than JSON files. The system automatically:

1. Polls the database for changes every few seconds
2. Compares configuration timestamps to detect modifications
3. Starts new stream processes for added configurations
4. Restarts processes when configurations are updated
5. Stops processes for expired or deleted configurations

This approach enables real-time configuration management without manual intervention.

Environment Variables

The application requires specific environment variables for proper configuration. These variables should be defined in a .env file located in the root directory of the project. Below is an example of the .env file:

# Database Configuration (Required for Database Mode)
DATABASE_URL='mysql+asyncmy://username:password@mysql/construction_hazard_detection'

# API Configuration
API_USERNAME='user'
API_PASSWORD='password'
DETECT_API_URL="http://yolo-server-api:6000/api/detect"
FCM_API_URL="http://fcm-server:8000/api/fcm"
VIOLATION_RECORD_API_URL="http://violation-server:8001/api/violations"
STREAMING_API_URL="http://streaming-server:8002/api/streaming_web"

# Redis Configuration (Optional - for streaming web functionality)
REDIS_HOST='redis'
REDIS_PORT=6379
REDIS_PASSWORD='password'

# LINE Messaging API (Optional - for notifications)
LINE_CHANNEL_ACCESS_TOKEN='YOUR_LINE_CHANNEL_ACCESS_TOKEN'

# Cloudinary Configuration (Optional - for image storage)
CLOUDINARY_CLOUD_NAME='YOUR_CLOUDINARY_CLOUD_NAME'
CLOUDINARY_API_KEY='YOUR_CLOUD_API_KEY'
CLOUDINARY_API_SECRET='YOUR_CLOUD_API_SECRET'

Variable Descriptions:

Database Configuration:

• DATABASE_URL: The connection URL for the MySQL database. Required for database mode operation. Format: mysql+asyncmy://username:password@host:port/database_name. The system uses this to:
  • Load stream configurations dynamically
  • Store violation records
  • Manage user authentication and permissions
  • Track configuration changes via timestamps

API Configuration:

• API_USERNAME: The username for authenticating with the detection API. Used by main.py.
• API_PASSWORD: The password for authenticating with the detection API. Used by main.py.
• DETECT_API_URL: The URL of the YOLO detection server API. Used for server-side object detection.
• FCM_API_URL: The URL of the FCM notification server API. Used for push notifications.
• VIOLATION_RECORD_API_URL: The URL of the violation recording server API. Used to store safety violations.
• STREAMING_API_URL: The URL of the streaming web server API. Used for real-time video streaming.

Redis Configuration (Optional):

• REDIS_HOST: The hostname for the Redis server. Required only if store_in_redis is enabled.
• REDIS_PORT: The port number for the Redis server. Default: 6379.
• REDIS_PASSWORD: The password for connecting to the Redis server. Leave empty if no authentication is required.

Notification Services (Optional):

• LINE_CHANNEL_ACCESS_TOKEN: The access token for the LINE Messaging API. Required for LINE notifications.
• CLOUDINARY_CLOUD_NAME: The Cloudinary cloud name for media management. Used for storing violation images.
• CLOUDINARY_API_KEY: The API key for accessing Cloudinary services.
• CLOUDINARY_API_SECRET: The API secret for accessing Cloudinary services.

Firebase Configuration (Optional):

• FIREBASE_CRED_PATH: Path to your Firebase Admin SDK credentials JSON file. Required for FCM notification management.
• FIREBASE_PROJECT_ID: Your Firebase project ID.

Configuration Modes:

For Database Mode (Recommended for Production):

# Minimal configuration for database mode
DATABASE_URL='mysql+asyncmy://user:pass@localhost/construction_hazard_detection'
DETECT_API_URL="http://localhost:6000/api/detect"

# Run with database configuration
python main.py --poll 10

For JSON Configuration Mode:

# No DATABASE_URL required for JSON mode
DETECT_API_URL="http://localhost:6000/api/detect"

# Run with JSON file
python main.py --config config/configuration.json

Note: Replace placeholder values with actual credentials and configuration details to ensure proper functionality. The DATABASE_URL is only required when running in database mode (without the --config parameter).

Now, you could launch the hazard-detection system in Docker or Python env:

Docker

Usage for Docker

To run the hazard detection system, you need to have Docker and Docker Compose installed on your machine. Follow these steps to get the system up and running:

1. Clone the repository to your local machine.

   git clone https://github.com/yihong1120/Construction-Hazard-Detection.git
   

2. Navigate to the cloned directory.

   cd Construction-Hazard-Detection
   

3. Build and run the services using Docker Compose:

   docker-compose build

4. To run the application, use the following command:

   docker-compose up

   You can view the detection results at http://localhost

5. To stop the services, use the following command:

   docker-compose down

Python

Usage for Python

To run the hazard detection system with Python, follow these steps:

1. Clone the Repository to Your Local Machine

Use the following command to clone the repository from GitHub:

git clone https://github.com/yihong1120/Construction-Hazard-Detection.git

---

2. Navigate to the Cloned Directory

Change the directory to the newly cloned repository:

cd Construction-Hazard-Detection

---

3. Install Required Packages

Run the following command to install the necessary Python packages:

pip install -r requirements.txt

---

4. Install and Launch MySQL Service (if Required)

For Ubuntu Users

1. Open the terminal and execute the following commands to install and start the MySQL server:

   sudo apt update
   sudo apt install mysql-server
   sudo systemctl start mysql.service

For Other Operating Systems

1. Download and install the appropriate version of MySQL for your operating system from the MySQL Downloads page.

---

Initialise the Database

After installing MySQL, use the following command to initialise the construction_hazard_detection database and create the users table:

mysql -u root -p < scripts/init.sql

You will be prompted to enter the MySQL root password. Ensure that the scripts/init.sql file contains the necessary SQL commands to set up the database and tables as described earlier.

---

5. Set Up Redis Server (Required Only for Streaming Web)

Redis is needed only when using the Streaming Web functionality. Follow the steps below to set up Redis.

For Ubuntu Users

1. Install Redis

   Open the terminal and run the following commands:

   sudo apt update
   sudo apt install redis-server

2. Configure Redis (Optional)

   If you need custom settings, edit the Redis configuration file:

   sudo vim /etc/redis/redis.conf

   To enhance security, enable password protection by adding or modifying the following line:

   requirepass YourStrongPassword
   

   Replace YourStrongPassword with a secure password.

3. Start and Enable Redis Service

   Start the Redis service:

   sudo systemctl start redis.service

   Enable Redis to start automatically on boot:

   sudo systemctl enable redis.service

For Other Operating Systems

Refer to the official Redis installation guide for instructions specific to your operating system.

---

6. Run the Object Detection API

Start the object detection API with the following command:

uvicorn examples.YOLO_server_api.backend.app:sio_app --host 0.0.0.0 --port 8001

---

7. Run the Main Application

The system supports two operation modes:

Option A: Database Mode (Recommended for Production)

Run the application using database configurations for dynamic stream management:

python3 main.py --poll 10

Database Mode Features:

• Dynamic configuration loading from MySQL database
• Automatic detection of configuration changes
• Real-time process management (start/stop/restart streams)
• Multi-user support with authentication
• Comprehensive violation logging

Parameters:

• --poll: Database polling interval in seconds (default: 10)

Option B: JSON Configuration Mode

Run the application using a static JSON configuration file:

python3 main.py --config config/configuration.json

JSON Mode Features:

• Simple setup with configuration file
• Suitable for single-user deployments
• Manual process management

Parameters:

• --config: Path to JSON configuration file

Note: Replace config/configuration.json with the actual path to your configuration file. If no --config parameter is provided, the system automatically switches to database mode.

---

8. Start the Streaming Web Service

Launching the Backend

For Linux Users

Run the following command to start the backend service on a Linux system:

uvicorn examples.streaming_web.backend.app:sio_app --host 127.0.0.1 --port 8002

For Windows Users

To start the backend service on a Windows system, use the following command:

waitress-serve --host=127.0.0.1 --port=8002 "examples.streaming_web.backend.app:streaming-web-app"

---

Setting Up the Frontend

Refer to the examples/YOLO_server_api/frontend/nginx.conf file for deployment instructions. Place the static web files in the following directory:

examples/YOLO_server_api/frontend/dist

Additional Information

• The system logs are available within the Docker container and can be accessed for debugging purposes.
• The output images with detections (if enabled) will be saved to the specified output path.
• Notifications will be sent through LINE messaging API during the specified hours if hazards are detected.

Notes

• Ensure that the Dockerfile is present in the root directory of the project and is properly configured as per your application's requirements.

For more information on Docker usage and commands, refer to the Docker documentation.

Dataset Information

The primary dataset for training this model is the Construction Site Safety Image Dataset from Roboflow. We have enriched this dataset with additional annotations and made it openly accessible on Roboflow. The enhanced dataset can be found here: Construction Hazard Detection on Roboflow. This dataset includes the following labels:

• 0: 'Hardhat'
• 1: 'Mask'
• 2: 'NO-Hardhat'
• 3: 'NO-Mask'
• 4: 'NO-Safety Vest'
• 5: 'Person'
• 6: 'Safety Cone'
• 7: 'Safety Vest'
• 8: 'Machinery'
• 9: 'Vehicle'

Models for detection

Model	size (pixels)	mAPval 50	mAPval 50-95	params (M)	FLOPs (B)
YOLO11n	640	58.0	34.2	2.6	6.5
YOLO11s	640	70.1	44.8	9.4	21.6
YOLO11m	640	73.3	42.6	20.1	68.0
YOLO11l	640	77.3	54.6	25.3	86.9
YOLO11x	640	82.0	61.7	56.9	194.9

Our comprehensive dataset ensures that the model is well-equipped to identify a wide range of potential hazards commonly found in construction environments.

TODO

• Add support for WhatsApp notifications.
• Correct the UI interface of examples/YOLO server_api/frontend mobile version

Contributing

We welcome contributions to this project. Please follow these steps:

1. Fork the repository.
2. Make your changes.
3. Submit a pull request with a clear description of your improvements.

License

This project is licensed under the AGPL-3.0 License.