ProctorAI

An AI-powered exam proctoring system that monitors students in real-time during online examinations. The system uses multiple computer vision models running entirely in the browser to detect suspicious activities such as phone usage, earphone presence, hand gestures, gaze deviation, face absence, and emotional anomalies.

Live Application: https://proctor-ai-one.vercel.app

Problem

With the rapid shift to online education, academic institutions face a major challenge in verifying student integrity during remote examinations. Traditional methods such as manual proctoring via Zoom are unscalable, prone to human error, and privacy-intrusive. There is a critical need for an automated system that can detect cheating behaviors in real-time without constant human supervision.

Solution

ProctorAI is a lightweight, client-side detection system that processes all video locally on the user's device. No video is uploaded to any server, ensuring student privacy. The system runs five independent AI detection modules simultaneously and provides instant visual feedback through a real-time monitoring dashboard.

Key Features

• Phone and Prohibited Object Detection -- Uses COCO-SSD (MobileNetV2-based CNN) to detect phones, books, laptops, and other unauthorized items in the webcam feed with bounding box visualization.
• Earphone Detection -- Identifies earphone/headphone usage through object detection inference, flagging potential audio-based cheating.
• Gaze Deviation Tracking -- Leverages MediaPipe Face Mesh (468 3D facial landmarks) to compute head pose estimation (yaw, pitch, roll) and detect when a student is looking away from the screen.
• Hand Gesture Detection -- Uses MediaPipe Hands to track hand movements and flag suspicious gestures such as signaling or reaching for hidden materials.
• Face Presence Monitoring -- Detects when a student leaves the frame or when multiple faces appear, indicating potential impersonation or external assistance.
• Emotion Analysis -- Runs facial expression classification using face-api.js (MobileNetV1 backbone) to identify signs of extreme anxiety, fear, or other anomalies that may indicate malpractice.
• Threat Level Indicator -- Aggregates alerts from all detection modules into a unified 5-level threat score displayed in the navigation bar.
• Activity Log -- Maintains a timestamped log of all detected events for post-exam review.
• Privacy-First Architecture -- All inference runs in the browser via TensorFlow.js and MediaPipe. No video data leaves the user's device.

Detection Modules

Module	Algorithm	Library
Object Detection	COCO-SSD (MobileNetV2-based CNN)	TensorFlow.js
Gaze Tracking	MediaPipe Face Mesh (Regression)	MediaPipe
Hand Gestures	MediaPipe Hands	MediaPipe
Face Presence	Face Mesh Confidence Score	MediaPipe
Emotion Analysis	MobileNetV1 CNN	face-api.js

Tech Stack

• Frontend: HTML, CSS, JavaScript
• AI/ML Inference: TensorFlow.js, COCO-SSD, face-api.js (vladmandic fork)
• Computer Vision: MediaPipe Face Mesh, MediaPipe Hands
• Fonts and Icons: Google Fonts (Inter, JetBrains Mono), Material Icons
• Deployment: Vercel

Project Structure

ProctorAI/
|-- index.html                  # Main application page
|-- style.css                   # Dashboard and UI styling
|-- js/
|   |-- main.js                 # Core application logic and orchestration
|   |-- detectors/
|       |-- object_detector.js  # Phone, earphone, and prohibited item detection
|       |-- gaze_tracker.js     # Gaze direction and head pose estimation
|       |-- hand_gesture_detector.js  # Hand tracking and gesture recognition
|       |-- face_monitor.js     # Face presence and multi-face detection
|       |-- emotion_detector.js # Facial expression classification
|-- models/
|   |-- yolo/                   # YOLO model weights (if applicable)
|-- ALGORITHMS.md               # Detailed algorithm documentation
|-- ALGORITHM_COMPARISON.md     # Comparison of detection approaches
|-- DATASET_ANALYSIS.md         # Dataset analysis notes
|-- TRAINING_GUIDE.md           # Model training instructions
|-- PROBLEM_STATEMENT.md        # Original problem statement
|-- PROJECT_JOURNEY.md          # Development timeline and decisions

How It Works

1. The user opens the application and grants webcam access.
2. Five AI models are loaded in parallel using TensorFlow.js and MediaPipe.
3. Each frame from the webcam is processed by all detection modules simultaneously.
4. Detections are displayed as real-time status indicators (Safe, Warning, Alert) with confidence bars.
5. Suspicious events are logged with timestamps in the Activity Log panel.
6. The Threat Level indicator aggregates all active alerts into a single score from 1 to 5.

Running Locally

1. Clone the repository:

   git clone https://github.com/jeeva5655/ProctorAI.git
   

2. Open index.html in a modern browser (Chrome or Edge recommended).

3. Allow webcam access when prompted.

No build step or server is required. The application runs entirely in the browser.

Deployment

The application is deployed on Vercel as a static site. To deploy your own instance:

1. Fork this repository.
2. Connect the repository to Vercel.
3. Deploy with default settings (no framework, no build command needed).

Algorithm Details

For detailed documentation on the algorithms used, see:

• ALGORITHMS.md -- Overview of all detection algorithms
• ALGORITHM_COMPARISON.md -- Comparison of different approaches evaluated
• TRAINING_GUIDE.md -- Guide for training custom models
• DATASET_ANALYSIS.md -- Analysis of datasets used

Browser Compatibility

• Google Chrome 90+ (recommended)
• Microsoft Edge 90+
• Firefox 85+ (partial MediaPipe support)

WebGL support is required for TensorFlow.js inference.

Author

Jeeva N -- GitHub | LinkedIn | Portfolio Dharshanraj

License

This project is open source and available for educational and research purposes.