Overhead camera tracking and analytics system for MIE444 autonomous rover competitions.
An instructor runs the Operator Panel (desktop app) to start/stop trials and log events. Students watch a live Dashboard (web browser) that updates in real time with their rover's path, distance, speed, and event timeline.
- System Overview
- Hardware Requirements
- Installation
- First-Time Setup – Camera Calibration
- Running the System
- Operator Panel Reference
- Student Dashboard Reference
- Configuration Reference
- Logging Modes – Auto vs Manual
- Trial Output Files
- Troubleshooting
- Project Structure
Overhead USB Camera
│
▼
Operator Panel ──────────────────────────────────────────┐
(operator_ui.py) │
• Live camera feed with green tracking box │
• Start / Stop trial │ writes to
• Log collisions (Class 1 / 2 / 3) ▼
• Log manual interventions trials/
• Auto or Manual-only logging modes └── RoverName_20260315_143022/
├── trajectory.csv
├── events.json
├── config_snapshot.yaml
└── recording.lock ← live indicator
│
▼
Student Dashboard
(dashboard.py)
• Live path map
• Distance / speed
• Event timeline
| Component | Recommendation |
|---|---|
| Overhead camera | Any USB webcam that can cover the full maze; 1080p preferred |
| Camera mount | ceiling mount directly above the maze centre |
| Computer | Any machine capable of running Python 3.10+ and OpenCV |
| Network | Both operator laptop and student display must be on the same network for dashboard sharing |
Maze dimensions configured in the system: 8 ft × 4 ft (2438.4 mm × 1219.2 mm)
git clone https://github.com/BidoCodeHub/RobotTracker.git
cd RobotTrackerpython3 -m venv .venv
source .venv/bin/activate # macOS / Linux
# .venv\Scripts\activate # Windowspip install -r requirements.txtpython3 -c "import cv2, streamlit, PIL; print('OK')"The system maps pixel positions from the camera to real-world millimetre coordinates using a homography matrix. You must calibrate once whenever the camera is moved.
python3 main.py --calibrateA window will open showing the live camera feed. Click the four corners of the maze in this exact order:
1 ── top-left 2 ── top-right
│ │
4 ── bottom-left 3 ── bottom-right
The calibration tool will print four pixel coordinate pairs to the terminal and write them to config.yaml under homography.pixel_points. The world_points (real-world mm coordinates) are already set correctly for an 8 ft × 4 ft maze and do not need to change.
Open two terminal windows (or two tabs). Both must be in the project directory with the virtual environment activated.
streamlit run dashboard.pyStreamlit will print a local URL (e.g. http://localhost:8501). Open this URL in any browser. To share with students on the same network, use the Network URL printed below it.
The dashboard will display a "Waiting for trial…" message until the operator starts a trial.
python3 operator_ui.pyA desktop window will open – this is the instructor interface.
Follow the steps in the Operator Panel (details in the next section).
┌─────────────────────────────────┬──────────────────────┐
│ │ 🏁 RobotTracker │
│ LIVE FEED │ Operator Panel │
│ (green box = tracked rover) ├──────────────────────┤
│ │ VIDEO SOURCE │
│ │ ○ Live Camera │
│ │ ● Video File [Browse]│
│ │ [ Connect ] │
│ ├──────────────────────┤
│ │ ROVER / TEAM NAME │
│ │ [ TeamName_______ ] │
│ ├──────────────────────┤
│ │ 00:00.0 │
│ │ [ ▶ START TRIAL ] │
│ │ [ ⏹ STOP TRIAL ] │
│ ├──────────────────────┤
│ │ LOGGING MODE │
│ │ Collisions: Auto │
│ │ Interventions: Auto │
│ ├──────────────────────┤
│ │ LOG COLLISION │
│ │ [Class1][Class2][C3] │
│ │ [🖐 MANUAL INTERV. ] │
│ ├──────────────────────┤
│ │ EVENT LOG │
│ │ (scrollable list) │
└─────────────────────────────────┴──────────────────────┘
| Step | Action |
|---|---|
| 1 | Select Live Camera or Video File, then click Connect |
| 2 | Wait for the live feed to appear in the left panel |
| 3 | Type the rover/team name in the ROVER / TEAM NAME field |
| 4 | Click ▶ START TRIAL – the timer starts, recording begins |
| 5 | Monitor the green bounding box in the feed; log collisions/interventions as needed |
| 6 | Click ⏹ STOP TRIAL when the run is complete |
When using a video file: the video stays paused on the first frame after Connect. It only starts playing when you click START TRIAL. The trial ends automatically when the video finishes.
Click the appropriate button when the rover hits a wall:
| Button | Colour | Severity |
|---|---|---|
| Class 1 | Orange | Minor graze – rover self-recovers immediately |
| Class 2 | Red | Moderate impact – rover briefly stalls |
| Class 3 | Purple | Hard collision – rover gets stuck or requires recovery |
Click 🖐 MANUAL INTERVENTION any time the instructor physically touches or repositions the rover.
The student dashboard auto-updates every 2 seconds while a trial is live.
- Trial selector – automatically follows the live trial; after recording stops the last completed trial is shown. Students can also select any past trial from the dropdown.
- Map – full-width overhead view of the maze with the rover's trajectory drawn in blue. Collision and intervention markers are shown on the path.
- Speedometer – current instantaneous speed of the rover.
- Speed Over Time – chart of rover speed across the trial duration.
- Timeline of all logged events (auto-detected and manual) with timestamps.
| Metric | Description |
|---|---|
| Duration | Total elapsed time of the trial |
| Distance Travelled | Net path length in feet – jitter, rotation-in-place, and tracker glitches are all excluded (see Distance Filtering) |
| Top Speed | Peak speed recorded during the trial |
| Avg Speed | Mean speed across all frames |
| Collisions | Count of wall collision events |
| Interventions | Count of manual intervention events |
Raw per-frame position data contains two types of noise that would inflate the distance metric if accumulated directly:
| Noise Type | Cause | Effect without filtering |
|---|---|---|
| Jitter | Sub-pixel centroid noise from background subtraction | ~300–600 mm of fake distance per second at 30 fps |
| Rotation-in-place | Rover spinning without translating; centroid orbits the rover centre | Accumulates as apparent travel even though the rover hasn't moved |
Distance is computed in three layers – in the tracker, in the logger, and in the dashboard – all using the same approach:
- 1-second time-window chunking – all frames within each second are averaged into a single position. Random jitter and rotation both cancel out over ~1 second, leaving only genuine net displacement.
- 30 mm dead-band – chunk-to-chunk steps smaller than 30 mm are ignored (residual noise after averaging).
- 400 mm teleport cap – chunk-to-chunk steps larger than 400 mm are also ignored (tracker glitch / lost tracking).
The dashboard adds a 15-frame median pre-filter and uses 2-second chunks for an extra layer of smoothing.
All tuneable parameters live in config.yaml. Do not hardcode values in Python files.
maze:
length_ft: 8.0 # Physical maze length
width_ft: 4.0 # Physical maze width
homography:
pixel_points: # Four corners in pixel space – set by --calibrate
- [x1, y1] # top-left
- [x2, y2] # top-right
- [x3, y3] # bottom-right
- [x4, y4] # bottom-left
perception:
bgs_algorithm: MOG2 # Background subtractor: MOG2 or KNN
bgs_history: 200 # Frames used to build background model
bgs_var_threshold: 40 # Sensitivity – lower = more sensitive
min_contour_area_px: 500 # Minimum blob size to consider as rover (pixels²)
max_contour_area_px: 50000
max_jump_mm: 200 # Max allowed single-frame displacement (mm)
max_miss_frames: 10 # Frames lost before track is reset
position_smoothing_alpha: 0.4 # EMA smoothing – lower = smoother, more lag
max_velocity_mms: 1500.0 # Physical speed cap (~5 ft/s)
events:
collision_margin_mm: 50.0 # Distance from wall to trigger collision
collision_debounce_s: 1.0 # Minimum gap between auto-detected collisions
stop_velocity_threshold_mms: 5.0 # Speed below which rover is "stopped"
stop_min_duration_s: 1.0 # How long stopped before logging a stop event
data:
output_root: trials # Directory where trial folders are saved
trajectory_flush_every: 30 # Write CSV to disk every N frames (~1 second)
sensor:
source: file # "file" or "camera"
file_path: data_/WIN_20260225_15_07_32_Pro.mp4
camera_index: 0 # USB camera index (0 = first camera)- Rover not detected / box appears on wrong object → increase
min_contour_area_pxor check lighting and camera angle. - Tracker loses the rover mid-run → decrease
bgs_var_threshold(more sensitive) or increasemax_miss_frames. - Distance is higher than expected → decrease
position_smoothing_alphafor more aggressive EMA smoothing; the 1-second chunking + 30 mm dead-band automatically filters jitter and rotation-in-place. - Auto-collision events fire incorrectly → increase
collision_debounce_sor switch to Manual-only collision mode.
The Operator Panel offers two independent mode toggles:
| Toggle | Auto mode | Manual only mode |
|---|---|---|
| Collisions | System detects wall proximity and logs automatically | Only collisions you click with Class 1/2/3 buttons are recorded |
| Interventions | System detects sudden velocity jumps and logs automatically | Only interventions you click 🖐 are recorded |
These can be mixed (e.g. auto collisions + manual interventions). The mode can be changed at any time, including mid-trial.
Each trial produces a folder under trials/ named <RoverName>_<YYYYMMDD_HHMMSS>/:
trials/
└── TeamAlpha_20260315_143022/
├── trajectory.csv ← per-frame rover state
├── events.json ← timestamped event log
└── config_snapshot.yaml ← exact config used for this trial
| Column | Description |
|---|---|
frame_idx |
Frame number |
timestamp_s |
Elapsed time in seconds |
x_mm |
Rover X position (0 = left wall) |
y_mm |
Rover Y position (0 = top wall) |
velocity_mms |
Instantaneous speed (mm/s) |
heading_deg |
Heading angle (degrees, CCW from East) |
event_flags |
Bitmask of active event flags |
[
{
"event_type": "wall_collision",
"timestamp_s": 14.3,
"x_mm": 52.1,
"y_mm": 610.4,
"metadata": { "source": "manual", "class": "Class 2" }
}
]The tracker lost the rover. Common causes and fixes:
- Camera is moving or vibrating → secure the mount
- Lighting changed (e.g. someone walked past a window) → use consistent artificial lighting
- Rover moved very fast → tracker should recover within
max_miss_frames(default 10) frames bgs_historytoo low → the background model hasn't fully learned the static scene; increase to 300–500
- Check that the file path is correct and the video file exists
- Supported formats:
.mp4,.avi,.mov,.mkv
- Verify the correct
camera_indexinconfig.yaml(try 0, 1, 2) - On Linux: check
ls /dev/video*to find available camera devices - Ensure no other application is using the camera
- Confirm
streamlit run dashboard.pyis running in a separate terminal - The dashboard auto-refreshes every 2 seconds – wait a moment after starting a trial
- Ensure both terminals are pointing to the same project directory (same
trials/folder) - Stuck on a stale "live" trial – if the Operator Panel was force-quit mid-trial, a
recording.lockfile may have been left behind. The dashboard treats any lock file older than 30 seconds as stale and ignores it automatically. If the dashboard is still stuck, delete the lock file manually:rm trials/<trial_name>/recording.lock
- The system automatically filters out jitter and rotation-in-place using 1-second time-window chunking with a 30 mm dead-band (see Distance Filtering)
- If distance is still inflated, reduce
position_smoothing_alphainconfig.yaml(e.g. to 0.25) for more aggressive EMA smoothing on the raw centroid position
- Re-run
python3 main.py --calibrateand click the corners more precisely - Ensure the camera is as close to directly overhead as possible; oblique angles introduce perspective error that the homography partially but not fully corrects
robottracker/
├── operator_ui.py ← Instructor desktop app (run this to start)
├── dashboard.py ← Student web dashboard (streamlit)
├── main.py ← CLI entry point (calibration, headless processing)
├── config.yaml ← All configuration parameters
├── requirements.txt
│
├── rover_tracker/
│ ├── perception/
│ │ ├── tracker.py ← MOG2/KNN background subtraction, MeanShift tracking,
│ │ │ adaptive EMA smoothing; reference images pre-loaded at
│ │ │ startup; BGS model is fully reset between trials
│ │ ├── homography.py ← Pixel ↔ world coordinate transform
│ │ ├── background1new.jpg ← Reference background for initial detection
│ │ └── background_mat.jpg ← Maze boundary mask for initial detection
│ ├── events/
│ │ └── event_detector.py ← Collision, stop, intervention detection
│ ├── state/
│ │ └── rover_state.py ← RoverState dataclass; StateHistory with
│ │ noise-filtered total_distance_mm()
│ ├── data/
│ │ └── trial_logger.py ← CSV/JSON logging, recording.lock management;
│ │ distance computed with 1-second chunking
│ ├── visualization/
│ │ └── dashboard.py ← Streamlit dashboard; stale-lock detection;
│ │ 2-second chunking + dead-band distance filtering
│ └── sensor/ ← Camera / video file input abstraction
│
├── tests/ ← pytest unit tests (24 tests, all passing)
│ ├── test_event_detector.py
│ ├── test_homography.py
│ ├── test_state.py
│ └── test_trial_logger.py
│
├── data_/ ← Sample reference videos (not committed to git)
└── trials/ ← Trial output folders (not committed to git)
python3 -m pytest tests/ -vRobotTracker – MIE491 Capstone Design (built for MIE444 Mechatronics Engineering Design rover competitions)