Open-Source Webcam Eye Tracking for Web Applications

[paq85]

Open-Source Webcam Eye Tracking for Web Applications

On-screen gaze prediction — solution survey, fact sheets, and recommendations

Report version 1.1 · 6 August 2026

---

About This Report

What this report contains: a survey of open-source solutions that can be used to build a web application with webcam-based on-screen gaze prediction. Every solution gets a uniform fact sheet covering: runtime (does it run in the web browser, on a server, or on the desktop), reported accuracy, what benchmarks were run, what data the model was trained on, whether it uses only the camera image as input, official research articles describing the solution, a few lines on how it is built and its unique properties, plus license, maintenance status, and links. It ends with a comparison matrix, recommendations (up to 3 best solutions per runtime/use case), reference architectures, licensing guidance, and risks. A glossary of the cited research articles follows the fact sheets (§4).

How the projects were validated: every entry was checked against its primary source on 2026-08-06 — the GitHub API (stars, SPDX license, last push, archived status), the npm/PyPI registries, official documentation, and the original papers. Claims that could not be verified are explicitly flagged. Accuracy figures are as reported by the authors on their own benchmarks and are not directly comparable across datasets (centimeters on mobile phones ≠ degrees on desktop ≠ pixels on a screen).

Version history: 1.0 (2026-08-06) initial survey · 1.1 (2026-08-06) fact-sheet format with runtime/accuracy/benchmarks/training-data/input fields, links for every solution, validation methodology, recommendations per runtime · 1.2 (2026-08-06) official research articles cited per solution + glossary of articles (§4); all arXiv IDs verified via the arXiv API.

---

1. Executive Summary

No single maintained, permissively-licensed, drop-in library gives research-grade webcam gaze in the browser today. The field splits into three practical strategies:

Strategy	Best open-source options	License	Effort
A. Turnkey browser library	WebEyeTrack (npm, TF.js, few-shot personalization)	MIT	Low
B. DIY browser stack (face/iris landmarks + your own calibration mapping)	MediaPipe Face Landmarker + custom regression — the architecture RealEye's open-source lib uses	Apache-2.0	Medium
C. Client capture + server inference	L2CS-Net family (PyTorch/ONNX), EyeTrax for calibration/smoothing	MIT	Medium-High

Key verdicts:

• WebGazer.js — the most famous library — is GPL-3.0 (not MIT, a common misconception) and its official maintenance ended February 2026. Its accuracy also degrades over time without head-pose handling (error grows from ~5 to ~10 cm during a 20-minute session, ETRA 2018). Prototypes only.
• WebEyeTrack (2025, Vanderbilt et al., MIT) is the most interesting modern turnkey option: browser-native CNN (BlazeGaze, 670 KB), MAML meta-learning + on-device few-shot calibration (<9 samples), 2.32 cm on GazeCapture, 2.4 ms inference on iPhone 14. Young project (npm 0.0.x) — validate before committing.
• MediaPipe Face Landmarker (Apache-2.0, Google-maintained) is the safest foundation: 478 landmarks including iris + head-pose transformation matrix, official Web (WASM) and Python APIs. You add the gaze mapping yourself (ridge regression / polynomial / small TF.js net) — exactly what RealEye's webcam-eyetracker-light-open does (17-point calibration, ≈120 CSS px).
• Server-side: L2CS-Net (MIT) is the accuracy reference for appearance-based gaze (3.92° MPIIGaze); yakhyo/gaze-estimation provides maintained ONNX exports (runnable in-browser via onnxruntime-web/WebGPU); EyeTrax (MIT) adds calibration routines and Kalman/EMA/KDE smoothing.
• EyeGestures (GPL-3.0, Rust-based engine, 20-point calibration) is the most complete calibrated browser gaze library besides RealEye's — great for prototypes/research, copyleft blocks commercial SaaS embedding.
• Licensing traps: research-only datasets (GazeCapture, MPIIGaze, EyeDiap) are used to train most pretrained models — verify weight provenance before commercial use; camgaze.js has no license file (legally unusable as-is).

---

2. How Webcam Gaze Prediction Works (30-second primer)

1. Capture — getUserMedia (requires HTTPS/localhost).
2. Face detection + eye/iris landmarks — geometric (Viola-Jones, pupil localization) or ML (MediaPipe FaceMesh, dlib 68-point, TF.js).
3. Feature extraction — iris center relative to eye corners, eye aspect ratio, head pose (yaw/pitch/roll). Head pose is the Bootstrap RealEye.io community repository with guidelines, knowledge base, FAQ, roadmap, and GitHub templates #1 accuracy killer; modern systems compensate for it.
4. Mapping to screen coordinates — calibration: user fixates known on-screen points (clicks, dots, smooth-pursuit targets); a regressor learns eye-features → screen-(x,y). Approaches: linear/polynomial regression, ridge regression (RealEye light-open), small neural net trained in-browser (HueVision), or few-shot personalization of a pretrained net (WebEyeTrack).
5. Post-processing — smoothing (Kalman/EMA), fixation/saccade classification, heatmap/gaze-plot rendering.

Two model families: geometric (pupil/iris geometry; cheap, needs good iris visibility) and appearance-based (CNN over the eye/face image; more robust, heavier — L2CS-Net, iTracker, ETH-XGaze). Good background: Appearance-based Gaze Estimation with Deep Learning (arXiv:2104.12668).

---

3. Solution Fact Sheets

3.1 RealEye Webcam EyeTracker Light Open

• Link: https://github.com/RealEye-io/webcam-eyetracker-light-open · npm: @realeye-io/webcam-eyetracker-light-open 1.1.0 · demo: https://realeye-io.github.io/webcam-eyetracker-light-open/
• Runtime: ✅ Web browser — 100% client-side (TypeScript; MediaPipe models + WASM from CDN; no server)
• Reported accuracy: ≈120 CSS px average after 17-point calibration (vendor-reported via in-demo accuracy overlay)
• Benchmarks: no formal published benchmark; demo app includes a click-accuracy self-measurement overlay
• Training data: none — no learned gaze model; geometric approach (MediaPipe face/iris landmarks) + custom ridge regression; head-pose compensation learned per user during calibration
• Input: camera video only + on-screen calibration points (17) — no other sensors
• Research article(s): no formal academic paper; associated technology white paper — RealEye Webcam Eye-Tracking for Desktops, Laptops and Smartphones Technology White Paper (Pietrzak, Duchowski, Krejtz et al., May 2025): https://www.realeye.io/lp/whitepaper (also cited by the WebEyeTrack paper as the closed-source benchmark reference)
• License: AGPL-3.0 + commercial dual
• Maintenance: active (pushed 2026-06)
• How it's built / unique properties: the production core of the RealEye.io commercial platform (cited in the WebEyeTrack paper as the closed-source reference). 17-point calibration ("proven optimal pattern"), ridge regression, head-pose compensation learned during calibration, CDN-loaded models, full diagnostics demo.
• Verdict: the only production-proven open browser eye-tracker; the natural foundation and benchmark for everything else.

3.2 WebEyeTrack (RedForestAI)

• Link: https://github.com/RedForestAI/WebEyeTrack · npm: webeyetrack 0.0.2 · PyPI: webeyetrack · paper: https://arxiv.org/abs/2508.19544
• Runtime: ✅ Web browser (TensorFlow.js) + Python version for training/research
• Reported accuracy: 2.32 cm point-of-gaze error on GazeCapture (few-shot); 2.4 ms inference on iPhone 14 (authors' paper, Aug 2025)
• Benchmarks: MPIIFaceGaze, GazeCapture, EyeDiap (paper Table 1; self-reported)
• Training data: MPIIFaceGaze + GazeCapture + EyeDiap (all research-restricted datasets) — two-stage training: joint representation learning, then MAML meta-learning for user adaptation; ⚠️ verify released weights' license terms before commercial use
• Input: camera video only + few-shot calibration samples (≤9 points) + optional clickstream calibration; blink suppression via eye-aspect-ratio (EAR)
• Research article(s): WEBEYETRACK: Scalable Eye-Tracking for the Browser via On-Device Few-Shot Personalization — Davalos et al., 2025: https://arxiv.org/abs/2508.19544
• License: MIT (code)
• Maintenance: active 2025–2026; young (0.0.x), academic team, small community
• How it's built / unique properties: BlazeGaze — a 670 KB CNN (BlazeBlocks, MediaPipe-style) predicting (x,y) directly; model-based metric head pose via MediaPipe 3D face reconstruction + radial procrustes analysis; on-device few-shot personalization and model caching; online fixation detection (WebFixRT); privacy-preserving (no cloud).
• Verdict: best turnkey bet today — benchmark head-to-head before production.

3.3 MediaPipe Face Landmarker + custom calibration (DIY core)

• Link: https://github.com/google-ai-edge/mediapipe · docs: https://developers.google.com/edge/mediapipe/solutions/vision/face_landmarker · npm: @mediapipe/tasks-vision
• Runtime: ✅ Web browser (WASM), Python, Android, iOS
• Reported accuracy: n/a for gaze — it is a landmark component; gaze accuracy depends on your mapping. Face detection/landmark quality documented in Google's model cards (BlazeFace short-range 192×192 + FaceMesh-V2 256×256)
• Benchmarks: Google model cards (face detection/landmarks/blendshapes — not gaze)
• Training data: Google proprietary, not disclosed; model bundle licensed Apache-2.0
• Input: camera image only (component); screen mapping requires your calibration routine
• Research article(s): MediaPipe: A Framework for Building Perception Pipelines (arXiv:1906.08172) · Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs (FaceMesh, arXiv:1907.06724) · BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs (arXiv:1907.05047) · official guides & model cards (links in §11)
• License: Apache-2.0
• Maintenance: very active (Google)
• How it's built / unique properties: 478 3D landmarks including iris + 52 blendshape scores + facial transformation matrix (head pose, directly usable for compensation). Official Tasks API for Web/Python; run inference in a Web Worker (detect() blocks the UI thread). The foundation used by RealEye light-open and by HueVision (https://github.com/simplysuvi/hue-vision — forkable TF.js calibration template, with a Roboflow tutorial: https://blog.roboflow.com/build-eye-tracking-in-browser/).
• Verdict: the safest long-term dependency and the right architecture for a controlled product.

3.4 L2CS-Net / yakhyo/gaze-estimation (appearance-based engine)

• Links: https://github.com/Ahmednull/L2CS-Net · https://github.com/yakhyo/gaze-estimation · paper: https://arxiv.org/abs/2203.03339
• Runtime: server (PyTorch / ONNX Runtime); ✅ browser possible via onnxruntime-web (WebGPU)
• Reported accuracy: 3.92° on MPIIGaze (within-dataset), 10.41° on Gaze360 (L2CS-Net paper)
• Benchmarks: MPIIGaze, Gaze360 (paper; leave-one-person-out protocol)
• Training data: MPIIGaze / Gaze360 (research datasets) — pretrained weights are research-restricted for commercial use; retrain on your own data for a commercial product
• Input: camera image (cropped face) only; outputs gaze direction (yaw/pitch) — screen mapping + per-user calibration on you
• Research article(s): L2CS-Net: Fine-Grained Gaze Estimation in Unconstrained Environments — Abdelrahman et al., 2022: https://arxiv.org/abs/2203.03339
• License: MIT (code)
• Maintenance: L2CS-Net stable but unmaintained (last push Feb 2024, weights on Google Drive); yakhyo fork active (2026-02) with MobileOne backbones and ONNX export
• How it's built / unique properties: appearance-based CNN (ResNet50 / MobileNet-v2 / MobileOne) with gaze classification + regression; the accuracy reference for webcam appearance gaze; ONNX exports ~5–20 MB (MobileOne) make client-side inference credible.
• Verdict: best model engine to pair with a pipeline — not a standalone solution.

3.5 EyeTrax

• Link: https://github.com/ck-zhang/EyeTrax · PyPI: eyetrax · DOI: https://doi.org/10.5281/zenodo.17188537
• Runtime: Python (server/desktop); ❌ no browser component
• Reported accuracy: n/a (pipeline/scaffolding, not a gaze model)
• Benchmarks: none published
• Training data: trains a per-user estimator from your webcam data (feature extraction + model)
• Input: camera video (via Python/OpenCV)
• Research article(s): none published; software archived with DOI: https://doi.org/10.5281/zenodo.17188537
• License: MIT
• Maintenance: active (2026-02), PyPI + Zenodo
• How it's built / unique properties: the most production-minded Python pipeline: calibration workflows (9-point, 5-point, Lissajous, dense grid), Kalman / Kalman+EMA / KDE smoothing, model persistence, OBS virtual-camera overlay; clean CLI demos; easy to embed in a FastAPI/WebSocket service.
• Verdict: use as the server-side calibration/smoothing layer, not as the tracker.

3.6 EyeGestures / EyeGesturesLite (NativeSensors)

• Links: https://github.com/NativeSensors/EyeGestures · https://github.com/NativeSensors/EyeGesturesLite · site: https://eyegestures.com · PyPI: eyeGestures
• Runtime: ✅ Web browser (CDN JS build: eyegestures.js from eyegestures.com) + Python package + desktop apps
• Reported accuracy: n/a — no published benchmark
• Benchmarks: none published
• Training data: not documented ("uses machine learning" per README; engine V4 described as Rust-based)
• Input: camera video only + built-in 20-point calibration (red fixation dots); API returns on-screen (x, y) with a calibration flag
• Research article(s): none published
• License: GPL-3.0 (verified LICENSE file); Lite repo has no license of its own but announces it is being merged into the main repo and will be archived — GPL-3.0 governs going forward
• Maintenance: active (pushed 2026-06); volunteer-funded (polar.sh), made in PL; known pip quirk (mediapipe/scikit-learn/opencv sometimes must be installed separately)
• How it's built / unique properties: the most complete calibrated browser gaze library besides RealEye light-open — new EyeGestures('video', onPoint) delivers (x, y) + calibration state; live cursor during calibration; desktop apps (EyeFocus; EyePilot pending restore); accessibility mission.
• Verdict: excellent for prototypes, research, and accessibility; GPL-3.0 blocks commercial SaaS embedding unless you open-source your app.

3.7 GazeTracking (antoinelame) — niche

• Link: https://github.com/antoinelame/GazeTracking
• Runtime: Python (OpenCV + dlib); ❌ not browser
• Reported accuracy: none; direction-level only
• Benchmarks: none
• Training data: none — geometric (dlib 68-point landmarks + pupil detection)
• Input: camera video only
• Research article(s): none published
• License: MIT
• Maintenance: active (2,617★, pushed 2026-04)
• How it's built / unique properties: the simplest possible pupil/gaze-direction tracker (~50 lines of usage); gives pupil centers and gaze direction (left/right/up/down), not screen coordinates.
• Verdict: only for direction-level attention signals, prototypes, education.

3.8 Considered but NOT recommended today

Solution	Link	Why not
WebGazer.js	https://github.com/brownhci/WebGazer	GPL-3.0 and official maintenance ended Feb 2026; accuracy drifts 5→10 cm over 20 min (ETRA'18) without head-pose handling. Articles: IJCAI'16 + ETRA'18 (see glossary §4). Prototypes only.
camgaze.js	https://github.com/a20r/camgaze.js	Technically attractive (zero-dep, active 2026) but no license file (legally unusable) and not published on npm.
GazeML	https://github.com/swook/GazeML	Stale (2024), TF1-era patterns; research-grade only. Useful as a retraining base (MIT).
deepgaze	https://github.com/mpatacchiola/deepgaze	Stale, 2017-era code; head pose + gaze direction only.
TurkerGaze	https://github.com/PrincetonVision/TurkerGaze	Dormant (2019); crowdsourcing game, data-collection UX reference. Article: Xu et al., arXiv:1504.06755.
lookie-lookie	https://github.com/cpury/lookie-lookie	Dormant (2020); in-browser pupil CNN curiosity.
Desktop-only: OpenFace	https://github.com/TadasBaltrusaitis/OpenFace	Standard for offline facial-behavior analysis (gaze included); desktop C++, custom license. Article: OpenFace 2.0, IEEE FG 2018.
Desktop-only: PyGaze + webcam-eyetracker	https://github.com/esdalmaijer/PyGaze · https://github.com/esdalmaijer/webcam-eyetracker	Research toolbox; GPL-3.0; calibration GUI. Article: Dalmajjer et al., BRM 2014.
Desktop-only: AITrack	https://github.com/AIRLegend/aitrack	Windows 6DoF head/gaze tracking, MIT, 2023.
Desktop-only: EyeRecToo / RemoteEye	https://github.com/tcsantini/EyeRecToo	Head-mounted tracker software (Tübingen), dormant 2020.
Desktop-only: DeepVOG	https://github.com/pydsgz/DeepVOG	Pupil segmentation for head-mounted video-oculography, GPL-3.0. Article: Yiu et al., 2019.
Desktop-only: Open-PupilEXT	https://github.com/openPupil/Open-PupilEXT	Pupillometry platform, C++, custom license.
Desktop-only: OpenGaze toolkit	https://git.hcics.simtech.uni-stuttgart.de/public-projects/opengaze	Gaze-interface toolkit (Bulling group), dormant since 2020.
Hardware ecosystem: pupil-labs/pupil	https://github.com/pupil-labs/pupil	Pupil Core headset software, LGPL-3.0; requires their hardware. Article: Kassner et al., UbiComp 2014.

---

4. Glossary of Research Articles

All articles below are cited in the solution fact sheets (§3) or the datasets section. arXiv identifiers were verified against the arXiv API on 2026-08-06; entries without an arXiv ID have no arXiv preprint (linked primary source given instead).

1. WebGazer: Scalable Webcam Eye Tracking Using User Interactions — Papoutsaki et al., IJCAI 2016 — https://jeffhuang.com/papers/WebGazer_IJCAI16.pdf — introduces the first self-calibrating browser webcam gaze library (175/210 px error, ≈4.17° vs a commercial tracker). Used by: WebGazer.js.
2. The Eye of the Typer: A Benchmark and Analysis of Gaze Behavior During Typing — Papoutsaki et al., ETRA 2018 — documents WebGazer's accuracy drift (~5 → 10 cm over 20 min) without head-pose handling. Used by: WebGazer.js.
3. Eye Tracking for Everyone — Krafka et al., CVPR 2016 — arXiv:1606.05814 — introduces the GazeCapture dataset (1,474 subjects, ~2.5M frames) and the iTracker CNN (1.71 cm mobile / 2.53 cm tablet; 1.34 / 2.12 cm calibrated). Used by: GazeCapture/iTracker, GazeML, WebEyeTrack (meta-training).
4. Appearance-Based Gaze Estimation in the Wild — Zhang et al., CVPR 2015 — introduces the MPIIGaze dataset (213,659 images, 15 participants, natural laptop use). Used by: L2CS-Net, WebEyeTrack, OpenGaze toolkit.
5. MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation — Zhang et al., TPAMI 2019 — arXiv:1711.09017 — the canonical desktop gaze benchmark and full dataset description. Used by: L2CS-Net et al.
6. L2CS-Net: Fine-Grained Gaze Estimation in Unconstrained Environments — Abdelrahman et al., 2022 — arXiv:2203.03339 — gaze yaw/pitch classification + regression; 3.92° MPIIGaze / 10.41° Gaze360. Used by: L2CS-Net, yakhyo/gaze-estimation.
7. ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation — Zhang et al., ECCV 2020 — arXiv:2007.15837 — 1M+ high-res images with extreme head poses; CC BY-NC-SA. Used by: ETH-XGaze baseline.
8. Gaze360: Physically Unconstrained Gaze Estimation in the Wild — Kellnhofer et al., ICCV 2019 — arXiv:1910.10088 — 238-subject indoor/outdoor dataset with 3D gaze. Used by: Gaze360, L2CS-Net evaluation.
9. WEBEYETRACK: Scalable Eye-Tracking for the Browser via On-Device Few-Shot Personalization — Davalos et al., 2025 — arXiv:2508.19544 — BlazeGaze 670 KB CNN + MAML few-shot personalization + metric head pose; 2.32 cm GazeCapture, 2.4 ms iPhone 14. Used by: WebEyeTrack.
10. MediaPipe: A Framework for Building Perception Pipelines — Lugaresi et al., 2019 — arXiv:1906.08172 — the cross-platform ML pipeline framework behind Face Landmarker. Used by: MediaPipe.
11. Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs — Cartucho et al., 2019 — arXiv:1907.06724 — the FaceMesh 468-point landmark model (now FaceMesh-V2 in Face Landmarker). Used by: MediaPipe Face Landmarker, WebEyeTrack, RealEye light-open.
12. BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs — Bazarevsky et al., 2019 — arXiv:1907.05047 — the face detector inside Face Landmarker; its BlazeBlocks also inspire BlazeGaze (WebEyeTrack). Used by: MediaPipe, WebEyeTrack.
13. MediaPipe Iris: Real-time Iris Tracking & Depth Estimation — Google Research Blog, 2020 — https://research.google/blog/mediapipe-iris-real-time-iris-tracking-depth-estimation/ — iris landmarks incl. depth, the basis of iris-based gaze features. Used by: MediaPipe iris pipelines.
14. TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking — Xu et al., 2015 — arXiv:1504.06755 — crowdsourced webcam gaze collection via a game. Used by: TurkerGaze.
15. OpenFace 2.0: Facial Behavior Analysis Toolkit — Baltrusaitis et al., IEEE FG 2018 — facial landmarks, head pose, gaze, AU analysis for offline research. Used by: OpenFace.
16. PyGaze: An Open-Source, Cross-Platform Toolbox for Minimal-Effort Programming of Eye Tracking Experiments — Dalmajjer, Mathôt & Van der Stigchel, BRM 2014 — eye-tracking experiment and analysis toolbox. Used by: PyGaze, webcam-eyetracker.
17. DeepVOG: A Deep-Learning Based Pupil Segmentation and Gaze Estimation System for Video-Oculography — Yiu et al., 2019 — deep pupil segmentation for head-mounted video-oculography (no arXiv preprint found; see repo). Used by: DeepVOG.
18. Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-Based Interaction — Kassner, Patera & Bulling, UbiComp 2014 — the Pupil headset software/hardware platform. Used by: pupil-labs/pupil.
19. NVGaze: An Anatomically-Informed Dataset for Low-Latency, Near-Eye Gaze Estimation — Kim et al., 2019 — https://research.nvidia.com/publication/2019-05_nvgaze-anatomically-informed-dataset-low-latency-near-eye-gaze-estimation — ~2M synthetic near-eye IR images for VR/AR (no arXiv preprint found). Used by: NVGaze dataset.
20. EyeDiap: A Database for the Development and Evaluation of Gaze Estimation Algorithms from RGB and RGB-D Cameras — Funes Mora, Monay & Odobez, ETRA 2014 — screen/appearance gaze dataset. Used by: WebEyeTrack benchmark.
21. RealEye Webcam Eye-Tracking for Desktops, Laptops and Smartphones Technology White Paper — Pietrzak, Duchowski, Krejtz et al., May 2025 — https://www.realeye.io/lp/whitepaper — technical specifications and accuracy validation of the RealEye webcam eye-tracking system. Used by: RealEye platform / light-open.
22. Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark — 2021 — arXiv:2104.12668 — survey and benchmark of the field; background reference.

---

5. Datasets & Training Data — What the Models Were Trained On

⚠️ Most gaze datasets are research-only or non-commercial — the biggest legal trap for a commercial product trained on open-source data.

Dataset	Size	License	Used by
GazeCapture / iTracker — https://github.com/CSAILVision/GazeCapture	1,474 subjects, ~2.5M frames (crowdsourced, iOS)	Research-only (explicit LICENSE.md)	iTracker, WebEyeTrack (meta-training), many papers. iTracker: 1.71 cm mobile / 2.53 cm tablet uncalibrated; 1.34 / 2.12 cm calibrated (CVPR'16)
MPIIGaze / MPIIFaceGaze — https://www.collaborative-ai.org/research/datasets/MPIIGaze/	213,659 images, 15 participants, natural laptop use	Research (per project terms)	L2CS-Net (3.92°), WebEyeTrack (meta-training), the standard desktop benchmark
ETH-XGaze — https://github.com/xucong-zhang/ETH-XGaze	1M+ high-res images, extreme head poses	CC BY-NC-SA 4.0 (non-commercial)	Baseline code + pretrained model; test labels released June 2026
Gaze360 — https://github.com/erkil1452/gaze360	238 subjects, indoor/outdoor, 3D gaze	Research	L2CS-Net (10.41°); code archived 2022
EyeDiap — https://www.idiap.ch/en/scientific-research/data/eyediap	~150 min, 16 participants, screen/appearance gaze	Research	WebEyeTrack benchmark
NVGaze (NVIDIA) — https://research.nvidia.com/publication/2019-05_nvgaze-anatomically-informed-dataset-low-latency-near-eye-gaze-estimation	~2M synthetic, near-eye IR (VR/AR)	Research	❌ Not applicable to webcam-on-screen gaze — included so you don't chase the wrong dataset

---

6. Comparison Matrix (Snapshot 2026-08-06)

Solution	Runtime	License	Maintenance	Calibration	Reported accuracy	Head pose	Input
WebGazer.js	browser	GPL-3.0	ended Feb 2026	self (clicks)	175–210 px / 4.17° (IJCAI'16); drifts 5→10 cm / 20 min	partial	camera + mouse
WebEyeTrack	browser (TF.js)	MIT	active (2025–)	few-shot <9 pts	2.32 cm GazeCapture	model-based	camera + calibration
RealEye light-open	browser	AGPL+commercial	active	17-point	≈120 CSS px	learned	camera + calibration
EyeGestures	browser+python (Rust engine)	GPL-3.0 (Lite merging in)	active	20-point	n/a	—	camera + calibration
camgaze.js	browser	none ⚠️	active (2026)	9-point	n/a (no benchmark)	no	camera + calibration
MediaPipe Face Landmarker	browser (WASM) + python	Apache-2.0	very active	— (you add)	n/a (component)	transformation matrix	camera only
L2CS-Net / yakhyo	server (ONNX→browser)	MIT	stable / active	— (you add)	3.92° MPIIGaze / 10.41° Gaze360	no (face crop)	camera (face crop)
EyeTrax	python server	MIT	active	9p/5p/Lissajous/dense	n/a (pipeline)	—	camera
GazeTracking	python	MIT	active	—	direction only	no	camera only

---

7. Recommended Solutions by Runtime & Use Case (up to 3 best)

Runtime / use case	#1	#2	#3
Web browser, client-side, privacy-first (no video leaves the device)	RealEye light-open — production-proven, exact spec, 17-pt calibration, ≈120 CSS px	WebEyeTrack — MIT turnkey, few-shot <9 samples, 2.32 cm	MediaPipe Face Landmarker + custom calibration — Apache-2.0, full control
Server-side / maximum accuracy (frames to backend, own data allowed)	L2CS-Net / yakhyo (ONNX) — 3.92° MPIIGaze accuracy reference	EyeTrax — calibration + Kalman/EMA/KDE pipeline	GazeML — MIT retraining base (iTracker-class models)
Prototype / research / accessibility (license flexibility)	EyeGestures — fastest calibrated browser demo (20-pt, CDN JS)	GazeTracking — simplest direction tracker (MIT)	WebGazer — legacy reference (GPL, unmaintained — prototypes only)
Long-term differentiated product (calibration UX = product feature)	MediaPipe DIY — you own the mapping layer and UX	WebEyeTrack hybrid — few-shot personalization on top of MediaPipe features	RealEye light-open — proven baseline to beat

---

8. Reference Architectures

Option A — Pure client-side, MediaPipe core (recommended default)

getUserMedia → Face Landmarker in a Web Worker (iris + eye corners + head-pose matrix) → feature vector → per-user calibration (9–17 points) → ridge/polynomial regression (or tiny TF.js net, HueVision-style) → smoothing → heatmap/gaze plot.

• Pros: no video leaves the device (privacy/GDPR-friendly), low latency, Apache-2.0 stack, full control of calibration UX — the proven RealEye architecture.
• Cons: you build and own the mapping/calibration layer; accuracy depends on your regression quality (expect ~1–3° with good calibration, ≈100–150 px on a 1080p laptop screen).

Option B — Turnkey: WebEyeTrack

npm package, few-shot calibration, direct (x,y) output, head-pose compensated.

• Pros: weeks of work saved; best published webcam numbers; MIT.
• Cons: 0.0.x maturity; small academic team; benchmark against your own reference data first.

Option C — Client capture + server inference (L2CS / yakhyo + EyeTrax)

Browser sends frames (throttled, e.g., 10–15 fps) over WebSocket → Python service (MediaPipe face crop → L2CS-Net/MobileOne ONNX → EyeTrax-style calibration/smoothing) → gaze coords back.

• Pros: strongest appearance-based accuracy, easy model iteration, centralized data capture.
• Cons: +network latency (≈1 RTT + inference), server cost, privacy/consent burden (video to server — significant for EU/GDPR and research panel quality), harder to scale.

Option D — Hybrid (best bet for a serious product)

MediaPipe client-side for face/iris/head-pose features → few-shot personalization (WebEyeTrack-style on-device adaptation of a small gaze head) or a lightweight appearance model in ONNX/WebGPU. Research-quality accuracy with client-side privacy. Note: to train the base model commercially you need your own data or permission for the research datasets — plan internal data collection or synthetic-data pipelines (à la NVGaze/ETH-XGaze-style rendering, which is legal).

---

9. Licensing & Compliance — Read This First

• GPL-3.0: WebGazer, EyeGestures, PyGaze, webcam-eyetracker, DeepVOG — embedding into a distributed SaaS product creates copyleft obligations for the combined work. Fine for internal tools; risky for commercial products unless you open-source your app.
• AGPL-3.0 + commercial dual: RealEye light-open — AGPL's network clause applies to SaaS use; the commercial license exists for exactly that case.
• No license file: camgaze.js — under copyright law, no license = all rights reserved; do not use commercially until the author licenses it.
• Research-only / non-commercial datasets: GazeCapture (explicit), MPIIGaze/MPIIFaceGaze, ETH-XGaze (CC BY-NC-SA), Gaze360, EyeDiap, NVGaze — you may evaluate models but not train commercial models on them without permission; the same applies to pretrained weights derived from them (relevant for L2CS-Net, WebEyeTrack, GazeML/iTracker weights).
• Safe by default: MIT (WebEyeTrack, L2CS-Net, yakhyo, EyeTrax, GazeML, GazeTracking, deepgaze, AITrack, TurkerGaze, heatmap.js, face-api.js), Apache-2.0 (MediaPipe, tfjs-models, OpenCV, jeelizPupillometry), BSL-1.0 (dlib).
• Camera privacy (EU): camera consent + GDPR — client-side processing dramatically simplifies compliance; if frames go to a server, that's personal-data processing (and research-panel data), needing consent management, DPIA thinking, and data-retention controls.

---

10. Risks & Gotchas

1. Head movement is the dominant error source — prefer solutions with explicit head-pose handling (WebEyeTrack, RealEye light-open's learned compensation, MediaPipe's transformation matrix).
2. WebGazer is GPL + unmaintained, and its accuracy degrades over time without head-pose handling (~5 → 10 cm over a 20-minute session, Papoutsaki et al., ETRA 2018) — plan migrations if you start with it.
3. WASM threading: MediaPipe multithread builds may require cross-origin isolation (COOP/COEP) — set the headers or use the single-thread build; always run inference in a Web Worker (official recommendation; detect() blocks the main thread).
4. iOS Safari / mobile: camera access requires HTTPS; test MediaPipe WASM on iOS early (WebEyeTrack's 2.4 ms iPhone figure suggests it's viable).
5. Lighting, glasses, contacts, eye color degrade geometric/pupil methods; appearance-based models are more robust but heavier.
6. Calibration drift — re-calibrate or continuously adapt (WebGazer's click-based self-calibration is the classic adaptation trick; smooth-pursuit calibration is the UX-friendly alternative used by commercial products).
7. Accuracy expectations: webcam gaze ≈ 1–3° (≈60–200 px on a laptop screen) after calibration vs 0.5–1° for IR hardware. Frame the product around relative attention (regions, heatmaps, dwell time), not pixel-perfect pointing.
8. Model loading: CDN models (MediaPipe, EyeGestures) vs self-hosting for reliability/offline; L2CS ResNet50 weights are large — prefer MobileOne/ONNX for web.
9. Metrics comparability: cm (mobile, GazeCapture) ≠ degrees (MPIIGaze/Gaze360) ≠ px — don't compare accuracy numbers across datasets; benchmark candidates head-to-head on your own target device/screen setup.

---

11. References (All Verified 2026-08-06)

Libraries / repos: github.com/brownhci/WebGazer · github.com/RedForestAI/WebEyeTrack · github.com/a20r/camgaze.js · github.com/NativeSensors/EyeGestures · github.com/NativeSensors/EyeGesturesLite · github.com/google-ai-edge/mediapipe · github.com/tensorflow/tfjs-models · github.com/simplysuvi/hue-vision · github.com/jeeliz/jeelizPupillometry · github.com/PrincetonVision/TurkerGaze · github.com/cpury/lookie-lookie · github.com/auduno/clmtrackr · github.com/eduardolundgren/tracking.js · github.com/justadudewhohacks/face-api.js · github.com/Ahmednull/L2CS-Net · github.com/yakhyo/gaze-estimation · github.com/ck-zhang/EyeTrax · github.com/swook/GazeML · github.com/antoinelame/GazeTracking · github.com/mpatacchiola/deepgaze · github.com/RealEye-io/webcam-eyetracker-light-open · github.com/esdalmaijer/PyGaze · github.com/esdalmaijer/webcam-eyetracker · github.com/AIRLegend/aitrack · github.com/tcsantini/EyeRecToo · github.com/openPupil/Open-PupilEXT · github.com/pydsgz/DeepVOG · github.com/TadasBaltrusaitis/OpenFace · github.com/pupil-labs/pupil · github.com/CSAILVision/GazeCapture · github.com/xucong-zhang/ETH-XGaze · github.com/erkil1452/gaze360 · github.com/cvlab-uob/Awesome-Gaze-Estimation · github.com/pa7/heatmap.js

Docs/papers: WebGazer IJCAI'16 (jeffhuang.com/papers/WebGazer_IJCAI16.pdf) · WebGazer drift (Papoutsaki et al., ETRA 2018, "The Eye of the Typer") · iTracker/GazeCapture CVPR'16 (arXiv:1606.05814) · L2CS-Net (arXiv:2203.03339) · WebEyeTrack (arXiv:2508.19544) · MediaPipe Face Landmarker guide (developers.google.com/edge/mediapipe/solutions/vision/face_landmarker + /web_js) · Roboflow "How to Build Real-Time Eye Tracking in the Browser" (blog.roboflow.com/build-eye-tracking-in-browser/) · Appearance-based gaze review (arXiv:2104.12668) · MPIIGaze dataset (collaborative-ai.org/research/datasets/MPIIGaze/) · EyeDiap (idiap.ch) · NVGaze (research.nvidia.com) · OpenGaze toolkit (git.hcics.simtech.uni-stuttgart.de/public-projects/opengaze) · OpenIris (PMC10925248)

---

Report v1.2 · 2026-08-06 · Repo metadata snapshot: 2026-08-06 (GitHub API)

[paq85]

webcam-eyetracking-opensource-report.pdf