FaceAuth — Rust-based Facial Recognition System

⚠️ AI-Generated Code Notice: This project documentation and code have been generated by AI. Review thoroughly before production use, as AI-generated code may contain errors, security vulnerabilities, or incomplete implementations. Test extensively and consider security audits.

FaceAuth is a Linux facial authentication system written in Rust, inspired by the Howdy project. It provides a PAM module for system login, screen unlocking, and sudo authorization using a camera (including IR cameras).

Features

• Video capture via OpenCV with manual exposure and rotation control.
• Face detection using Haar cascades (OpenCV), ONNX models (e.g., UltraLightFaceDetector), or OpenVINO-accelerated inference on Intel NPU / GPU / CPU.
• Embedding extraction with neural network models (MobileFaceNet, ArcFace, etc.) via ONNX Runtime or OpenVINO.
• Automatic hardware acceleration — OpenVINO backend automatically selects the best available device (NPU → GPU → CPU).
• Comparison with reference models using threshold checks.
• PAM integration via the faceauth-auth daemon (uses pam_exec).
• CLI for management of user models, configuration, and testing.
• Comparison with reference models using threshold checks.
• IR camera support — works with infrared cameras (e.g., Windows Hello) via exposure settings and device selection.

IR Camera Support and Login in the Dark

Standard RGB webcams can't see faces in complete darkness (no visible light). For dark login, an infrared camera with IR illumination is required (like on many Windows Hello laptops): the frame is monochrome, but the face is lit in IR.

1. Find the IR node in V4L2:

   v4l2-ctl --list-devices

2. In /etc/faceauth/config.toml, specify the device and enable IR mode:

   [video]
   device_path = "/dev/video3"
   ir_mode = true
   exposure = 100   # adjust manually for your IR camera if needed

3. Models must be captured with the same IR camera as used by faceauth-auth (same IR conditions and angle). Example:

   sudo faceauth add -u "$USER" -s 10 -d 3 --ir

   The --ir flag equals ir_mode = true in the config for that command (handy if faceauth.toml lacks this field).

ir_mode in code: disables frame rejection by "darkness" (IR brightness often looks "too dark"), and uses softer minNeighbors for Haar to detect faces more often on monochrome.

The MobileFaceNet model was trained mostly on RGB; IR quality may be slightly lower. If false rejections occur often, slightly increase distance_threshold in [recognition] (consciously, considering risk).

Architecture

┌─────────────────┐
│   PAM (pam_exec)│
└────────┬────────┘
         │ calls
┌────────▼────────┐
│ faceauth-auth   │
│ (Rust binary)   │
└────────┬────────┘
         │ uses
┌────────▼───────────────────────────────────────────────┐
│ FaceAuth Libraries                                     │
│  • camera — frame capture                              │
│  • detection — face detection (Haar / ONNX / OpenVINO) │
│  • recognition — embeddings (ONNX / OpenVINO)          │
│  • openvino_backend — OpenVINO inference wrapper       │
│  • database — model storage                            │
│  • config — TOML configuration                         │
└────────────────────────────────────────────────────────┘

Installation (for Arch Linux)

FaceAuth can be installed by building from source or via PKGBUILD (AUR).

Dependencies

• opencv – computer vision libraries
• opencv-data – Haar cascades for face detection
• v4l-utils – video camera utilities
• pam – PAM library (usually pre-installed)
• rust and cargo – for building from source
• openvino-runtime (optional) – for Intel NPU/GPU/CPU acceleration (see OpenVINO section below)

Method 1: Build from Source

1. Install dependencies:

   sudo pacman -S opencv opencv-data v4l-utils rust

2. Clone the repo and build:

   git clone https://github.com/vlad-wild/faceauth.git
   cd faceauth
   cargo build --release

3. Install system files:

   sudo cp target/release/faceauth /usr/local/bin/
   sudo cp target/release/faceauth-auth /usr/local/bin/
   sudo cp target/release/faceauth-ui /usr/local/bin/
   sudo mkdir -p /etc/faceauth
   sudo cp faceauth.toml /etc/faceauth/config.toml

Build without OpenVINO: If you don't need Intel NPU/GPU/CPU acceleration via OpenVINO, build with:

cargo build --release --no-default-features

Method 2: Install via PKGBUILD (AUR)

For Arch Linux, use the provided PKGBUILD.

1. Navigate to PKGBUILD directory:

   cd faceauth

2. Build and install:

   makepkg -si

   Or via AUR helper (if uploaded to AUR):

   yay -S faceauth

   The package installs:

   • /usr/bin/faceauth – CLI management utility
   • /usr/bin/faceauth-auth – PAM daemon
   • /usr/bin/faceauth-ui – face capture window (optional)
   • /etc/faceauth/config.toml – default config
   • /usr/share/doc/faceauth/README.md – documentation
   • /usr/share/doc/faceauth/pam-example – PAM config example
   • /usr/lib/systemd/system/faceauth.service – systemd service (optional)

3. Configure PAM after installation (see below).

PAM Configuration

Add to /etc/pam.d/system-auth (or /etc/pam.d/sudo):

auth [success=2 default=ignore] pam_exec.so quiet /usr/local/bin/faceauth-auth
auth [success=1 default=bad] pam_unix.so try_first_pass nullok

The username comes from the PAM_USER environment variable set by pam_exec (shell $USER substitution doesn't work in pam.d files). Optionally specify user explicitly: ... faceauth-auth -u name (for single-user machines).

Adding a Face Model

sudo faceauth add --user $USER --samples 5
# IR/dark: same camera as in /etc/faceauth/config.toml (device_path + ir_mode), e.g.:
sudo faceauth add -u "$USER" -s 10 -d 3 --ir

Multiple Appearances and Model Augmentation

Stores a primary embedding set and named variants (e.g., glasses). Uses the best match across all sets during checks.

Action	Example
Full model replacement (reset variants)	faceauth add -u USER -s 10
Append shots to primary set	faceauth add -u USER -s 8 --append
"Glasses" variant (replace named set)	faceauth add -u USER -s 10 --variant glasses
Append to existing variant	faceauth add -u USER -s 5 --variant glasses --append

faceauth-ui uses radio buttons for the same modes.

OpenVINO / Intel NPU Acceleration

FaceAuth supports OpenVINO inference backend for both face detection (Ultra-Light) and face recognition (MobileFaceNet). When enabled, the framework automatically selects the best available device in priority order:

1. NPU (Intel Neural Processing Unit / AI Boost)
2. GPU (Intel integrated graphics)
3. CPU (fallback)

Enabling OpenVINO

Both [detection] and [recognition] have use_openvino = true by default. To disable OpenVINO and force CPU inference via tract-onnx, set:

[detection]
use_openvino = false

[recognition]
use_openvino = false

Installing OpenVINO Runtime

On Arch Linux, OpenVINO is available via the AUR (not in official repositories). Choose one of the following methods:

Build from source (AUR):

yay -S openvino

Pre-compiled binary (AUR, may be outdated):

yay -S openvino-bin

Alternative (Python/PyPI): If you only need the runtime libraries and have Python available, you can also install the shared libraries via pip in a virtual environment (the Rust crate uses runtime-linking and needs the .so files at runtime):

python -m venv /opt/openvino-runtime
source /opt/openvino-runtime/bin/activate
pip install openvino

Then ensure /opt/openvino-runtime/lib/python3.x/site-packages/openvino/libs (or wherever the .so files end up) is in your LD_LIBRARY_PATH, or symlink them to /usr/local/lib.

Note: use_openvino settings during enrollment and authentication do not need to match (the model files are identical). However, accuracy may differ slightly between NPU/GPU and CPU backends. If you encounter false rejections after switching devices, consider re-enrolling with the new backend.

Graphical Interface (faceauth-ui)

Camera preview window for face capture without CLI: preview, capture progress, same settings as faceauth add.

cargo run --release --bin faceauth-ui
# or after install:
faceauth-ui

Config loaded in order: ./faceauth.toml → ~/.config/faceauth/config.toml → /etc/faceauth/config.toml. Relative ONNX paths resolve relative to config file directory.

Configuration

Example /etc/faceauth/config.toml:

[video]
device_path = "/dev/video0"
timeout = 4
dark_threshold = 50.0
certainty = 3.5
max_height = 320.0
rotate = 0
exposure = -1
ir_mode = false

[detection]
model_path = "models/ultra_light_640.onnx"
yunet_path = "models/face_detection_yunet_2023mar.onnx"
use_cnn = false
use_openvino = true   # enable OpenVINO for detector (NPU → GPU → CPU)
confidence_threshold = 0.7

[recognition]
model_path = "models/mobilefacenet.onnx"
use_openvino = true   # enable OpenVINO for recognizer
embedding_size = 128
distance_threshold = 0.6

[debug]
end_report = false
save_failed = false
save_successful = false

Security

• All data (embeddings) stored locally in encrypted home directory.
• Facial auth is an additional factor, not a password replacement.
• IR camera recommended for photo spoofing protection.

Development

Project Structure

src/
├── camera.rs            # Video capture, frame processing
├── config.rs            # Config load/save
├── detection.rs         # Face detection (Haar / ONNX / OpenVINO)
├── enroll.rs            # Model enrollment (CLI + UI)
├── recognition.rs       # Embedding extraction (ONNX / OpenVINO)
├── openvino_backend.rs  # OpenVINO inference wrapper (auto NPU → GPU → CPU)
├── database.rs          # Model storage and comparison
├── main.rs              # CLI utility
└── bin/
    ├── auth.rs          # PAM daemon
    └── ui.rs            # Face capture window

Adding New Recognition Model

1. Place ONNX model files in models/.
2. Update config.toml with model paths.
3. Implement preprocessing in recognition.rs.
4. OpenVINO will be used automatically if use_openvino = true and the model is compatible; otherwise, tract-onnx is used as a fallback.

License

MIT