Face Detection & Recognition (Jetson-ready)

Highlights

• GPU-aware model loading with CUDA/TensorRT/DirectML when available; CPU fallback is automatic.
• Improved cosine-similarity recognition for better tolerance to slight side profiles.
• Preloaded models and tuned JPEG encoding to reduce latency and memory churn.
• Production-friendly logging with APP_LOG_LEVEL (INFO by default).
• Refreshed NiceGUI layout with clear status chips, directory view, and upload tester.

Install

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install --upgrade pip
pip install -r requirements.txt

Ubuntu: CUDA (Recommended for NVIDIA GPUs)

1. Install NVIDIA Drivers

   • Make sure you have the latest drivers for your GPU:

     sudo apt update
     sudo apt install nvidia-driver-535  # Or latest for your GPU
     sudo reboot

2. Install CUDA Toolkit (11.8 recommended)

   • Download from https://developer.nvidia.com/cuda-toolkit-archive
   • Follow NVIDIA's instructions for your Ubuntu version.
   • Add CUDA to your PATH and LD_LIBRARY_PATH (usually done by the installer):

     export PATH=/usr/local/cuda/bin:$PATH
     export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

3. (Optional) Install cuDNN

   • Download cuDNN for CUDA 11.x from https://developer.nvidia.com/cudnn
   • Extract and copy the files to your CUDA folders as per NVIDIA's instructions.

4. Install ONNX Runtime GPU

   • In your Python environment:

     pip uninstall -y onnxruntime
     pip install onnxruntime-gpu

5. Verify GPU is available

   • Run:

     import onnxruntime
     print(onnxruntime.get_available_providers())

   • You should see CUDAExecutionProvider in the output.

6. (Optional) TensorRT for Advanced Users

   • Install TensorRT from https://developer.nvidia.com/tensorrt
   • Add the TensorRT lib directory (containing nvinfer*.so) to your LD_LIBRARY_PATH.
   • Most users do not need TensorRT; CUDA is fast and stable.

Provider selection: By default, this app uses CUDA if available, then CPU. You can change this in config.py via PROVIDER_PRIORITY.

Windows: CUDA + cuDNN

1. Install NVIDIA Drivers

   • Get drivers from: https://www.nvidia.com/Download/index.aspx

2. Install CUDA Toolkit

   • Install a CUDA toolkit supported by your onnxruntime-gpu wheel (11.8 or 12.x commonly).
   • Ensure cudart64_*.dll is on your PATH (CUDA installer normally does this).

3. Install cuDNN

   • Download cuDNN matching your CUDA version from NVIDIA (developer account required).
   • Copy cudnn64_*.dll into your CUDA bin folder (for example C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin) or add the cuDNN bin folder to your PATH.

4. Install ONNX Runtime GPU wheel

   pip uninstall -y onnxruntime
   pip install onnxruntime-gpu

5. Verify providers

   import onnxruntime
   print(onnxruntime.get_available_providers())

   • If CUDAExecutionProvider is present, GPU should be usable. If you see errors about missing cudnn64_*.dll, ensure cuDNN is installed and its bin is on PATH.

If you prefer TensorRT on Windows, install TensorRT and add its bin directory (containing nvinfer_*.dll) to PATH.

Run

python main.py

Configuration

Set via environment variables or edit config.py:

• IP_CAMERA_URL (e.g. rtsp://user:pass@host:554/stream; if set, overrides camera index)
• VIDEO_SOURCES (comma-separated list, supports id=uri values, e.g. gate=rtsp://...,desk=0)
• ACTIVE_SOURCE_ID (optional source id selected at startup)
• CAMERA_INDEX (default 0)
• FRAME_WIDTH / FRAME_HEIGHT (default 1280x720)
• TARGET_FPS (default 10)
• PROCESS_EVERY_N_FRAMES (default 1)
• DETECTION_INPUT_SIZE (detector input side length; smaller is faster, default profile-based)
• DETECTION_SCALE (downscale before detection; smaller is faster, default profile-based)
• MAX_FACES_PER_FRAME (caps faces processed per frame, default 8)
• RECOGNITION_THRESHOLD (cosine similarity, default 0.35)
• FACES_REFRESH_SECONDS (DB sync cadence, default 2s)
• UNKNOWN_INSERT_COOLDOWN_SECONDS (default 2)
• UNKNOWN_RETENTION_HOURS (default 72)
• INFERENCE_BACKEND (python or deepstream, default python)
• DEEPSTREAM_CONFIG_PATH (optional DeepStream app config path)
• HEALTH_ENABLED / METRICS_ENABLED (default enabled)
• ACCESS_TOKEN (optional shared token for /video/frame, /health, /metrics)
• AUTH_SECRET (enables signed expiring tokens and RBAC)
• AUTH_USERS_FILE (default deploy/users.json, local user/role store)
• TOKEN_TTL_SECONDS (signed token lifetime in seconds, default 3600)
• OPERATOR_PIN (optional lock for enroll/promote/delete actions in UI)
• OPERATOR_SESSION_TIMEOUT_SECONDS (operator unlock session timeout; default 900)
• WRITE_QUEUE_MAXSIZE / WRITE_TIMEOUT_SECONDS (async write queue tuning)
• APP_LOG_LEVEL (INFO/DEBUG/WARN)
• JPEG_QUALITY (default 85)

RBAC Bootstrap

1. Copy deploy/users.example.json to deploy/users.json.
2. Generate password hashes:

python scripts/make_password_hash.py "your-strong-password"

3. Replace each password_hash in deploy/users.json.
4. Set AUTH_SECRET (long random string) and optional TOKEN_TTL_SECONDS.
5. Restart the app.

When auth is enabled, roles are:

• viewer: view streams and status
• operator: viewer + source switch + enroll/promote/delete (with operator lock if enabled)
• admin: operator + audit trail visibility

Multi-source Example

# Linux/macOS
export VIDEO_SOURCES="gate=rtsp://user:pass@192.168.1.20:554/stream1,desk=0"
export ACTIVE_SOURCE_ID="gate"

# Windows PowerShell
$env:VIDEO_SOURCES="gate=rtsp://user:pass@192.168.1.20:554/stream1,desk=0"
$env:ACTIVE_SOURCE_ID="gate"

The UI includes an "Active source" selector in the Live Monitor tab to switch sources at runtime.

Documentation Index

• docs/01-architecture.md - system architecture and data flow
• docs/02-knowledge-base.md - operational KB and tuning notes
• docs/03-similar-projects-research.md - external project research and best strategies
• docs/04-jetson-nano-production-playbook.md - Jetson Nano production optimization guide
• docs/05-stream-sources-setup.md - stream onboarding and source config examples
• docs/06-roadmap-and-implementation-status.md - implemented changes and next roadmap
• docs/07-production-deployment.md - Linux/systemd deployment guide
• docs/08-deepstream-backend.md - DeepStream backend mode and migration notes
• docs/09-security-and-auth.md - endpoint security and operator lock controls
• docs/10-monitoring-grafana.md - Prometheus and Grafana setup
• docs/11-reverse-proxy-tls.md - Nginx/Caddy TLS reverse proxy templates
• docs/12-deep-hardening-layer.md - summary of deep hardening pass
• docs/13-project-description-and-architecture-master.md - complete product description, feature matrix, and architecture reference

Operations Endpoints

• GET /health - service status, stream status, queue depth, freshness and FPS
• GET /metrics - Prometheus-style counters for throughput, drops, reconnects, retention cleanup
• POST /auth/login - body { "username": "...", "password": "..." }, returns signed token when AUTH_SECRET is enabled
• POST /auth/logout - stateless logout event endpoint

All protected endpoints accept either:

• Authorization: Bearer <token>
• X-Access-Token: <token>
• query ?access_token=<token>

Jetson And Service Scripts

• scripts/run_jetson_profile.sh - launches app with Nano-friendly defaults and optional clock boost
• scripts/install_systemd_service.sh - installs and starts systemd service
• deploy/systemd/visionid.service - service unit template
• deploy/systemd/visionid.env.example - production env template
• deploy/grafana/visionid-dashboard.json - Grafana dashboard import template
• deploy/proxy/nginx.visionid.conf - Nginx TLS reverse proxy example
• deploy/proxy/Caddyfile.visionid - Caddy TLS reverse proxy example

Project Layout

├── main.py              # NiceGUI app, camera loop, upload/registration
├── face_utils.py        # Provider selection, detection, recognition
├── db.py                # SQLite helpers for embeddings
├── config.py            # Runtime tunables
├── requirements.txt     # Minimal deps; swap onnxruntime for GPU build on Jetson
├── models/              # InsightFace models (auto)
├── storage/             # Saved snapshots (auto)
└── embeddings.db        # SQLite store (auto)

Notes

• InsightFace downloads the chosen model into models/ if not present.
• Recognition uses cosine similarity; keep faces reasonably frontal or with light yaw for best accuracy.
• The UI displays model info and current provider; logs show which provider was selected at startup.