Skip to content

About

Teach a local AI what your camera sees and turn it into Home Assistant sensors — garage door open/closed, gate shut, car parked. 100% local, any CPU.

Topics

Resources

Security policy

Stars

107 stars

Watchers

1 watching

Forks

Repository files navigation

VisionState

Turn your cameras into Home Assistant sensors — with a local AI you teach in minutes.
Is the garage door open? Who just came home, and is that our car? What does the water meter say?

CI Version Architectures Runs locally License Buy me a coffee

Add repository to my Home Assistant

VisionState with a driveway, a garage door and a water meter sensor

Three kinds of sensor

What you get How
🚪 States Garage door open, closed or halfway, gate shut, lights on in the shed Click a few examples of each state; it learns in about a second
🧍 Objects & people Emma is home, our car is in the driveway, a cat on the lawn, an unknown person at the door Pick from 80 objects — no training. Name people by their face once; every camera knows them
🔢 Readings The water, gas or power meter, the minutes left on the washer Draw a box around the number. Counters only go up and land in the Energy dashboard

An object sensor recognising Emma by her face and telling our car from any car

A reading sensor reading the wheels of a real water meter The People page with the faces taught for Emma
Real meters — the rolling wheels of a water meter, read right even while a wheel is half way between two digits. People — a few photos from your phone are enough to start; faces never leave your Home Assistant.

Highlights

  • 🧠 100 % local on any CPU — Intel, AMD, Raspberry Pi 4/5. No cloud, no GPU, no subscription, no telemetry.
  • 🏠 Native Home Assistant — a sidebar app; every sensor is a normal device through MQTT discovery.
  • 🎥 Any camera — every camera.* entity, or an RTSP / HTTP snapshot URL.
  • 🎯 Watch only what matters — draw a region around the door or the driveway; the rest is ignored.
  • ⚡ Checks when something happens — on motion, a door contact or a change in the image, and can switch on a light for a camera in the dark.
  • 🔁 Gets better as you use it — unsure frames go to a review queue, wrong boxes are fixed with a tap, and suspicious labels are flagged.

In Home Assistant

A garage door state sensor as a Home Assistant device The driveway object sensor as a Home Assistant device, with Emma and our car
State sensor — the state with its confidence, the last frame, a button to check now and a switch to pause it. Object sensor — an on/off sensor and a count for every object, person and own label.
# Example: tell me when the garage door has been open for 10 minutes
triggers:
  - trigger: state
    entity_id: sensor.garage_door
    to: open
    for: "00:10:00"
actions:
  - action: notify.mobile_app_phone
    data:
      message: The garage door has been open for 10 minutes.

Install

Requirements: Home Assistant OS or Supervised (2025.10 or newer), the Mosquitto broker app with the MQTT integration, and a camera.

  1. Click Add repository above — or add https://github.com/oleost/VisionState under Settings → Apps → App store → ⋮ → Repositories.
  2. Install VisionState, start it and open it from the sidebar.
  3. New sensor → pick a camera, draw a region, choose what to watch. The wizard tests it on a fresh frame right away.

New features come to a separate beta app first: add https://github.com/oleost/VisionState#beta instead. Run only one of the two at a time (they publish the same sensors); move sensors with Export and Import.

The user guide — every entity, triggers, faces, readings, review — is on the app's Documentation tab (visionstate/DOCS.md).

How it works

camera ─► your region ─► pretrained model (ONNX, CPU) ─► small per-sensor model ─► debounce ─► MQTT ─► Home Assistant
  • States: DINOv2 turns the region into a feature vector; a tiny classifier per sensor, trained on your examples, picks the state.
  • Objects: D-FINE finds the COCO objects; boxes you corrected or gave an own label teach it your camera.
  • Faces: YuNet finds a face on each person, and only a clear one is compared (SFace, or AuraFace on a fast PC) with the faces you named.
  • Readings: PaddleOCR reads digits; mechanical counters use a small wheel reader made for VisionState (tools/wheelreader). Every value is checked before it is published.

Privacy & security

  • Nothing leaves your machine; there is no telemetry.
  • Only reachable through Home Assistant Ingress (a Home Assistant login).
  • Camera passwords are hidden in logs and removed from exports.
  • Face recognition is off until you turn it on for a camera; faces are kept in the app's own storage and exported only if you tick it.

Contributing

Pull requests go to the beta branch. CLAUDE.md is the contributor guide (run it locally against a fake camera, checks, UI tests on desktop and phones) and docs/ARCHITECTURE.md maps the code. Issues and ideas: GitHub Issues. Security: SECURITY.md.

Licence

Apache-2.0. Bundled models: DINOv2 (Meta), D-FINE, PaddleOCR PP-OCRv6 and the VisionState wheel reader, all Apache-2.0. Face models, downloaded only when a camera recognises faces: YuNet (MIT), SFace or AuraFace (Apache-2.0). The screenshots use CC0 and public-domain images.

If VisionState is useful to you, you can buy me a coffee ☕

About

Teach a local AI what your camera sees and turn it into Home Assistant sensors — garage door open/closed, gate shut, car parked. 100% local, any CPU.

Topics

Resources

Security policy

Stars

107 stars

Watchers

1 watching

Forks

Releases

Sponsor this project

Packages

Contributors

Languages