Turn your cameras into Home Assistant sensors — with a local AI you teach in minutes.
Is the garage door open? Who just came home, and is that our car? What does the water meter say?
| What you get | How | |
|---|---|---|
| 🚪 States | Garage door open, closed or halfway, gate shut, lights on in the shed | Click a few examples of each state; it learns in about a second |
| 🧍 Objects & people | Emma is home, our car is in the driveway, a cat on the lawn, an unknown person at the door | Pick from 80 objects — no training. Name people by their face once; every camera knows them |
| 🔢 Readings | The water, gas or power meter, the minutes left on the washer | Draw a box around the number. Counters only go up and land in the Energy dashboard |
- 🧠 100 % local on any CPU — Intel, AMD, Raspberry Pi 4/5. No cloud, no GPU, no subscription, no telemetry.
- 🏠 Native Home Assistant — a sidebar app; every sensor is a normal device through MQTT discovery.
- 🎥 Any camera — every
camera.*entity, or an RTSP / HTTP snapshot URL. - 🎯 Watch only what matters — draw a region around the door or the driveway; the rest is ignored.
- ⚡ Checks when something happens — on motion, a door contact or a change in the image, and can switch on a light for a camera in the dark.
- 🔁 Gets better as you use it — unsure frames go to a review queue, wrong boxes are fixed with a tap, and suspicious labels are flagged.
![]() |
![]() |
| State sensor — the state with its confidence, the last frame, a button to check now and a switch to pause it. | Object sensor — an on/off sensor and a count for every object, person and own label. |
# Example: tell me when the garage door has been open for 10 minutes
triggers:
- trigger: state
entity_id: sensor.garage_door
to: open
for: "00:10:00"
actions:
- action: notify.mobile_app_phone
data:
message: The garage door has been open for 10 minutes.Requirements: Home Assistant OS or Supervised (2025.10 or newer), the Mosquitto broker app with the MQTT integration, and a camera.
- Click Add repository above — or add
https://github.com/oleost/VisionStateunder Settings → Apps → App store → ⋮ → Repositories. - Install VisionState, start it and open it from the sidebar.
- New sensor → pick a camera, draw a region, choose what to watch. The wizard tests it on a fresh frame right away.
New features come to a separate beta app first: add https://github.com/oleost/VisionState#beta
instead. Run only one of the two at a time (they publish the same sensors); move sensors with
Export and Import.
The user guide — every entity, triggers, faces, readings, review — is on the app's
Documentation tab (visionstate/DOCS.md).
camera ─► your region ─► pretrained model (ONNX, CPU) ─► small per-sensor model ─► debounce ─► MQTT ─► Home Assistant
- States: DINOv2 turns the region into a feature vector; a tiny classifier per sensor, trained on your examples, picks the state.
- Objects: D-FINE finds the COCO objects; boxes you corrected or gave an own label teach it your camera.
- Faces: YuNet finds a face on each person, and only a clear one is compared (SFace, or AuraFace on a fast PC) with the faces you named.
- Readings: PaddleOCR reads digits; mechanical
counters use a small wheel reader made for VisionState (
tools/wheelreader). Every value is checked before it is published.
- Nothing leaves your machine; there is no telemetry.
- Only reachable through Home Assistant Ingress (a Home Assistant login).
- Camera passwords are hidden in logs and removed from exports.
- Face recognition is off until you turn it on for a camera; faces are kept in the app's own storage and exported only if you tick it.
Pull requests go to the beta branch. CLAUDE.md is the contributor guide (run it
locally against a fake camera, checks, UI tests on desktop and phones) and
docs/ARCHITECTURE.md maps the code. Issues and ideas:
GitHub Issues. Security: SECURITY.md.
Apache-2.0. Bundled models: DINOv2 (Meta), D-FINE, PaddleOCR PP-OCRv6 and the VisionState wheel reader, all Apache-2.0. Face models, downloaded only when a camera recognises faces: YuNet (MIT), SFace or AuraFace (Apache-2.0). The screenshots use CC0 and public-domain images.
If VisionState is useful to you, you can buy me a coffee ☕





