Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extended Dlib image_processing to save images to directories and bugfixes #8515

Closed

Conversation

tchellomello
Copy link
Contributor

@tchellomello tchellomello commented Jul 17, 2017

Description:

During the image_processing.dlib* configuration process I found a little bit complicated to understand if the component was working or not.
So the idea of this PR is to make the process a little more comprehensive by allowing the user some options to save the images captures by Dlib to a directory to determine how accurate the face_recognition is.

This PR also fixes the issue reported at #8493 since dlib behaves a little bit different than microsoft_face_recognition.

This also allows dlib_face_identify to show the name of the face detected instead showing the meta object on the frontend.

Related issue (if applicable): fixes #8493

Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.github.io#<home-assistant.github.io PR number goes here> wip

Example entry for configuration.yaml (if applicable):

#configuration.yaml
image_processing: !include image_processing.yaml
automation: !include automations.yaml
android_ip_webcam:
  - host: 192.168.169.121
    scan_interval: 3

#image_processing.yaml
- platform: dlib_face_detect
  keep_faces: True
  keep_no_faces: True
  source:
    - entity_id: camera.ip_webcam
      name: "detect face"

- platform: dlib_face_identify
  keep_known_faces: True
  keep_unknown_faces: True
  source:
    - entity_id: camera.ip_webcam
      name: "Identify mmello"
  faces:
    mmello: /home/hass/.homeassistant/faces/mmello.png
    mmello-fail: /home/hass/.homeassistant/faces/mmello-fail.jpg

#automation.yaml
- alias: mmello detected
  trigger:
    platform: event
    event_type: image_processing.detect_face
    event_data:
      entity_id: image_processing.identify_mmello
      name: mmello
  action:
    service: homeassistant.turn_on
    entity_id: switch.kitchen_light

image

image

When enabled, the options above will save the images on the directories {CONFIGURATION_HASS}/:

(ha-py36) ↪ ls -lad dlib_*
drwxrwxr-x. 2 mdemello mdemello 12288 Jul 17 01:23 dlib_faces/
drwxrwxr-x. 2 mdemello mdemello  4096 Jul 17 01:23 dlib_known_faces/
drwxrwxr-x. 2 mdemello mdemello 12288 Jul 17 01:23 dlib_nofaces/
drwxrwxr-x. 2 mdemello mdemello 12288 Jul 17 01:23 dlib_unknown_faces/

(ha-py36) ↪ ls -la dlib_known_faces/ | head
total 1296
drwxrwxr-x. 2 mdemello mdemello  4096 Jul 17 01:23 .
drwxrwxr-x. 9 mdemello mdemello  4096 Jul 17 01:23 ..
-rw-rw-r--. 1 mdemello mdemello 16673 Jul 17 00:36 camera.ip_webcam_Jul-17-2017_00:36:14.jpg
-rw-rw-r--. 1 mdemello mdemello 18964 Jul 17 00:36 camera.ip_webcam_Jul-17-2017_00:36:36.jpg
-rw-rw-r--. 1 mdemello mdemello 18681 Jul 17 00:36 camera.ip_webcam_Jul-17-2017_00:36:47.jpg
-rw-rw-r--. 1 mdemello mdemello 18046 Jul 17 00:36 camera.ip_webcam_Jul-17-2017_00:36:58.jpg
-rw-rw-r--. 1 mdemello mdemello 17850 Jul 17 00:37 camera.ip_webcam_Jul-17-2017_00:37:09.jpg
-rw-rw-r--. 1 mdemello mdemello 16434 Jul 17 00:37 camera.ip_webcam_Jul-17-2017_00:37:42.jpg
-rw-rw-r--. 1 mdemello mdemello 16771 Jul 17 00:38 camera.ip_webcam_Jul-17-2017_00:38:04.jpg

Checklist:

If user exposed functionality or configuration variables are added/changed:

If the code does not interact with devices:

  • Local tests with tox run successfully. Your PR cannot be merged unless tests pass
  • Tests have been added to verify that the new code works.

…pdate' (fixes home-assistant#8493)

  - Introduced ability to debug picture when a face is or not detected.
  - Treats exception when image cannot be parsed
  - Modified state_attributes to return the face_name attribute instead object
@pvizeli
Copy link
Member

pvizeli commented Jul 17, 2017

It is not the Task of the image_processing to store image. @fabaff is working on a snapshot servcies, so you can trigger that service with a automation.

I can not see the real fix for that bug. I will make it soon. Need only 1 line code.

@pvizeli pvizeli closed this Jul 17, 2017
@arsaboo
Copy link
Contributor

arsaboo commented Jul 17, 2017

@pvizeli The problem with the way the image_processing components are currently set up is that there is no feedback to the user about what is going on. I have been using both OpenCV and Dlib and feel like these are black boxes. It would be good to know what was detected so that I can act accordingly. The coordinates in the image_processing entities are of little value to users. Also, saving images/video allow us to use them in notifications, which make these components significantly more useful.

May be there is a better way of handling the image storage, but we definitely need something like this.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dlib: AttributeError: 'tuple' object has no attribute 'update'
5 participants