Online learning dev #5

cagbal · 2017-07-28T08:25:29Z

Hi,
I added some new functionalities to the repo all of them written in Python such as:

Deep learning + clustering based online learning function. By using this method, the system can automatically learn new faces and save them in its database. Please refer to flowchart for the working mechanism. It says -1 to unknown persons, and assigns a new positive integer to known people. If it does not assign a new label to a new person, please consider changing the parameters inside the online_recognition_params.yaml file.
Added Dlib's face detection package which is HOG(Histogram of Oriented Gradients). Thanks to dlib, now we can extract the facial key points, they are not published, but the program uses them internally.
recognition_method parameter is given to program as argument, so if it is set to 4, program automatically switches to new learning system by killing face_recognizer and face_capture_nodes.
@ipa-rmb said that the new system should be compatible with the old client. So, the client's update_label and delete_label features can be used for this new system.
Added toggle online learning service. Initially, online learning is always off. You can turn it on by calling the service.
Unfortunately, some new dependencies must be installed. So, I wrote an install.sh script. Please refer to it to install them.

Flowchart:

After the installations, please set the correct paths of openface and model directories inside the face_detector_params.yaml and online_face_recognizer_params.yaml. If you suggest a automatized way to do this, I would really appreciate that.

TODO

Keep track of the labels. @ipa-rmb, I could not add it yet. It will take some time.
After the pull request is accepted(integration is completed), the documentation needs to be updated as well.

This is my first pull-request for this repo. So, I appreciate your valuable comments. Thanks.
ipa-cgo (cagatay odabasi)

ipa-rmb

Please update your code according to the comments, then I will check it again. And please use the nice drawing for documenting your code on ros wiki.

ipa-rmb · 2017-07-28T13:37:33Z

cob_people_detection/ros/launch/face_detector_params.yaml

@@ -21,7 +21,7 @@ faces_min_search_scale_y: 30
 # when this flag is activated, the 3d size of a face is determined and only reasonable sizes may continue in the processing pipeline
 # moreover, if activated, the background behind the head is cleared in the color image resulting in more robust recognition performance
 # bool
-reason_about_3dface_size: true
+reason_about_3dface_size: false


Did you turn this off on purpose? Why? If not, turn it on again.

ipa-rmb · 2017-07-28T13:39:46Z

cob_people_detection/ros/launch/face_detector_params.yaml

+
+# openface folder
+# string
+openface_directory: "/home/cag/openface/openface/"


Better use "~/openface/openface" . If you only read from there and never write, you can drop this parameter and use rospack to find the path because you need to wrap openface in a ROS package anyways (see comment at your install script).

ipa-rmb · 2017-07-28T13:44:13Z

cob_people_detection/ros/launch/people_detection.launch

 for Care-O-bot default is cam3d_nodelet_manager -->
  <arg name="start_manager" default="false"/>     <!-- if you do not like to use the nodelet manager provided by the openni driver, specify your own by setting this parameter to true and providing the name of your manager with argument nodelet_manager-->
+  <arg name="display_results_with_image_view" default="true"/>		<!-- set to false if you do not like to display the camera image with names attached to detected faces within an image_view window -->
+  <arg name="recognition_method" default="1"/>


We need some comment here what the different options are and what they mean, e.g. 

Make 2 as default option as it was initially. 1=Eigenfaces performs badly.

ipa-rmb · 2017-07-28T13:56:10Z

cob_people_detection/ros/launch/face_detector_hog.launch

+<?xml version="1.0"?>
+
+<launch>
+  <arg name="camera_namespace" default=""/>


add a default value like "camera"

ipa-rmb · 2017-07-28T13:58:13Z

cob_people_detection/ros/launch/online_face_recognizer.launch

+<launch>
+ <arg name="camera_namespace" default=""/>
+ <arg name="in_topic_name" value="/face_detector/face_positions"/>
+ <arg name="recognition_method" />


Please repeat the comment about the different options here and set a default value, e.g. 2.


ipa-rmb · 2017-07-28T15:19:11Z

cob_people_detection/ros/launch/face_detector_hog.launch

+<?xml version="1.0"?>
+
+<launch>
+  <arg name="camera_namespace" default=""/>


Is this necessary? Remove it if not, it does not make sense to me.

ipa-rmb · 2017-07-28T15:20:01Z

cob_people_detection/ros/launch/face_detector_hog.launch

+  <node name="face_detector_hog" pkg="cob_people_detection" type="face_detector_hog_node.py" output="screen">
+    <rosparam command="load" file="$(find cob_people_detection)/ros/launch/face_detector_params.yaml"/>
+    <param name="data_directory" type="string" value="$(find cob_people_detection)/common/files/"/>
+    <remap from="~head_positions" to="$(arg camera_namespace)$(arg in_topic_name)"/>


Are you sure to use $(arg camera_namespace) in front of the topic?

ipa-rmb · 2017-07-28T15:20:34Z

cob_people_detection/ros/launch/face_recognizer.launch

@@ -1,10 +1,12 @@
 <?xml version="1.0"?>

 <launch>
+  <arg name="recognition_method" />


We need some explanatory comment here, e.g.


Also set a default value, e.g. 2.

ipa-rmb · 2017-07-28T15:24:03Z

cob_people_detection/ros/launch/online_face_recognizer.launch

+  <!-- learn and recognizes faces online and publish their positions -->
+  <node name="online_face_recognizer" pkg="cob_people_detection" type="online_face_recognition_node.py" output="screen">
+    <rosparam command="load" file="$(find cob_people_detection)/ros/launch/online_face_recognizer_params.yaml"/>
+    <remap from="~face_detector/face_positions" to="$(arg camera_namespace)$(arg in_topic_name)"/>


It seems like your pull request does not affect the camera_namespace argument. Just remove it or make a systematic proposal in all files. But for that start a new pull request.

ipa-rmb · 2017-07-28T15:24:20Z

cob_people_detection/ros/launch/online_face_recognizer.launch

+<?xml version="1.0"?>
+
+<launch>
+ <arg name="camera_namespace" default=""/>


ipa-rmb

Looks good so far, only a few simple requests.

ipa-rmb · 2017-08-01T14:06:52Z

cob_people_detection/ros/launch/people_detection.launch

+
+  <!-- remaining launch files -->
+  <param name="data_storage_directory" type="string" value="$(env HOME)/.ros/cob_people_detection/files/"/>
+  <param name="face_detector_method" type="int" value="$(arg face_detector_method)"/> <!-- 0 -> hog: Histogram of Oriented Gradients,


Do you really need to create this parameter here? Is it used from some program later on?

No, thanks! I missed it which is not useful anymore.

ipa-rmb · 2017-08-01T14:08:52Z

cob_people_detection/ros/launch/people_detection.launch

+
+  <!--Face detector-->
+  <group if="$(arg launch_face_detector)">
+    <include if="$(arg face_detector_method)" file="$(find cob_people_detection)/ros/launch/face_detector_hog.launch"/>


Did you test this? I am always confused with if/unless so is 0=HOG and 1=Viola Jones as commented above?

Yeah, you are right. I missed it really bad 👎 Sorry for that. I will reverse the order of if/unless.

ipa-rmb · 2017-08-01T14:12:03Z

cob_people_detection/ros/launch/people_detection_display.launch

@@ -1,17 +1,21 @@
 <?xml version="1.0"?>

 <launch>
+
+  <arg name="display_results_with_image_view" default="true"/>   <!-- set to false if you do not like to display the camera image with names attached to detected faces within an image_view window -->
+  <arg name="face_positions_topic_name_in" value="/face_detector/face_positions"/>


Did you test this? Does it work with "/" at the beginning? Before it was without.

It is working with both options, but I returned to its original.

ipa-rmb · 2017-08-01T14:13:56Z

cob_people_detection/ros/launch/people_detection.launch

  </group>

-  <!-- remaining launch files -->
-  <param name="data_storage_directory" type="string" value="$(env HOME)/.ros/cob_people_detection/files/"/>
+  <group>


This should be part of above's group <group if="$(arg launch_face_recognizer)">

cagbal · 2017-08-07T07:21:32Z

I pushed some commits according to your comments, Also opened a PR in cob_extern repo.

ipa-rmb · 2017-08-08T09:37:26Z

Looks good. I am going to test the software now.

By the way, did you try whether all dependencies can be installed? Like this:

rosdep install --from-paths src/ -y -i

I am receiving the following error:

ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
cob_people_detection: Cannot locate rosdep definition for [libopenface]

cagbal · 2017-08-08T10:18:44Z

I am testing it now.

…

On Tue, Aug 8, 2017 at 11:37 AM, Richard Bormann ***@***.***> wrote: Looks good. I am going to test the software now. By the way, did you try whether all dependencies can be installed? Like this: rosdep install --from-paths src/ -y -i I am receiving the following error: ERROR: the following packages/stacks could not have their rosdep keys resolved to system dependencies: cob_people_detection: Cannot locate rosdep definition for [libopenface] — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG26LJEI6tY3dDlj0ugnqmJu2QWjvj5Vks5sWCxWgaJpZM4OmT8x> .

ipa-rmb · 2017-08-08T10:22:11Z

Sorry, this is connected to the missing pull request to cob_extern. Please finish that one first, then we can proceed here.

cagbal added 6 commits July 27, 2017 17:08

Added Dlib's HOG based Face detector in Python

55af579

Added script for dependencies of online learning method

633c100

Added online learning method based on Deep Learning

6d44381

recognition_method is now an argument

0ac1331

Added service to toggle online learning

e1ff963

Updated display launch file

a070e79

ipa-rmb requested changes Jul 28, 2017

View reviewed changes

cagbal and others added 2 commits August 1, 2017 13:21

Updated files according to the comments except openface wrapper

614c72e

Corrected typo in comment

a4c42b9

ipa-rmb requested changes Aug 1, 2017

View reviewed changes

cagbal added 4 commits August 1, 2017 16:56

Transferred to cob_extern/libopenface

8cf01cc

Some follow up commits for PR

40122f6

Fixed bug in openface directory name

7e80d41

Disabled imshow inside the code

06c4bcc

Added comment

8739e4c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Online learning dev #5

Online learning dev #5

cagbal commented Jul 28, 2017 •

edited

ipa-rmb left a comment •

edited

ipa-rmb Jul 28, 2017

ipa-rmb Jul 28, 2017

ipa-rmb Jul 28, 2017

ipa-rmb Jul 28, 2017

ipa-rmb Jul 28, 2017 •

edited

ipa-rmb Jul 28, 2017

ipa-rmb Jul 28, 2017

ipa-rmb Jul 28, 2017 •

edited

ipa-rmb Jul 28, 2017

ipa-rmb Jul 28, 2017

ipa-rmb left a comment

ipa-rmb Aug 1, 2017

cagbal Aug 1, 2017

ipa-rmb Aug 1, 2017

cagbal Aug 1, 2017

ipa-rmb Aug 1, 2017

cagbal Aug 1, 2017

ipa-rmb Aug 1, 2017

cagbal Aug 1, 2017

cagbal commented Aug 7, 2017

ipa-rmb commented Aug 8, 2017

cagbal commented Aug 8, 2017 via email

ipa-rmb commented Aug 8, 2017

Online learning dev #5

Are you sure you want to change the base?

Online learning dev #5

Conversation

cagbal commented Jul 28, 2017 • edited

ipa-rmb left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ipa-rmb Jul 28, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ipa-rmb Jul 28, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ipa-rmb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cagbal commented Aug 7, 2017

ipa-rmb commented Aug 8, 2017

cagbal commented Aug 8, 2017 via email

ipa-rmb commented Aug 8, 2017

cagbal commented Jul 28, 2017 •

edited

ipa-rmb left a comment •

edited

ipa-rmb Jul 28, 2017 •

edited

ipa-rmb Jul 28, 2017 •

edited