Privacy-Preserving Robot Vision with Anonymized Faces by Extreme Low Resolution (ORB-SLAM2), IROS'19

By Myeung-Un Kim, Harim Lee, Hyun Jong Yang and Michael S. Ryoo.

Abstract

Our proposed camera system detects privacy-sensitive blocks, i.e., human face, from extreme low resolution (LR) images, and then dynamically enhances the resolution of only privacy-insensitive blocks, e.g., backgrounds. Keeping all the face blocks to be extreme LR of 15x15 pixels, we can guarantee that human faces are never at high resolution (HR) in any of processing or memory, thus yielding strong privacy protection even from cracking or backdoors.

Our camera system produces an image on a real-time basis, the human faces of which are in extreme LR while the backgrounds are in HR. We experimentally confirm that our proposed face detection camera system outperforms the state-of-the-art small face detection algorithm, while the robot performs ORB-SLAM2 well even with videos of extreme LR faces. Therefore, with the proposed system, we do not too much sacrifice robot perception performance to protect privacy.

[Paper]

If you find our work useful in your research, please consider cite:

@inproceedings{kim-iros2019,
  title = {Privacy-Preserving Robot Vision with Anonymized Faces by Extreme Low Resolution},
  author = {Myeung Un Kim and Harim Lee and Hyun Jong Yang and Micheal S. Ryoo},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year = {2019}
}

Feel free to contact Myeung Un Kim (myoungunkim92@gmail.com) if you have any questions!

Some Results

Result of ORB-SLAM2 using high resolution video (top) and our privacy-preserving face detection video (bottom)

As shown in the results on the right, we can extract enough features for ORB-SLAM2 in privacy-preserving face detection video. Full video is available at the following link. [Youtube]

How to use it

Preparation

First of all, clone the code.

git clone https://github.com/myeungun/ORB-SLAM2.git

Composition of the system

1. On Xavier

• To run ZED camera

roslaunch zed_wrapper zed.launch

• To run our face detection network and publish the results

rosrun preserve-network network_multiple_bounding_with_ZED_stereo_odometry.py

or

rosrun preserve-network network_with_ZED_for_stereo_odometry

• To run ORB-SLAM2 and publish the results

roslaunch orb_slam2_ros orb_slam2_zed_stereo_with_network.launch

2. On the remote PC

• To run ROS

roscore

• To receive a video from Xavier

rosrun preserve-network result_subscribe_view_with_Buffer.py

or

rosrun orb_slam2_hr orb_slam2_subscriber.py

First one is only for face detection network results, and second one is for visual odometry results by ORB_SLAM2.

• To remotely control Turtlebot

roslaunch turtlebot3_teleop turtlebot3_teleop_key.launch

3. On Turtlebot raspberry pi

• To remotely control Turtlebot

roslaunch turtlebot3_bringup turtlebot3_robot.launch

• To match time synch, it is necessary to be connected to the Internet.

sudo ntpdate 1.ro.pool.ntp.org
sudo timedatectl set-timezone Asia/Seoul

Setting files

Location of ORB-SLAM2 Setting files

• ZED Camera

/src/orb_slam_2_ros/orb_slam2/config

• Launch file

/src/orb_slam_2_ros/ros/launch

There are two things to consider carefully in this launch file:

1. Specifying which "topic" to "subscribe"
2. Specifying which "camera" to use
   "orb_slam2_zed_stereo_with_network.launch" file is configured as follows:
3. "subscribe" the results of "face detection network"

<remap from="/image_left/image_color_rect" to="/preserve_network/left/result_only_Network" />
<remap from="/image_right/image_color_rect" to="preserve_network/right/result_only_Network" />

2. use "ZED camera"

$(find orb_slam2_ros)/orb_slam2/config/ZedStereoHD.yaml”

Location of ZED camera Setting files

/src/zed-ros-wrapper/zed_wrapper/launch/zed.launch

Considerations in the above file are the resolution and FPS of the video from the ZED camera.

<arg name=”resolution”	default=”2” />
<arg name=”frame_rate”	default=”30” />