Skip to content
A tensorflow implementation with SSD model for person detection and Kalman Filtering combined for tracking
Python Jupyter Notebook Shell
Branch: master
Clone or download
Latest commit 7efc08c Jan 23, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.ipynb_checkpoints
__pycache__
anchor_generators
box_coders
builders Version v1 using kalman filter tracking Oct 10, 2018
core
data Version v1 using kalman filter tracking Oct 10, 2018
data_decoders Version v1 using kalman filter tracking Oct 10, 2018
dataset_tools Version v1 using kalman filter tracking Oct 10, 2018
g3doc Version v1 using kalman filter tracking Oct 10, 2018
inference Version v1 using kalman filter tracking Oct 10, 2018
matchers Version v1 using kalman filter tracking Oct 10, 2018
meta_architectures Version v1 using kalman filter tracking Oct 10, 2018
metrics
models Version v1 using kalman filter tracking Oct 10, 2018
protos
samples
ssd_mobilenet_v1_coco_2017_11_17
test_ckpt Version v1 using kalman filter tracking Oct 10, 2018
test_data Version v1 using kalman filter tracking Oct 10, 2018
test_images Version v1 using kalman filter tracking Oct 10, 2018
utils Version v1 using kalman filter tracking Oct 10, 2018
CONTRIBUTING.md Version v1 using kalman filter tracking Oct 10, 2018
LICENSE Create LICENSE Jan 20, 2019
Output.mp4
Person_det_track.py
README.md
__init__.py Version v1 using kalman filter tracking Oct 10, 2018
__init__.pyc
camera.py
detector.py Version v1 using kalman filter tracking Oct 10, 2018
eval.py Version v1 using kalman filter tracking Oct 10, 2018
eval_util.py
eval_util_test.py
evaluator.py
export_inference_graph.py Version v1 using kalman filter tracking Oct 10, 2018
exporter.py
exporter_test.py
helpers.py Version v1 using kalman filter tracking Oct 10, 2018
inputs.py
inputs_test.py Version v1 using kalman filter tracking Oct 10, 2018
jupyter.py
model_hparams.py
model_lib.py Version v1 using kalman filter tracking Oct 10, 2018
model_lib_test.py
model_main.py Version v1 using kalman filter tracking Oct 10, 2018
model_tpu_main.py
person detection and track.gif
tracker.py Version v1 using kalman filter tracking Oct 10, 2018
tracker.pyc
train.py
trainer.py
trainer_test.py Version v1 using kalman filter tracking Oct 10, 2018
visualization_utils.py Version v1 using kalman filter tracking Oct 10, 2018

README.md

Person-Detection-and-Tracking

Introduction

The Project is based on Person Detection and tracking and I am mainly focusing on the Person tracking, if you go through the output gif in the README.md or watch output.mp4 you will be able to see that each person will be provided with an idea as soon as he enters a frame and the number remains with his regardless of the detection happening in concurrent frames. So basically the project focuses on Person Detection and track him as long as he remains in the frame.

Person_det_track.py detects and tracks the person using SSD and Kalman Filter

Requirements

Please try to Stick on with the version provided as much as possible other-wise you will face compatibility issues. I have used the best combination possible during the time of coding.

  • opencv [v3.1]

    • Installation in linux: For complete installation of opencv in ubuntu you can refer here.
    • Installation in windows For complete installation of opencv in windows you can refer here
  • Tensorflow [v1.5.0]

    • Installation Tensorflow has an amazing documentation here

Methodology / Approach

The method Proposed here is divided into 2 main parts

  • Person Detection - The person detection in Real-time is done with the help of Single Shot MultiBox Detector. SSD achieves 75.1% mAP, outperforming a comparable state of the art Faster R-CNN model. and the SSD model is available in the Tensorflow detection zoo. The seamless integration of SSD with tensorflow helps in further optimization and implementation of the algorithm. The SSD object detection composes of 2 parts:

    • Extract feature maps, and
    • Apply convolution filters to detect objects. Even though SSD is capable of detecting multiple objects in the frame, in this project I limited its detection to just human.
  • Person Tracking - Bounding box can be achieved around the object/person by running the Object Detection model in every frame, but this is computationally expensive. The tracking algorithm used here is Kalman Filtering . The Kalman Filter has long been regarded as the optimal solution to many tracking and data prediction tasks. Its use in the analysis of visual motion. The purpose of Filtering is to extract the required information from a signal, ignoring everything else. In this project the Kalman Filter is fed with the velocity, position and direction of the person which helps it to predict the future location of the Person based on his previous data.

The tracking part still faces some problem at the time of an occlusion. (Working on it)

Performance of Code

  • The code was run on both CPU and GPU:
    • Nvidia Quadro 4000 - ~30FPS
    • Nvidia Jetson TX2 - ~20FPS
    • Intel i5 CPU - ~10FPS

Output

Alt Text

Overview / Usage

The system consist of two parts first human detection and secondly tracking. Early research is biased to human recognition rather than tracking. Monitoring the movements of human being raised the need for tracking. Monitoring movements are of high interest in determining the activities of a person and knowing the attention of person. This project focuses on Person Detection and tracking.

Identity retrieval - Tracking of human being can be used as a prior step in biometric face recognition. Keeping continuous track of person will allow to identify person at any time. Thus even if his face identification is not possible at a particular set of frames his identity can be found out. This can be very useful in the case of anomaly detection as the person's face may not face to the camera when an anomaly is detected. So with the help of tracking his identity can be revealed.

Reducing the computation power requirement - A normal objection detection algorithm just detects the Person but do not assign an Id for an Person thus has to be run in every frame to get the bounding box. Tracking will help to reduce the number of times the Detection algorithm has to be run i.e instead of running the Detection algorithm every frame we can run it once in every 5 frames.

Object Detection model failure compensation - there might be some poses where SSD may not detect the person. Even occlusion can affect the detector to a significant level that is where tracking algorithm can be of great help to us.

You can’t perform that action at this time.