Skip to content

A C++ module for Object Detection and tracking (in this case humans are detected, but 80 classes from COCO dataset can be detected). The coordinates of all the objects detected in the camera frame are transformed into robot reference frame for further processing.

License

Notifications You must be signed in to change notification settings

mjoshi07/Human-Tracker-Bot

Repository files navigation

Acme-Robotics-Human-Detector-Tracker

Build Status Coverage Status License: MIT

A C++ module for Object Detection and tracking (in this case humans are detected, but 80 classes from COCO dataset can be detected). The coordinates of all the objects detected in the camera frame are transformed into robot reference frame for further processing.

Project contibutors

  1. Naitri Rajyaguru, Robotics Graduate Student at University of Maryland, College Park.
  2. Mayank Joshi, Robotics Graduate Student at University of Maryland, College Park.

Overview

Human detection is a part of object detection where primary class “human/person” is detected in images or video frames. In robotics, human detection is very necessary as humans are dynamic obstacles and no one would want a robot colliding with a human. Therefore, for navigation, a robot needs to detect and identify humans in its field of view.

We propose a human detection and tracking module, which uses computer vision algorithms to perceive the environment. The module uses a monocular camera to get frames in which humans are first detected by a deep neural network model and then those are tracked till they are present in the frame. Alot of research has been done in the field of object detection, especially human detection. Although quite a lot algorithms are quite efficient, like HOG (Histogram of Oriented Gradients) but they are not as robust as a deep neural network. Now days we have processing power that could run DNN model in real time and the accuracy of the model can be increased by training it on more data, therefore a scope of improvement is always present. Our module also has the capability to obtain training data and check the performance of DNN model on the test data, which can be provided by the user.

We plan to use YoloV4-tiny, a pre-trained model based on Darknet framework.

image

Image taken from this article

Deliverables

  • Project: Acme Robotics Human(s) obstacle detector and tracker in robot reference frame
  • Overview of prosposed work, including risks, mitigation, timeline
  • UML and activity diagrams
  • Travis code coverage setup with Coveralls
  • Developer-level documentation

Potential Risks and Mitigation

  • The model is trained on MS COCO dataset which contains RGB images therefore would not able to work with infrared cameras or in low light. Training on additional low light and data from infrared camera can make the model more robust
  • Since yoloV4-tiny is the compressed version of yoloV4, therefore it lacks in accuracy. YoloV4 achieves an average precision of 64.9% while yoloV4-tiny achieves an average precision of 40.2%. Check full comparison here
  • Running the system on the robot can memory expensive due to deep learning model, one possible solution is to quantize the weights of the model so that it takes less memory and is faster

UML Diagram

  • The class dependency diagram of the proposed design:

image

  • Activity diagram image

  • Quadchart and Proposal of 2 pages can be found here

Dependencies with licenses

  • OpenCV 4.5.0 (covered under the open-source Apache 2 License)
  • Eigen 3.4 the Mozilla Public License 2.0
  • GTest BSD 3-Clause "New" or "Revised" License

Install OpenCV 4.5.0 and other dependencies using the following command found in Acme-Robotics-Human-Tracker directory

sh requirements.sh

Build Instructions

With the following steps you can clone this repository in you local machine and build it.

git clone --recursive https://github.com/mjoshi07/Acme-Robotics-Human-Tracker
cd Acme-Robotics-Human-Tracker
mkdir build
cd build
cmake ..
make

To run tests of this module, use the following command

 ./tests/human-tracker-test

To use, webcam uncomment the line in Autobot.cpp #216 and comment line #219

./app/human-tracker

Results

  • Detection output of running the detector on some random images with known ground truth
  • Green Bounding Box represents the GROUND TRUTH and Red Bounding Box represents the DETECTOR Output image
  • The results on live webcame image
  • Human location in ROBOT REFERENCE FRAME image

Metrics

  • We used Intersection over Union as a metric for our detector as it is a better metric than measuring error between centroids image image
  • IOU takes into consideration the height and width of the predicted bounding box whereas centroid shift only takes into consideration the location of centroid of the bounding box which is an ambigous metric as displayed below image
  • Centroid of the predicted bounding box highlighted in Red color is same for both images and also coincides with the centroid of the Ground Truth bounding box highlighted in Green Color but it is quite obvious that both the predicted bounding boxes are quite inaccurate

Technology

  • We will be following AIP (Agile Iterative Process) and implement the software using TDD (Test Driven Development)
  • Quality shall be ensured by unit testing and cppcheck, adhering to Google C++ style guide
  • Operating System (Ubuntu 18.04 LTS)
  • Programming Language (Modern C++)
  • Build System (CMake >= 3.2.1)
  • External Library (Opencv >= 4.5.0)
  • Pre-trained model (yoloV4-tiny)

Development Process

Following the Agile Iterative Process for Development, we switch roles of driver and navigator. Product backlog, iteration backlog and worklog can be found here and Sprint planning with review notes can be found here

Generate Doxygen Documentation

To install doxygen run the following command:

sudo apt-get install doxygen

Now from the cloned directory run:

cd docs
doxygen Doxyfile

Generated doxygen files are in html format and you can find them in ./docs folder. With the following command

cd docs
cd html
google-chrome index.html

Running Cppcheck and Cpplint

Run cppcheck: Results are stored in ./results/cppcheck.txt

sh run_cppcheck.sh

Run cpplint: Results are stored in ./results/cpplinttxt

sh run_cpplint.sh

About

A C++ module for Object Detection and tracking (in this case humans are detected, but 80 classes from COCO dataset can be detected). The coordinates of all the objects detected in the camera frame are transformed into robot reference frame for further processing.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published