This project will mainly use Mediapipe and Django. In the following part we will include the setup/installation for the requirements, explain how the core part of the code works and some bugs left to deal with.
Note that this project can be run under both Windows OS and Ubuntu 20.04. Both versions have similar code, but the setup and performance (can-dos and cannot-dos) are slightly different.
-
Installation: Ubuntu/Windows https://google.github.io/mediapipe/getting_started/install.html. One thing to note that for Windows, one will have to use the specified OpenCV 3.4.10, while for Ubuntu one can just use the newest version for it.
-
Python Solution Installation: This is the solution that we currently use for the running API. For both OS, go to https://google.github.io/mediapipe/getting_started/python.html and do the installation with
pip
accordingly. One can do this in apython virtual environment
or not, this is optional. -
Python ready-to-use Solution: https://google.github.io/mediapipe/solutions/hands.html shows an example script of the calling the hand detector. This will be used as part of the API at
../script/hand_video_detector.py
. -
C++ example: https://google.github.io/mediapipe/getting_started/cpp.html shows the example for buliding the target for detection with C++ and other stuff. This part shall be able to run once we have done part 1, the main installation.
However, I personally have done thehello world example
but fails at thehand detection example
. Here are the links to the related issues when buliding the example C++targets under both Windows OS and Ubuntu 20.04.
Windows: google-ai-edge/mediapipe#2172
Ubuntu 20.04: google-ai-edge/mediapipe#2001 (comment)
Developers can refer to the issues and specifications. -
Further work: For this project, one can use
Graph
andCalculators
of Mediapipe to add more features like hand gestures detection. For more information, one can refer to: https://gist.github.com/TheJLifeX/74958cc59db477a91837244ff598ef4a
-
Installation: Do installation following https://docs.djangoproject.com/en/3.2/intro/install/.
-
If the developer has no experience with Django, he/she is strongly encouraged to take the tutorial before looking into this project, from: https://docs.djangoproject.com/en/3.2/intro/tutorial01/
-
Once this is done, one can look into the Django code with implements
Mediapipe
andcv2
. -
Quick Review: (for the html pages) under
template
,home.html
is the home page.image_upload
andvideo_input
are two other pages written in html format and are triggered by some buttons. These all are based on thebase
one.