Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
IkaLog on Digilent Pynq board (Project Inq)
Pynq is the evaluation platform of Xilinx Zynq APSOC(All Programmable System on Chip). PYNQ-Z1 is the available evaluation board with dual-core ARM processors, 512MB DDR3 Memory, and 2X HDMI Ports. Xilinx also provides Linux image that can be written to micro SD card and boot the board easily.
IkaLog is now functionable with PYNQ-Z1 (as POC.)
Generally, IkaLog needs these goods to run:
- A computer (Windows, Mac, and Linux) to run IkaLog
- An HDMI Capture Device supported by corresponding operating system
- An HDMI video splitter to feed the video to the computer, as well as your TV
- Cables to connect these devices
- And, Wii U console, of course
To obtain these, you may need to spend $300 or more. If you don't have a computer that works with IkaLog, additional thousands of dollars will be spent.
Project Inq aims to run IkaLog on an inexpensive computer.
We do know Raspberry Pi 3 is available today, but video input features are lacking if you want to run IkaLog on the board. Of course, you still need to get a splitter.
Digilent PYNQ-Z1 board has everything we need to run IkaLog. It has dual-core ARM Cortex A9 processor that show better performance compared to RPi 3(on IkaLog workload), and HDMI Input/Output that we have an opportunity to configure those ports as HDMI video capture device and splitter. PL also can be configured as some algorithm accelerators.
Project Inq is still Proof of Concept.
How to run IkaLog on PYNQ-Z1
This procedure is subject to change.
- Write Linux image to a micro SD card. pynq_z1_image_2016_09_14.img should work.
- Insert the micro SD card into Pynq, set the jumper to SD mode. Linux should come up.
- Login to Pynq using id/pass = xilinx/xilinx. The default IP address is 192.168.2.99/24.
- Install dependencies.
pip3 install u-msgpack-python ...
- Clone IkaLog from GitHub.
- Configure IkaConfig.py.
Since Pynq doesn't have X Window system, you should disable
base.bitoverlay used by Jupiter Notebook demonstration. Make sure loading the overlay before running IkaLog. Run python and do (as root):
from pynq import Overlay Overlay("base.bit").download()
- Run IkaLog.py (as root; IkaLog needs to access to HDMI interface)
No! I don't have Wii U here!
dhcp44-82:IkaLog_github hasegaw$ wget https://dl.dropboxusercontent.com/u/14421778/IkaLog/ikalog_sample.mp4 dhcp44-82:IkaLog_github hasegaw$ python3 IkaLog.py -f ikalog_sample.mp4
Build OpenCV with additional HAL
OpenCV 3.1 included in PYNQ doesn't utilize NEON instructions. You need to recompile OpenCV to maximize IkaLog performance. In our test, NEON-enabled OpenCV performs 2X throughput in color conversion functions.
312 git clone --depth 1 https://github.com/Itseez/opencv.git 313 git clone --depth 1 https://github.com/Itseez/opencv_contrib.git 314 cd opencv 315 cd build/ 316 cmake -D CMAKE_INSTALL_PREFIX=$HOME/local/ -D CMAKE_BUILD_TYPE=RELEASE \ -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \ -D BUILD_opencv_python3=ON -D BUILD_opencv_python2=OFF \ -D WITH_CUDA=OFF -D WITH_NEON=ON -D BUILD_EXAMPLES=OFF .. 317 make -j3 build && make install
The build process takes several hours to complete. Run the build before you sleep.
cat << EOF >> ~/.bash_profile (or somewhere) export LD_LIBRARY_PATH=$HOME/local/lib/:$HOME/local/lib/python3.4/dist-packages/ export PYTHONPATH=.:$HOME/local/lib/python3.4/dist-packages/ EOF
Build IkaMatcher HAL for IkaLog
On some branches we have proof of concept of NEON acceleration (though we have more overhead on Python ctypes layer right now)
cd lib make
How to switch HAL used by IkaMatcher2?
How to profile IkaLog?
python3 -m cProfile -s tottime IkaLog.py -f ikalog_sample.mp4 | tee output.txt
The opportunity of HW H.264 decoder on Raspberry Pi
If you are trying to process video files on the filesystem, cv2.VideoCapture.read() functions will be the significant bottleneck. It shows 50% of the runtime is consumed by H.264 decoder.
In Raspberry Pi, HW H.264 decoder can be enabled by separately sold IP license. The hardware decoder seems to be work with GStreamer. Since we already support GStreamer source, IkaLog should be able to offload the decode process to HW decoder on Raspberry Pi.
See our Linux page in this wiki to get IkaLog works with GStreamer. This is still an idea and not proven.
Consideration about PYNQ default bitstream
Have a fun and hack!
Any feedbacks, pull requests are appreciated. If you are going to work on Project Inq, please contact hasegaw join IkaLog slack to maximize our bandwidth and collaborate.