This project was created during the last sommer 2021 semester as I took a Computer Vision class at my university with the main usage point: to use it as a practical part of my future Bachelor Thesis.
After reading https://rdmilligan.wordpress.com/2015/10/15/augmented-reality-using-opencv-opengl-and-blender/ I was quite surprized how simple that could be. But since I wanted to use this technology from any smartphone there were necessary some improvements.
The whole idea of streaming OpenGL objects over a network was to grab its picture from the buffer right before it goes on the PC screen. After creating the OpenGL texture in init with glTexImage2D all further work with texture was done withing glTexSubImage2D in order to not overflow the GPU memory during the long usage of the programm. The part with AruCo marker detection was completely rewritten. Thanks to Flask web application framework the collected from buffer image of an OpenGL texture was send over the local network withing reading pixels using glReadPixels and then converting this JPEG image into a binary string that one can send to the browser via HTTP. For more details how Flask manages this exercise please have a look at https://www.pyimagesearch.com/2019/09/02/opencv-stream-video-to-web-browser-html-page/
The video was recorded on Android smartphone during opening web browser with this stream on it one can see bellow:
IMG_2836.mov
python3 -m venv env source env/bin/activate python3 -m pip install requests.txt
-then please make sure you've installed DroidCam app on you smartphone (in order you want to use your smartphone camera) and change the IP adress in WebcamVideoStream() function to IP address from DroidCam app. -then you need to print Aruco markers from working directory (or you may simply open them on your device)
and you are ready to go: just type in your console python3 main_flask.py and target your camera on the Aruco marker to see how it works. You will see on the screen the IP address
which you could open from any device in your local network to see this streaming. (please make aware there were predefined only 2 markers DICT_4X4_50_id24 and DICT_4X4_50_id5! In case you want to try use other markers you need to make some changes in the code. Same as you want to use other 3d objects (.obj) -> first predefine according to comments in the source code.
MIT License
Copyright (c) 2021 Alexey Obukhov
Thanks for your attention and please don't hesitate to send me any advise, questions or possible improvements.