OE-24 "Revised HighGUI Module" discussion

[vpisarev]

the feature request about evolution proposal OE-24

[paroj]

imgui gives you both OpenGL buttons as well as (optionally embedded) font loading.
There are example backends for OpenGL as well as for DirectX, but why bother with DirectX?

[catree]

GLFW + imgui similar to the approach used by librealsense (realsense-viewer) maybe?

[LaurentBerger]

What do we want to do with highgui module? menu, button, slider, file browser

[LaurentBerger]

do we want a highui module for all platforms? android, ios, linux, windows, opencv.js

[LaurentBerger]

Must we use a third party lib ?

wikipedia list and previous post

gtkmm
FLTK
FOX toolkit
Juce
Qt
wxWidgets
Ultimate++
openscenegraph
imgui
GLFW
vtk
orgre3d

[ebachard]

I confirm ImGui works very well at least on Windows and Linux. Should work on Mac OS X too.

As example, I'm mixing OpenCV + ImGui + OpenGL + SDL2 + NativeFileDialog + AweSomeFont an more in miniDart ( see details here : https://github.com/ebachard/miniDart ).

Last, there are other projects who worth a try:

• cvui : https://github.com/Dovyski/cvui
• canvascv : https://github.com/sagi-z/CanvasCV

[vpisarev]

ImGui is definitely the first thing to look it; probably, it's too much for us, but the license should let us extract some essential parts out of it.

@LaurentBerger@

• menu - no
• button - yes
• slider - already there
• file browser - no
• on the other items - they are actually listed in the proposal

plaforms - just desktop ones. Android and iOS have their own, sometimes strict, guidelines on how to do UI. We are not going to pretend our UI is production-ready. HighGUI is just for experiments. web interface is also out of scope, because things are done quite differently in web.

use of 3rd-party UI toolkits. I would avoid using them as much as we can. Hopefully, we can do everything we want with GTK+ + OpenGL on Linux/BSD, Cocoa + OpenGL on Mac, Win32 + DirectX on Windows.

[vpisarev]

@paroj, DirectX on Windows is much better supported than OpenGL, so, I think, we will need it.

[vpisarev]

@ebachard, thanks for the links! cvui and CanvasCV look quite interesting, indeed!

[patrickelectric]

Any downvote over Qt ? It sounds always a nice thing to use

[damiles]

Another reference based only on GLFW, multiplatform, works on windows linux and macos. Non blocking UI and with buttons and other common ui, zoom images, plot tools, histogram in real time etc.

https://damiles.github.io/OpenCVGUI/

[D-Alex]

Apple is concidering OpenGL and OpenCL as deprecated technologies. At the moment Vulkan seems to be the way to go (on Mac OS Vulkan + https://github.com/KhronosGroup/MoltenVK).

[tompollok]

I vote for using Qt.

Qt works over all major platforms and will support Vulkan on Mac over MoltenVK. Qt has many developers that extend and maintain the library. This is a very importand aspect because you dont want to implement code for a library that only depends on one developer and then when the core developer doesnt fix stuff then your stuck. There are thousands of answered questions on stack overflow and they have a clear documentation. Qt is widely used in the industry for user interfaces like QWidgets or QML that is used in the automobile industry. Also Qt offers (or plans to offer by qt5.12) official python support.

[ebachard]

First over all, I'm sincerily sorry, but I don't know well Viz, and it would be great someone to complete the analysis, and add further information about it, because it's certainly a highly respectable API.

That said, and before to see influence or commercial debate, I'd suggest to think twice about what is awaited, expected, and first answer @vpisarev requests. Reference : https://github.com/opencv/opencv/wiki/OE-24.-Module-HighGUI

BTW, for one of them (better text rendering) I'd suggest to have a look at : http://www.hasper.info/worry-free-text-writing-into-opencv-images/ (user is @PhilLab if I'm not wrong)

Licenses issues (protect the developpers and/or the project ?):

• cvui and Dear ImGui : MIT license (better for OpenCV integration, no hesitation)
• Qt : GPL or LGPL, or expensive SDK for other solutions. This SDK is not an option for little devs like me.

@vpisarev : I'm not sure linking a binary to some LGPL library forces the linked binary to become LGPL too. If I'm not too wrong, you only need to reverse the improvement you did in the LGPL code, in case you did some (and that's much normal imho)

What are the dependencies ?

• Dear ImGui : OpenGL or DirectX or Vulkan soon (if I'm not too wrong). Maybe C++11 but I'm unsure.
• Qt : I don't know well the SDK, but that's a lot of lines of codes.
• cvui : just needs OpenCV. One include one #define , and go !

On this side, IMHO 1. cvui 2. Dear ImGui 3. Qt

Easy to build ?
No hesitation:

• cvui: already there. Only needs some documentation
• Dear ImGui : more difficult, and needs some explanations about Immediate Mode use (but so far, OpenCV is close to that paradigm in most of the proposed examples)

Qt is the last on this side : the API is extremely complete (eveything exists already), but the cost is bigger on the build side.

How does it work : two diffrent schools here. Retained mode vs Immediate mode. Both have pro and cons, and I won't say more, letting people add information on this side.

Binaries size :

• cvui : included in OpenCV, means not easy to quantify, maybe 1MB caused by cvui use, but not much.
• Qt : I did just one or two tries, but we're not far of around 80 MB at least. Said diffrently, what is the size of a simple window, including one button to quit, built as a static binary, and linked to OpenCV ?

Dear ImGui : max 700kB of binary. I'm testing since two years only, but I can say it can be mixed (including event loops) with SDL and ImGui and OpenCV (see: https://framagit.org/ericb/miniDart).

e.g. miniDart in my case, uses Dear ImGui. At the end, I can play vidéos, record (without sound), extract images, and a lot of things using less than 1 MB of binary on Linux (linking with shared libraries) , and ~ 20 MB only as static on Windows. In fact, a lot of @vpisarev requests are included in miniDart (select an area, the zoom -based in Dear ImGui-, record videos and other features (using FFMPEG). It's not written, but once I'll publish the code, it will very probably become MIT one year after or something like that. That's very easy : I wrote everything on the miniDart side (no other contributor) ;-)

Features :

• cvui provides the basics, and needs probably some documentation
• ImGui is more rich + docking and viewports branches are extrelemy promising (see : Docking branch available for testing ocornut/imgui#2109 and Multi-viewports / virtual viewports ocornut/imgui#1542 for further explanation)
• Qt : extrelemy complete, provides everything already. Should work without changes, but I'll let specialists confirm)/

Window Layout:

• Qt is the winner here. Second could be (imho) Dear ImGui, and last cvui (complete, a lot has to be written imho)

Feel free to correct me, and add any constructive information.

Hope this will help in the final choice.

[crackwitz]

(not sure if I should post here or make a new issue...)
I'd like to shift the meaning of the delay argument of waitKey().

first, I'd like to establish -1 as the canonical "infinity" value to pass in, while maintaining <= 0 as infinity.

second, after every use of waitKey(0) (as infinite wait) on the whole internet has disappeared, I would like 0 delay to actually mean zero delay, i.e. polling without a single millisecond of sleep. I've patched the OpenCV I am using to work like that, and it's been working just fine (apart from having to change some 0 to -1 or no argument at all). I also added a short piece of code to disable vsync, which has saved me some execution time as well.

is the first measure something that would be accepted as a pull request?

[alalek]

I'd like to establish -1 as the canonical "infinity" value to pass in, while maintaining <= 0 as infinity.

It make sense for new API. Unfortunately it is too late to change that.

Changing waitKey() itself is API breaking change without a known path of smooth migration for OpenCV Users.

---

mean zero delay

What is about new API for that? spinOnce(), handleEvents(), etc

[crackwitz]

I see the point in requiring a new API for this. I'd suggest pollKey() to carry the meaning in the name parallel to waitKey, i.e. same purpose/function but with different temporal behavior.

I think I can come up with a patch for window_w32 but the other backends might need some digging. in the worst case they'd be equivalent to waitKey(1) until someone with platform-specific knowledge comes along.