imgui UI Automation

[shettynitin87]

Hi,

• I have created an UI for an application using imgui.
• I need to automate and validate that UI.
• Is there any procedure or APIs exposed by imgui to perform UI Automation?
• Thanks in advance for your help and suggestions.

[ocornut]

Sorry there's no way to do any of that at the moment. This would be part of the effort for #435 but I don't have the resource to go in that direction for now.

[shettynitin87]

Hi @ocornut ,

• Thanks for your reply.
• As a work around, I tried to use LDTP tool on the UI but it is not listing my window in the list(which is retrieved using getwindowlist()).
• Is there any way or additional packages that I need to install to use LDTP tool to automate my UI that has been created using imgui?

[ocornut]

I don't know what is LDTP nor how to use it. If you mean Linux Desktop Testing Project I don't think that sort of project would be able to know anything about dear imgui internals, the data isn't exposed anywhere.

[shettynitin87]

Thanks for the update @ocornut
Yes, I meant Linux Desktop Testing Project.

[ocornut]

Well, the answer is that it's not possible to easily automate testing on dear imgui application at the moment.
If you are interested in funding this feature you can contact me privately to discuss it, otherwise it'll have to wait until this becomes a natural priority for me (no ETA) or someone does it. This generally doesn't align very much with current dear imgui use cases, but I appreciate it'll be useful down the line.

[MrSapps]

You won't get windows or anything from tools because of how its rendered... what you could do is something like that "ImGui remoting" project where it also renders ImGui some where else across the network?

But instead of rendering on another machine render to another app on the same machine which draws the window rects using whatever API's allow your testing framework to "see" them?

[ocornut]

I imagine a testing framework would also want to simulate interacting with the app, opening menu xx, clicking button xx, drag stuff around, which imply some support for those things in imgui.

[shettynitin87]

Hi @paulsapps , thanks for your suggestion but as @ocornut mentioned we also need to interact and also validate the interactions with the UI.
Only cloning might not help us with the interaction and validation part.

[MrSapps]

I can't find the issue where the guy posted the remoting stuff, but I recall it might have been bi-directional. Might still be worth a quick look if you can find it.

[ocornut]

@paulsapps I think the issue is that you can't automate things like "find where is button XXX", "active menu item given name", etc. They are totally missing constructs and to seriously interact with an imgui app you'd probably need a bit of a framework to make those things natural and easy to do.
Using remoteimgui or remotepdesktop won't really help there, as I understand the problem.

@shettynitin87 Can we close this issue as agreed it is not happening at the moment, or do you have something else in mind?

[shettynitin87]

sure @ocornut , will close this for now. In case if I am able to find an approach to overcome this, I will reopen and shall update on this thread accordingly.
Thanks for your help.

[shettynitin87]

Hi,

• I have found a work around to automate my application, though a raw approach for now. It serves the purpose.
• Kindly, refer the attached block diagram for more details.
• Following is the detail with respect to each block in the diagram:
  1. xdotool: Refer following link for more information.
  2. Scrot(snapshot): Refer following link for more information.
  3. Image Comparison: Refer following article for comparing 2 images using python-opencv approach.