Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertion failed: g.IO.KeyMods with GLFW backend #3575

Closed
michaelquigley opened this issue Nov 4, 2020 · 29 comments
Closed

Assertion failed: g.IO.KeyMods with GLFW backend #3575

michaelquigley opened this issue Nov 4, 2020 · 29 comments

Comments

@michaelquigley
Copy link

michaelquigley commented Nov 4, 2020

Dear ImGui 1.77 (17700)
--------------------------------
sizeof(size_t): 8, sizeof(ImDrawIdx): 4, sizeof(ImDrawVert): 20
define: __cplusplus=199711
define: _WIN32
define: _WIN64
define: _MSC_VER=1926
--------------------------------
io.BackendPlatformName: imgui_impl_glfw
io.BackendRendererName: imgui_impl_opengl3
io.ConfigFlags: 0x00000000
io.ConfigInputTextCursorBlink
io.ConfigWindowsResizeFromEdges
io.ConfigWindowsMemoryCompactTimer = 60.0f
io.BackendFlags: 0x0000000E
 HasMouseCursors
 HasSetMousePos
 RendererHasVtxOffset
--------------------------------
io.Fonts: 1 fonts, Flags: 0x00000000, TexSize: 512,64
io.DisplaySize: 1280.00,720.00
io.DisplayFramebufferScale: 1.00,1.00
--------------------------------
style.WindowPadding: 8.00,8.00
style.WindowBorderSize: 1.00
style.FramePadding: 4.00,3.00
style.FrameRounding: 0.00
style.FrameBorderSize: 0.00
style.ItemSpacing: 8.00,4.00
style.ItemInnerSpacing: 4.00,4.00

I'm running into something that I can't seem to get to the bottom of, and I'm hoping you can help. I'm using a framework on top of ImGui, which implements key bindings and actions. I'm running into a situation where when I execute an action from a key binding, and then I launch a native file dialog, I'm running into an assert in ImGui when the dialog returns control back to ImGui.

I get the following assert:

Assertion failed: g.IO.KeyMods == expected_key_mod_flags && "Mismatching io.KeyCtrl/io.KeyShift/io.KeyAlt/io.KeySuper vs io.KeyMods", file ..\imgui\imgui.cpp, line 6765

My code is structured like this:

inline void host::draw(const ImVec2& size) {
    ImGui_ImplOpenGL3_NewFrame();
    ImGui_ImplGlfw_NewFrame();
    ImGui::NewFrame();

    _app->draw(ImGui::GetIO().DisplaySize);
    _app->respond();

    ImGui::Render();
    glViewport(0, 0, size.x, size.y);
    glClearColor(_bg.x, _bg.y, _bg.z, _bg.w);
    glClear(GL_COLOR_BUFFER_BIT);
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
    glfwSwapBuffers(_window);
}

void workspace_app::respond() {
    auto io = ImGui::GetIO();
    if(!io.WantCaptureKeyboard) {
        for(const auto& action : _actions) {
            if(action->triggered(io)) {
                action->execute();
                return;
            }
        }
    }
}

bool action::triggered(const ImGuiIO& io) const {
    return _key_binding->triggered(io);
}

bool key_binding::triggered(const ImGuiIO& io) const {
    return (io.KeyCtrl == ctrl && io.KeyAlt == alt && io.KeyShift == shift && ImGui::IsKeyPressed(key));
}

The workspace_app::respond is the concrete overload responding to the _app->respond() line in host::draw above.

When I invoke this same code path through clicking on a user interface element, the code path works perfectly. It only triggers this assert when I use the keyboard shortcut (CTRL-S) in my case, to invoke the code path. If I change my keyboard shortcut to just S instead of CTRL-S, it also works perfectly. It only seems to hit this assert when modifier keys are involved.

Any insight would be very appreciated.

@ocornut
Copy link
Owner

ocornut commented Nov 4, 2020

Hello,

The assert generally triggers when you have code modifying the io.KeyCtrl io.KeyAlt or io.KeyShift modifiers after ImGui::NewFrame(). I think you should search in your codebase for such modifications.

@ocornut
Copy link
Owner

ocornut commented Nov 4, 2020

Note the comment exactly above this assert:

// Verify that io.KeyXXX fields haven't been tampered with. Key mods should not be modified between NewFrame() and EndFrame()
// One possible reason leading to this assert is that your backends update inputs _AFTER_ NewFrame().
const ImGuiKeyModFlags expected_key_mod_flags = GetMergedKeyModFlags();
IM_ASSERT(g.IO.KeyMods == expected_key_mod_flags && "Mismatching io.KeyCtrl/io.KeyShift/io.KeyAlt/io.KeySuper vs io.KeyMods");

@michaelquigley
Copy link
Author

Yes. I already saw that. But my code does is not directly modify those values at all. That was the first thing I looked for.

@michaelquigley
Copy link
Author

michaelquigley commented Nov 4, 2020

In fact, if I add:

        auto io = ImGui::GetIO();
        auto mods = io.KeyMods;
        ImGui::NewFrame();

and after my code:

        ImGui::GetIO().KeyMods = mods;
        ImGui::Render();

I still get the same assertion.

I also tried capturing those fields after NewFrame:

        ImGui::NewFrame();
        auto io = ImGui::GetIO();
        auto mods = io.KeyMods;
        auto ctrl = io.KeyCtrl;
        auto alt = io.KeyAlt;
        auto shift = io.KeyShift;

Same result.

Could something be occuring because the dialog is blocking this loop for a period of seconds (or more)?

@ocornut
Copy link
Owner

ocornut commented Nov 4, 2020

Well, something does and it's not core imgui.
The only piece of code that you are running which intentionally write to io.KeyXXX are the key handlers in imgui_impl_glfw.cpp which are generally triggerd when your app calls glfwPollEvents().

Could something be occuring because the dialog is blocking this loop for a period of seconds (or more)?

Yes if you keep polling GLFW events, as stated above.

I still get the same assertion.

Because that's making it wrong. If you move the first mods = assignment after ImGui::NewFrame() that's a correct check.

@michaelquigley
Copy link
Author

michaelquigley commented Nov 4, 2020

I don't see how anything in my code could be modifying ImGui::GetI().KeyMods within the following block, yet the assert persists:

inline void host::draw(const ImVec2& size) {
    _size = size;

    ImGui_ImplOpenGL3_NewFrame();
    ImGui_ImplGlfw_NewFrame();
    ImGui::NewFrame();
    auto saved_mods = ImGui::GetIO().KeyMods;
    
    _app->draw(ImGui::GetIO().DisplaySize);
    _app->respond();

    ImGui::GetIO().KeyMods = saved_mods;
    ImGui::Render();
    glViewport(0, 0, size.x, size.y);
    glClearColor(_bg.x, _bg.y, _bg.z, _bg.w);
    glClear(GL_COLOR_BUFFER_BIT);
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
    glfwSwapBuffers(_window);
}

And keyboard shortcuts work delightfully everywhere other than when I launch a native file dialog, delaying this loop.

Code down the stack, below _app->respond() is launching a native file dialog. The entire loop is blocked when that happens. When it returns, it's throwing the assert.

@ocornut
Copy link
Owner

ocornut commented Nov 4, 2020 via email

@michaelquigley
Copy link
Author

The following code resolves the issue. I'll take it from here. Thank you.

inline void host::draw(const ImVec2& size) {
    _size = size;

    ImGui_ImplOpenGL3_NewFrame();
    ImGui_ImplGlfw_NewFrame();
    ImGui::NewFrame();
    auto saved_ctrl = ImGui::GetIO().KeyCtrl;
    auto saved_alt = ImGui::GetIO().KeyAlt;
    auto saved_shift = ImGui::GetIO().KeyShift;
    auto saved_super = ImGui::GetIO().KeySuper;

    _app->draw(ImGui::GetIO().DisplaySize);
    _app->respond();
 
    ImGui::GetIO().KeyCtrl = saved_ctrl;
    ImGui::GetIO().KeyAlt = saved_alt;
    ImGui::GetIO().KeyShift = saved_shift;
    ImGui::GetIO().KeySuper = saved_super;
    ImGui::Render();
    glViewport(0, 0, size.x, size.y);
    glClearColor(_bg.x, _bg.y, _bg.z, _bg.w);
    glClear(GL_COLOR_BUFFER_BIT);
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
    glfwSwapBuffers(_window);
}

@michaelquigley
Copy link
Author

(Also, thank you for the help... Surely you can understand how these things go. My code doesn't directly alter those values. There's got to be some kind of a compiler side effect or some minor mistake or something. And I'm kind of puzzled how this only happens in this one specific case... but yes, you're right, something is changing those fields between those 2 statements, and it's ultimately my problem... But thanks!)

@ocornut
Copy link
Owner

ocornut commented Nov 10, 2020

As a quick test,

I've tried today to use https://github.com/mlabbe/nativefiledialog to create a long blocking dialog in the middle of the Dear ImGui frame, calling it while modifiers were held, and tested this with GLFW, SDL and Win32 backends without a problem.

Maybe your and @str0yd app have some kind of reentrant/threaded code calling GLFW back-end events during the blocking call. I'm not sure.

@str0yd
Copy link

str0yd commented Nov 10, 2020

This is strange. I use my own key callback:

void key_callback(GLFWwindow* window, GLint key, GLint scancode, GLint action, GLint mode)
{
	if (!ImGui::GetIO().WantCaptureKeyboard) 
	{ // event not handled by gui
		if (key >= 0 && key < 1024)
		{
			if (action == GLFW_PRESS)
				sps_visual.keys[key] = GL_TRUE;
			else if (action == GLFW_RELEASE) {
				sps_visual.keys[key] = GL_FALSE;
				sps_visual.keys_processed[key] = GL_FALSE;
			}
		}
	}
}

I first set all my callbacks:

glfwSetKeyCallback(window, key_callback);
glfwSetCharModsCallback(window, char_callback);
glfwSetMouseButtonCallback(window, mouse_click_callback);
glfwSetScrollCallback(window, scroll_callback);
glfwSetCursorPosCallback(window, mouse_callback);

Then I initialize ImGui:

ImGui_ImplGlfw_InitForOpenGL(window, true);

Usually ImGui should call my own key callback function at the right time, shouldn't it?

I don't use the callbacks anywhere else.

Could it maybe be something with the Windows Openfile dialog? I use the GetOpenFileNameA function: https://docs.microsoft.com/en-us/windows/win32/api/commdlg/nf-commdlg-getopenfilenamea

@michaelquigley
Copy link
Author

As a quick test,

I've tried today to use https://github.com/mlabbe/nativefiledialog to create a long blocking dialog in the middle of the Dear ImGui frame, calling it while modifiers were held, and tested this with GLFW, SDL and Win32 backends without a problem.

Maybe your and @str0yd app have some kind of reentrant/threaded code calling GLFW back-end events during the blocking call. I'm not sure.

I definitely do not have any threaded or re-entrant code as part of my application, and certainly none that interact with GLFW directly.

I do have a callback registered with GLFW to invoke my draw loop when the host window gets resized. That draw loop contains this code:

ImGui_ImplOpenGL3_NewFrame();
ImGui_ImplGlfw_NewFrame();
ImGui::NewFrame();

ImGuiIO& io = ImGui::GetIO();
auto saved_ctrl = io.KeyCtrl;
auto saved_alt = io.KeyAlt;
auto saved_shift = io.KeyShift;
auto saved_super = io.KeySuper;

// Application Drawing Here

io.KeyCtrl = saved_ctrl;
io.KeyAlt = saved_alt;
io.KeyShift = saved_shift;
io.KeySuper = saved_super;
ImGui::Render();
glViewport(0, 0, size.x, size.y);
glClearColor(_bg.x, _bg.y, _bg.z, _bg.w);
glClear(GL_COLOR_BUFFER_BIT);
ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
glfwSwapBuffers(_window);

But I'm pretty sure that's not causing this issue.

I've left my workaround in place because it doesn't seem to be adversely affecting the behavior of any part of my application. I've been through my codebase extensively... there does not seem to be anything that's directly changing the state of any ImGui structures in any way. Reading them... extensively. Changing them... no.

@ocornut
Copy link
Owner

ocornut commented Nov 10, 2020 via email

@str0yd
Copy link

str0yd commented Nov 10, 2020

I had no time to test it yet, will do this tomorrow.

@str0yd
Copy link

str0yd commented Nov 11, 2020

I think the problem is in the ImGui_ImplGlfw_KeyCallback. I press "Ctrl + S" to quicksave my things. While pressing "Ctrl + S" the Windows file dialog opens. I relase "Ctrl + S" and choose a file. As soon as I leave the file dialog the Keykallback is triggered (I relased Ctrl and S) and resets the io.KeyCtrl. After resetting the program runs and ASSERTS

g.IO.KeyMods == expected_key_mod_flags // same as GetMergedKeyModFlags()
// g.IO.KeyMods is 1, expected_key_mod_flags  is 0

This evaluates FALSE because the ImGui_ImplGlfw_KeyCallback has already reset the Ctrl flag (g.IO.KeyMods is set at the beginning of the frame).

I think this is a bug beacause if you could press and release the Ctrl button in one frame you should also get this error message.

I'm not sure when exactly GLFW calls the kallback function. If it is periodically or inerrupting. But if I'm right with my theory here it should call the callback interrupting.

@ocornut
Copy link
Owner

ocornut commented Nov 11, 2020

I'm not sure when exactly GLFW calls the kallback function

You can set a breakpoint to find out but I've also written it above. They are called when you call glfwPollEvents() and the point is your should not call glfwPollEvents() in the middle of an ImGui frame, so you should look at where you are calling it.

@michaelquigley
Copy link
Author

michaelquigley commented Nov 11, 2020

You can set a breakpoint to find out but I've also written it above. They are called when you call glfwPollEvents() and the point is your should not call glfwPollEvents() in the middle of an ImGui frame, so you should look at where you are calling it.

My code does not call glfwPollEvents anywhere, ever, between ImGui::NewFrame and ImGui::Render, and still has this issue.

Maybe there's some sort of strange side-effect happening in GLFW when opening a dialog, that's causing that callback to execute despite not calling glfwPollEvents? Unfortunately it's not super easy for me to fixture this up in a debugger in a way that makes this easy to isolate.

@str0yd
Copy link

str0yd commented Nov 11, 2020

I'm debugging the code right now. My main loop calls the pollevents before any imgui stuff:

glfwPollEvents();

...

// initialize GUI for new frame
ImGui_ImplOpenGL3_NewFrame();
ImGui_ImplGlfw_NewFrame();
// draw GUI
ImGui::NewFrame();
// get all the gui stuff
VisualGui::getInstance().DrawGui();
// render the gui
ImGui::Render();
ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());

...

I do not call the pollEvents anywhere else in my program.

Here is an image of the callstack:
grafik
As you can see the breakpoint is in the ImGui_ImplGlfw_KeyCallback function. I come from extern code. The curser (green arrow) points in my function VisualGui::openFile. In this function the curser is in the

iResult = GetSaveFileNameA(&ofn);

grafik

Sooo it looks like the GetSaveFileNameA (a Windows function) calls the glfwPollEvents.

@michaelquigley
Copy link
Author

glfwPollEvents();

...

// initialize GUI for new frame
ImGui_ImplOpenGL3_NewFrame();
ImGui_ImplGlfw_NewFrame();
// draw GUI
ImGui::NewFrame();
// get all the gui stuff
VisualGui::getInstance().DrawGui();
// render the gui
ImGui::Render();
ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());

...

I do not call the pollEvents anywhere else in my program.

My code is structured exactly the same.

@ocornut
Copy link
Owner

ocornut commented Nov 11, 2020

I looked at GLFW codebase and at a first glance it seems like only the _glfwPlatformPollEvents() function dispatch win32 messages which turns into callback events.

I have created a minimum repro calling GetSaveFileNameA() in the middle of the dear imgui frame and couldn't get things to trigger.

As this point if you would like to correctly get to the end of this, could you try creating a minimum repro for it? Ideally sticking some code inside existing examples/ app.

@ocornut ocornut changed the title Assertion failed: g.IO.KeyMods Assertion failed: g.IO.KeyMods with GLFW backend Nov 11, 2020
@michaelquigley
Copy link
Author

I can probably put something together this weekend.

@ocornut
Copy link
Owner

ocornut commented Nov 11, 2020

OK so weirdly I tried to build a repro in another sample project and it worked this time...

In examples/example_glfw_opengl3/main.cpp


#include <windows.h>
#include <commdlg.h>
#include <GLFW/glfw3.h>
#define GLFW_EXPOSE_NATIVE_WIN32
#include <GLFW/glfw3native.h>   // for glfwGetWin32Window

[...]

if (ImGui::IsKeyPressed(GLFW_KEY_O))
{
    CHAR buf[256] = "hello.txt";
    OPENFILENAME of;
    memset(&of, 0, sizeof(of));
    of.lStructSize = sizeof(OPENFILENAME);
    of.hwndOwner = NULL;
    of.lpstrFilter = NULL;
    of.lpstrFile = buf;
    of.hwndOwner = glfwGetWin32Window(window);
    of.nMaxFile = 256;
    of.lpstrFileTitle = NULL;
    of.nMaxFileTitle = 0;
    of.lpstrInitialDir = (LPSTR)NULL;
    of.Flags = OFN_SHOWHELP | OFN_OVERWRITEPROMPT;
    of.lpstrTitle = "lpstrTitle";
    ::GetSaveFileName(&of);
}

This repros it for me.

Installed GLFW with debug symbols
# vcpkg install glfw3:static

With callstack:

>	example_glfw_opengl3.exe!__glfwInputKey�()	C
 	example_glfw_opengl3.exe!__glfwInputWindowFocus�()	C
 	example_glfw_opengl3.exe!_glfwGetWin32Window�()	C
 	user32.dll!__InternalCallWinProc@20�()	Unknown
 	user32.dll!UserCallWinProcCheckWow()	Unknown
 	user32.dll!CallWindowProcAorW(long (*)(struct HWND__ *,unsigned int,unsigned int,long),struct HWND__ *,enum _WM_VALUE,unsigned int,long,int)	Unknown
 	user32.dll!_CallWindowProcW@20�()	Unknown
 	opengl32.dll!_wglWndProc@16�()	Unknown
 	user32.dll!__InternalCallWinProc@20�()	Unknown
 	user32.dll!UserCallWinProcCheckWow()	Unknown
 	user32.dll!DispatchClientMessage()	Unknown
 	user32.dll!___fnDWORD@4�()	Unknown
 	ntdll.dll!_KiUserCallbackDispatcher@12�()	Unknown
 	comdlg32.dll!_InvokeNewFileOpenSave(struct IFileDialog *,unsigned short,struct HWND__ *,struct _OFNINITINFO *,struct HWND__ *)	Unknown
 	comdlg32.dll!_CreateNewFileOpenSaveInProc(unsigned short,struct HWND__ *,struct _OFNINITINFO *)	Unknown
 	comdlg32.dll!NewGetFileName(struct OPENFILEINFO *,int)	Unknown
 	comdlg32.dll!_GetFileName@8�()	Unknown
 	comdlg32.dll!_GenericGetFileNameA@8�()	Unknown
 	comdlg32.dll!_GetSaveFileNameA@4�()	Unknown
 	example_glfw_opengl3.exe!main(int __formal, char * * __formal) Line 182	C++

@ocornut
Copy link
Owner

ocornut commented Nov 11, 2020

So GLFW is clearing keyboard input when the windows loses focus, and with those native modal dialog the events are pushed mid-frame.

With the current direct-write to IO i don't have a super satisfying solution right now. When we transitioning to queuing events (e.g. https://gist.github.com/ocornut/8417344f3506790304742b07887adf9f) it'll be trivial.

Considering the nature of that mid-frame modification my suggested solution would be to make the assert more lenient by only asserting on mid-frame presses rather than mid-frame release. This way, the assert would STILL catch on the common error it's trying to catch, while acting friendly on this sort of situation we stumbled on here.

ocornut added a commit that referenced this issue Nov 11, 2020
@ocornut
Copy link
Owner

ocornut commented Nov 11, 2020

Commit with this fix: 3600ceb

Commit where the check was first added: ccf0cc8
It's an important check as a frequent error was to call events after ImGui::NewFrame() and while things would appear to mostly work, some of the nav queries done in NewFrame would essentially break.

Thanks all for your patience with this, glad we got this through.

@ocornut ocornut closed this as completed Nov 11, 2020
ocornut added a commit that referenced this issue Nov 11, 2020
@michaelquigley
Copy link
Author

Confirmed. This fixes the issue in my codebase. Thank you!

kabergstrom added a commit to kabergstrom/renderer_prototype that referenced this issue Mar 28, 2021
Process imgui input before beginning imgui frame (ocornut/imgui#3575)
aclysma pushed a commit to aclysma/rafx that referenced this issue Mar 28, 2021
* Add a few different tonemapper options
* Process imgui input before beginning imgui frame (ocornut/imgui#3575)
* Fix issue with internal #![rustfmt::skip] attribute on nightly
vug added a commit to vug/pyren that referenced this issue Jul 26, 2023
Otherwise, pressing control or alt key crashes the app triggering an assertion in imgui.
See ocornut/imgui#3575 Shouldn't poll events after ImGui starts
yyc-git added a commit to Meta3D-Technology/Meta3D that referenced this issue Dec 26, 2023
…e may cause Assertion failed: g.IO.KeyMods error" bug

reason
enter chinese may modifying the io.KeyCtrl io.KeyAlt or io.KeyShift modifiers

refer to:
ocornut/imgui#3575

solution
now catch error;
label now change to update when blur;
@phynae
Copy link

phynae commented Mar 30, 2024

I have a similar issue. When I press Ctrl + S the following error pops up. My BeginFrame -> EndFrame code looks like this.

I am using dx11 and win32 backends.
image
image

@ocornut
Copy link
Owner

ocornut commented Mar 31, 2024

I have a similar issue. When I press Ctrl + S the following error pops up. My BeginFrame -> EndFrame code looks like this.

What’s in the draw() function then?
Try to narrow it down to a minimum repro.

I don’t think the 2020 issue is related anymore, as nowadays events are queued so even if a backend receives event’s asynchronously it won’t interfere with what the assert is complaining about. It may be a problem on your end.

@phynae
Copy link

phynae commented Mar 31, 2024

the draw function itself only uses ImGui::GetBackgroundDrawList and ImGui::GetTextSize functions to draw "certain stuff" over video games. until that error occured i have never heard of KeyMods in any situation. Therefore I certainly do not modify them activley and knowingly.

Fyi: I am internal, so maybe the game itself interfers with the keymods? idk i started that whole learning process a few weeks ago.

@ocornut
Copy link
Owner

ocornut commented Mar 31, 2024

Therefore I certainly do not modify them activley and knowingly.

You’ll need to investigate on your own. We can’t really help with that amount of information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants