[AI] BARb: fix out-of-bounds access #32
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gajop
added a commit
to gajop/bar-spring
that referenced
this pull request
Nov 25, 2025
Root cause: Context::Render was exposed to Lua without proper frame
management (BeginFrame/PresentFrame). When called from Lua for
render-to-texture workflows, PushLayer/PopLayer would execute outside
the frame lifecycle, causing the layer stack to underflow and access
freed memory.
Solution: Wrap the Lua binding for Context::Render with a lambda that
calls BeginFrame before and PresentFrame after rendering, ensuring
proper layer stack management for all callers including RTT use cases.
ASAN error:
==37278==ERROR: AddressSanitizer: heap-use-after-free on address 0x503000460eec at pc 0x5c05ab21d77a bp 0x7ffcce4f73e0 sp 0x7ffcce4f73d0
READ of size 4 at 0x503000460eec thread T0 (recoil-main)
#0 0x5c05ab21d779 in RenderInterface_GL3_Recoil::PopLayer() /build/src/rts/Rml/Backends/RmlUi_Renderer_GL3_Recoil.cpp:1727
beyond-all-reason#1 0x5c05ad40d908 in Rml::RenderManager::PopLayer() /build/src/rts/lib/RmlUi/Source/Core/RenderManager.cpp:326
beyond-all-reason#2 0x5c05ad2fac86 in Rml::ElementEffects::RenderEffects(Rml::RenderStage) /build/src/rts/lib/RmlUi/Source/Core/ElementEffects.cpp:336
beyond-all-reason#3 0x5c05ad2a037a in Rml::Element::Render() /build/src/rts/lib/RmlUi/Source/Core/Element.cpp:255
beyond-all-reason#4 0x5c05ad2a02eb in Rml::Element::Render() /build/src/rts/lib/RmlUi/Source/Core/Element.cpp:253
beyond-all-reason#5 0x5c05ad2a02eb in Rml::Element::Render() /build/src/rts/lib/RmlUi/Source/Core/Element.cpp:253
beyond-all-reason#6 0x5c05ad1e000a in Rml::Context::Render() /build/src/rts/lib/RmlUi/Source/Core/Context.cpp:221
beyond-all-reason#7 0x5c05ab3a434b in bool sol::member_function_wrapper<bool (Rml::Context::*)(), bool, Rml::Context>::call<bool (Rml::Context::*&)()>(bool (Rml::Context::*&)(), Rml::Context&) /build/src/rts/lib/sol2/sol.hpp:17338
beyond-all-reason#8 0x5c05ab3a434b in decltype(auto) sol::member_function_wrapper<bool (Rml::Context::*)(), bool, Rml::Context>::caller::operator()<bool (Rml::Context::*&)()>(bool (Rml::Context::*&)(), Rml::Context&) const /build/src/rts/lib/sol2/sol.hpp:17344
beyond-all-reason#9 0x5c05ab3a434b in eval<true, sol::argument_handler<sol::types<bool> >&, sol::member_function_wrapper<bool (Rml::Context::*)(), bool, Rml::Context>::caller, bool (Rml::Context::*&)(), Rml::Context&> /build/src/rts/lib/sol2/sol.hpp:16078
beyond-all-reason#10 0x5c05ab3a434b in decltype(auto) sol::stack::stack_detail::call<true, , bool, , sol::member_function_wrapper<bool (Rml::Context::*)(), bool, Rml::Context>::caller, bool (Rml::Context::*&)(), Rml::Context&>(sol::types<bool>, sol::types<>, std::integer_sequence<unsigned long>, lua_State*, int, sol::member_function_wrapper<bool (Rml::Context::*)(), bool, Rml::Context>::caller&&, bool (Rml::Context::*&)(), Rml::Context&) /build/src/rts/lib/sol2/sol.hpp:16131
beyond-all-reason#11 0x5c05ab3a434b in decltype(auto) sol::stack::call<true, bool, , sol::member_function_wrapper<bool (Rml::Context::*)(), bool, Rml::Context>::caller, bool (Rml::Context::*&)(), Rml::Context&>(sol::types<bool>, sol::types<>, lua_State*, int, sol::member_function_wrapper<bool (Rml::Context::*)(), bool, Rml::Context>::caller&&, bool (Rml::Context::*&)(), Rml::Context&) /build/src/rts/lib/sol2/sol.hpp:16150
beyond-all-reason#12 0x5c05ab3a434b in int sol::stack::call_into_lua<true, true, bool, , , sol::member_function_wrapper<bool (Rml::Context::*)(), bool, Rml::Context>::caller, bool (Rml::Context::*&)(), Rml::Context&>(sol::types<bool>, sol::types<>, lua_State*, int, sol::member_function_wrapper<bool (Rml::Context::*)(), bool, Rml::Context>::caller&&, bool (Rml::Context::*&)(), Rml::Context&) /build/src/rts/lib/sol2/sol.hpp:16198
beyond-all-reason#13 0x5c05ab3a434b in int sol::call_detail::lua_call_wrapper<Rml::Context, bool (Rml::Context::*)(), true, false, true, 0, true, void>::call<bool (Rml::Context::*&)(), Rml::Context&>(lua_State*, bool (Rml::Context::*&)(), Rml::Context&) /build/src/rts/lib/sol2/sol.hpp:18103
beyond-all-reason#14 0x5c05ab3a434b in int sol::call_detail::lua_call_wrapper<Rml::Context, bool (Rml::Context::*)(), true, false, true, 0, true, void>::call<bool (Rml::Context::*&)()>(lua_State*, bool (Rml::Context::*&)()) /build/src/rts/lib/sol2/sol.hpp:18093
beyond-all-reason#15 0x5c05ab3a434b in int sol::call_detail::call_wrapped<Rml::Context, true, false, 0, true, true, bool (Rml::Context::*&)()>(lua_State*, bool (Rml::Context::*&)()) /build/src/rts/lib/sol2/sol.hpp:18506
beyond-all-reason#16 0x5c05ab3a434b in int sol::u_detail::binding<char [7], bool (Rml::Context::*)(), Rml::Context>::call_with_<true, false>(lua_State*, void*) /build/src/rts/lib/sol2/sol.hpp:23023
beyond-all-reason#17 0x5c05ab3a434b in int sol::u_detail::binding<char [7], bool (Rml::Context::*)(), Rml::Context>::call_<true, false>(lua_State*) /build/src/rts/lib/sol2/sol.hpp:23029
beyond-all-reason#18 0x5c05aa5925ba in sol::detail::lua_cfunction_trampoline(lua_State*, int (*)(lua_State*)) /build/src/rts/lib/sol2/sol.hpp:8398
beyond-all-reason#19 0x5c05ab361ccf in int sol::detail::static_trampoline<&(int sol::u_detail::binding<char [7], bool (Rml::Context::*)(), Rml::Context>::call_<true, false>(lua_State*))>(lua_State*) /build/src/rts/lib/sol2/sol.hpp:8423
beyond-all-reason#20 0x5c05ab361ccf in int sol::detail::typed_static_trampoline<int (*)(lua_State*), &(int sol::u_detail::binding<char [7], bool (Rml::Context::*)(), Rml::Context>::call_<true, false>(lua_State*))>(lua_State*) /build/src/rts/lib/sol2/sol.hpp:8490
beyond-all-reason#21 0x5c05ab361ccf in int sol::u_detail::binding<char [7], bool (Rml::Context::*)(), Rml::Context>::call<true, false>(lua_State*) /build/src/rts/lib/sol2/sol.hpp:23034
beyond-all-reason#22 0x5c05ac11b57f in luaD_precall(lua_State*, lua_TValue*, int) /build/src/rts/lib/lua/src/ldo.cpp:320
beyond-all-reason#23 0x5c05ac1499a2 in luaV_execute(lua_State*, int) /build/src/rts/lib/lua/src/lvm.cpp:620
beyond-all-reason#24 0x5c05ac11c484 in luaD_call(lua_State*, lua_TValue*, int) /build/src/rts/lib/lua/src/ldo.cpp:378
beyond-all-reason#25 0x5c05ac1056b8 in f_call /build/src/rts/lib/lua/src/lapi.cpp:812
beyond-all-reason#26 0x5c05ac119bb8 in luaD_rawrunprotected(lua_State*, void (*)(lua_State*, void*), void*) /build/src/rts/lib/lua/src/ldo.cpp:116
beyond-all-reason#27 0x5c05ac11cdc2 in luaD_pcall(lua_State*, void (*)(lua_State*, void*), void*, long, long) /build/src/rts/lib/lua/src/ldo.cpp:464
beyond-all-reason#28 0x5c05ac10d695 in lua_pcall(lua_State*, int, int, int) /build/src/rts/lib/lua/src/lapi.cpp:833
beyond-all-reason#29 0x5c05ac11272f in luaB_pcall /build/src/rts/lib/lua/src/lbaselib.cpp:389
beyond-all-reason#30 0x5c05ac11b57f in luaD_precall(lua_State*, lua_TValue*, int) /build/src/rts/lib/lua/src/ldo.cpp:320
beyond-all-reason#31 0x5c05ac1499a2 in luaV_execute(lua_State*, int) /build/src/rts/lib/lua/src/lvm.cpp:620
beyond-all-reason#32 0x5c05ac11c484 in luaD_call(lua_State*, lua_TValue*, int) /build/src/rts/lib/lua/src/ldo.cpp:378
beyond-all-reason#33 0x5c05ac1056b8 in f_call /build/src/rts/lib/lua/src/lapi.cpp:812
beyond-all-reason#34 0x5c05ac119bb8 in luaD_rawrunprotected(lua_State*, void (*)(lua_State*, void*), void*) /build/src/rts/lib/lua/src/ldo.cpp:116
beyond-all-reason#35 0x5c05ac11cdc2 in luaD_pcall(lua_State*, void (*)(lua_State*, void*), void*, long, long) /build/src/rts/lib/lua/src/ldo.cpp:464
beyond-all-reason#36 0x5c05ac10d695 in lua_pcall(lua_State*, int, int, int) /build/src/rts/lib/lua/src/lapi.cpp:833
beyond-all-reason#37 0x5c05aa1e5175 in ScopedLuaCall /build/src/rts/Lua/LuaHandle.cpp:397
beyond-all-reason#38 0x5c05aa1e5175 in CLuaHandle::RunCallInTraceback(lua_State*, LuaHashString const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, int, int, int, bool) /build/src/rts/Lua/LuaHandle.cpp:483
beyond-all-reason#39 0x5c05aa1e6908 in CLuaHandle::RunCallInTraceback(lua_State*, LuaHashString const&, int, int, int, bool) /build/src/rts/Lua/LuaHandle.cpp:494
beyond-all-reason#40 0x5c05aa2102b2 in CLuaHandle::RunCallIn(lua_State*, LuaHashString const&, int, int) /build/src/rts/Lua/LuaHandle.h:425
beyond-all-reason#41 0x5c05aa2102b2 in CLuaHandle::DrawScreenCommon(LuaHashString const&) /build/src/rts/Lua/LuaHandle.cpp:2863
beyond-all-reason#42 0x5c05aa2102b2 in CLuaHandle::DrawScreen() /build/src/rts/Lua/LuaHandle.cpp:2881
beyond-all-reason#43 0x5c05ab9492b0 in CEventHandler::DrawScreen() /build/src/rts/System/EventHandler.cpp:708
beyond-all-reason#44 0x5c05aa24cdef in CLuaInputReceiver::Draw() /build/src/rts/Lua/LuaInputReceiver.cpp:71
beyond-all-reason#45 0x5c05ad971694 in CGame::DrawInputReceivers() /build/src/rts/Game/Game.cpp:1574
beyond-all-reason#46 0x5c05ad98b020 in CGame::Draw() /build/src/rts/Game/Game.cpp:1526
beyond-all-reason#47 0x5c05aba75cce in SpringApp::Update() /build/src/rts/System/SpringApp.cpp:889
beyond-all-reason#48 0x5c05aba8323b in SpringApp::Run() /build/src/rts/System/SpringApp.cpp:927
beyond-all-reason#49 0x5c05ab9edb79 in Run(int, char**) /build/src/rts/System/Main.cpp:51
beyond-all-reason#50 0x5c05a9f546d3 in main /build/src/rts/System/Main.cpp:104
beyond-all-reason#51 0x782b5082a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
beyond-all-reason#52 0x782b5082a28a in __libc_start_main_impl ../csu/libc-start.c:360
beyond-all-reason#53 0x5c05aa01a049 in _start (/home/gajop/projects/spring-projects/spring-bar/build-linux/install/spring+0x931049) (BuildId: 0100fe9eb07611ab4faec887c8a97ffbdbc3df06)
0x503000460eec is located 28 bytes inside of 32-byte region [0x503000460ed0,0x503000460ef0)
freed by thread T0 (recoil-main) here:
#0 0x782b512fc4d8 in free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52
beyond-all-reason#1 0x782b40ef9d00 (/lib/x86_64-linux-gnu/libnvidia-glcore.so.535.274.02+0x14f9d00) (BuildId: 513f593c743a6a4f8ccb0183cb093aa171cef856)
previously allocated by thread T0 (recoil-main) here:
#0 0x782b512fd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
beyond-all-reason#1 0x782b40ef9851 (/lib/x86_64-linux-gnu/libnvidia-glcore.so.535.274.02+0x14f9851) (BuildId: 513f593c743a6a4f8ccb0183cb093aa171cef856)
SUMMARY: AddressSanitizer: heap-use-after-free /build/src/rts/Rml/Backends/RmlUi_Renderer_GL3_Recoil.cpp:1727 in RenderInterface_GL3_Recoil::PopLayer()
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.