Export project crashes only when exporting without "Export with debug" option

[ericdsw]

Godot version:

3.1.stable.official

OS/device including version:

macOS Mojave 10.14.4

Issue description:

I made this project to test out how spawning a lot of bullets would impact performance, so it is practically a port from the 2.1 version's "shower of bullets" demo to 3.1. The expected behavior is that after 1 second multiple bullet waves will spawn from the center of the screen, the node "DetectableArea" will increate it's counter when it detects a collision with any bullet, all bullets will be deleted when they reach the screen's boundaries, and after 10 seconds the bullets will stop spawning.

The project runs just fine in most cases:

• Running it through the editor with "play"
• Running it through the command line (navigating to the project's root and running the godot executable)
• Exporting it with the option "Export with debug" for both mac OSX and Windows Desktop.

However, when I try to export it without the "Export with debug" flag, it creates an executable that crashes as soon as it's opened without any warnings or clear signs., making this issue very difficult to pinpoint.

Since I don't have access to the development tools when it crashes, I'm not sure if the issue arises from something broken in the engine itself (maybe the Physics2DServer API?), or a bad implementation from my part (which I still think would be weird since both debug and release versions should produce the same errors).

Steps to reproduce:

1. Open the project
2. Select "Project" -> "Export"
3. Select any export target and click "Export project"
4. In the "Save a File" dialogue, disable the "Export with debug" option and click "save"
5. Try to open the game, it should crash.

Minimal reproduction project:

BulletSpawnerTest.zip

[qarmin]

This is an error from console

ERROR: _gdtype_from_datatype: Parser bug: converting unresolved type.
   At: modules/gdscript/gdscript_compiler.cpp:166.
SCRIPT ERROR: GDScript::load_byte_code: Compile Error: Parser bug: unresolved data type.
   At: res://AreaBulletSpawner.gdc:37.
ERROR: load_byte_code: Method/Function Failed, returning: ERR_COMPILATION_FAILED
   At: modules/gdscript/gdscript.cpp:793.
ERROR: load: Condition ' err != OK ' is true. returned: RES()
   At: modules/gdscript/gdscript.cpp:2173.
ERROR: _load: Method/Function Failed, returning: RES()
   At: core/io/resource_loader.cpp:285.
ERROR: poll: res://AreaBulletSpawner.tscn:3 - Parse Error: [ext_resource] referenced nonexistent resource at: res://AreaBulletSpawner.gd
   At: scene/resources/resource_format_text.cpp:440.
ERROR: load: Condition ' err != OK ' is true. returned: RES()
   At: core/io/resource_loader.cpp:208.
ERROR: _load: Method/Function Failed, returning: RES()
   At: core/io/resource_loader.cpp:285.
ERROR: poll: res://Game.tscn:4 - Parse Error: [ext_resource] referenced nonexistent resource at: res://AreaBulletSpawner.tscn
   At: scene/resources/resource_format_text.cpp:440.
ERROR: load: Condition ' err != OK ' is true. returned: RES()
   At: core/io/resource_loader.cpp:208.
ERROR: _load: Method/Function Failed, returning: RES()
   At: core/io/resource_loader.cpp:285.
ERROR: start: Condition ' !scene ' is true. returned: false
   At: main/main.cpp:1739.

This is line AreaBulletSpawner.gd:37

	Physics2DServer.free_rid(circle_shape)

[ericdsw]

Oh I see, just one question @qarmin
How did you get the console to report that error? The debug version never displayed that error and the release one would just not show any logs.

[qarmin]

I used Ubuntu and compiled by mine debug and release templates so probably one of this is an answer.
I think that is more likely, that mine exported templates has more debug symbols than in striped official templates(they have smaller size).

[akien-mga]

The macOS editor binary should have reported this issue in the editor debugger (and in the console if you run the editor from a console). If it's not in the editor debugger, that's a bug.

[ericdsw]

yeah, no errors were shown when running through the editor, nor when running the game through the terminal. Deleting that line prevented the game from crashing on an export without "Export with debug", but another issue arises: the areas are not being freed when the spawner exits the tree so collisions are being detected with invisible objects (happens on both debug and release exports).

The calls to free_rid prevent this by erasing the areas when no longer in use, but for some reason is crashing in release with the "referenced nonexistent resource" message, so I'm really not sure what to do here.

[ericdsw]

I have updated the project to better reflect what's happening:
BulletSpawnerTest.zip

The main changes are that there is now an icon representing DetectableArea and it can be moved with WASD; and that pressing 0 will free the AreaBulletSpawner node.

Exporting it without changing anything will yield a working executable, but pressing 0 while there are bullets on the screen will cause the following behavior:

• The AreaBulletSpawner node will be destroyed and, since bullets are drawn using the _draw method inside this node, they will disappear as well.
• Since there is no call to free_rid on the Physics2DServer, none of the created areas are destroyed.
• Moving the DetectableArea to where a bullet used to be just before pressing "0" will cause it to detect a collision, meaning the area is still there.
• This will also happen if the project is run through the editor.

If, however, lines 33, 34 and 35 are uncommented from AreaBulletSpawner, the game will produce an executable that crashes as soon as it's opened but running it through the editor will work as expected:

• All existing areas created with the Physics2DServer are freed.
• Moving the DetectableArea will not detect any collision whatsoever.

So it seems like communication with Physics2DServer works differently between debug and release builds.

Note: another issue that I saw here related to Physics2DServer is that the callback "area_shape_entered" inside DetectableArea (script is RandomArea2.gd) is correctly called when
the overlap is detected, but all parameters are either 0 or null.

[akien-mga]

@ericdsw Can you still reproduce it with a recent version of Godot?

I've tried with a release export on Linux using 3.2.2 RC 1, and it's not crashing for me.

[ericdsw]

@akien-mga hi akien, gonna try it as soon as I can. Will keep you updated.

[barrwani]

I've experienced a similar issue, however in my case there is a scene which I am autoloading which doesn't function properly when exported without debugging enabled. It's making it impossible to release games commercially

[KaDokta]

Can also confirm that the issue still exists. Pretty major problem, especially since it just killed my game jam submission.

This happens to me with HTML and with the Windows version. Not sure if autoloading is the problem, since it's during the game long after the autoloading relevant code has been executed. Works fine in the editor of course, and when exporting with debug on.

[Calinou]

In 3.2.2, this could be due to the dangling variant fix being only applied in debug mode (for performance reasons). In 4.0, a different way is used to solve this problem and will also work in release mode.

It's making it impossible to release games commercially

While not ideal, it's still possible to release a debug build of your game. Many commercial games have actually done this, either by mistake or intentionally 🙂

[kolecreates]

In case this helps anyone:

Exporting for MacOS crashes on 3.2.2.stable.official unless I use "Export all resources in the project" mode under the resources tab.

[KaDokta]

In 3.2.2, this could be due to the dangling variant fix being only applied in debug mode (for performance reasons). In 4.0, a different way is used to solve this problem and will also work in release mode.

It's making it impossible to release games commercially

While not ideal, it's still possible to release a debug build of your game. Many commercial games have actually done this, either by mistake or intentionally 🙂

That's good to know! However, I am wondering how I could debug this in the current 3.2.2. version, since I don't have a console in the release build? There's also no relevant warning when I execute the game in debug mode, which is even more confusing to me.

[Calinou]

However, I am wondering how I could debug this in the current 3.2.2. version, since I don't have a console in the release build?

You can enable file logging in the Project Settings then export the project again. Run the game, wait for it to crash then open the project's user data folder (%APPDATA%\Godot\app_userdata\<Project Name> by default on Windows). Open the logs folder inside then you should see some log files.

[KaDokta]

Sadly even with file logging enabled, I don't get any warnings or error messages in the log file whatsoever. Seems to be a deeper problem. Thank you for your help, though!

[Calinou]

@akien-mga Is it possible to compile release export templates that include debugging symbols with MSVC? This would be helpful to diagnose crashes like this one.

[akien-mga]

@Calinou Yes, with scons p=windows tools=no target=release debug_symbols=yes (or debug_symbols=full).

[sjharb]

I am having the same issue with the android build. Works great to deploy via usb with remote/debug enabled, but when that one singular option is disabled the application has problems. The application loads the main page but errors on the second. I'm new to android so i'm not sure how to view the logs without having remote/debug enabled. The APK exported also seem to hang in the same spot when manually deploying/installing on the phone. Im using GLES2 (migrated from GLES3 to work for android and changed ETC settings etc.). I have tried all sorts of export options. Thanks!

[asg08dev]

I am also having this issue on windows. My game crashes when spamming inputs for the most part however in debug mode this is not an issue.

[sjharb]

I am having the same issue with the android build. Works great to deploy via usb with remote/debug enabled, but when that one singular option is disabled the application has problems. The application loads the main page but errors on the second. I'm new to android so i'm not sure how to view the logs without having remote/debug enabled. The APK exported also seem to hang in the same spot when manually deploying/installing on the phone. Im using GLES2 (migrated from GLES3 to work for android and changed ETC settings etc.). I have tried all sorts of export options. Thanks!

Solved:

Short answer: I enabled Android Network Permissions ('Access Network State' and 'Internet') in the Project -> Export settings. Since my application uses networking this is obvious in hindsight. But it does not explain the fact that when using Remote/Debug enabled Android Export permissions are modified (possibly just networking 'Access Network State') by Godot, otherwise it would crash on my networking call as it does when not using remote/debug. This specific issue is probably not a Godot bug because it is most likely mandatory to have networking ('Access Network State') enabled to Remote/Debug on Android, the confusion is that the user does not realize Godot is changing their Export setting when using Remote/Debug.

Long Answer: Deploying manually and without the remote/debug enabled, had been a problem since I started using Android a month ago, I brushed it off at first because I knew there was a solution. My application started in all cases since my code that uses networking is invoked after the main page. The root of my ignorance was not knowing how to see my log files from the errors being thrown without debug on the android phone (FYI: find your adb.exe and use the command 'adb logcat' to see your android logs). Once I had my logs this is the error I was getting:

E/godot   ( 2168): **ERROR**: Condition "_sock == -1" is true. Returned: FAILED
E/godot   ( 2168):    At: drivers/unix/net_socket_posix.cpp:328:open() - Condition "_sock == -1" is true. Returned: FAILED
E/godot   ( 2168): **ERROR**: Condition "!is_open()" is true. Returned: ERR_UNCONFIGURED
E/godot   ( 2168):    At: drivers/unix/net_socket_posix.cpp:383:bind() - Condition "!is_open()" is true. Returned: ERR_UNCONFIGURED
E/godot   ( 2168): **ERROR**: Couldn't create an ENet multiplayer server.
E/godot   ( 2168):    At: modules/enet/networked_multiplayer_enet.cpp:108:create_server() - Condition "!host" is true. Returned: ERR_CANT_CREATE
E/godot   ( 2168): **ERROR**: Supplied NetworkedMultiplayerPeer must be connecting or connected.
E/godot   ( 2168):    At: core/io/multiplayer_api.cpp:144:set_network_peer() - Condition "p_peer.is_valid() && p_peer->get_connection_status() == NetworkedMultiplayerPeer::CONNECTION_DISCONNECTED" is true.
E/godot   ( 2168): **ERROR**: No network peer is assigned. Unable to get unique network ID.
E/godot   ( 2168):    At: core/io/multiplayer_api.cpp:814:get_network_unique_id() - Condition "!network_peer.is_valid()" is true. Returned: 0

With this error message I was able to solve my issue (see Short Answer above). But for others it might be helpful to know that Godot probably changes the Android Export setting without you knowing when using Remote/Debug.

Hope this helps someone!

Thanks,
-Shaun

[sjharb]

I am also having this issue on windows. My game crashes when spamming inputs for the most part however in debug mode this is not an issue.

Can someone correct me if I am wrong. Doesn't the 'Deploy with Remote/Debug' only get used when you are deploying remotely? By Remotely I mean remotely to your Phone or remotely to a Server? As far as I know this option does not need to be used when debugging local windows applications. Your debug breakpoints should trigger without using Remote/Debug enabled.

-Shaun

[asg08dev]

My Issue happens on windows when I do not check that box in the screen shot.

[sjharb]

My Issue happens on windows when I do not check that box in the screen shot.

It sounds like you need your error message from your crashing release build exported exe (stderr).

Godot allows you to enable logging and stdout and stderr in the project settings.

Run the 'start' command below in the command prompt to start your game and redirect the log output (stdout and stderr if enabled in your project settings when you built your exported exe). There might be easier ways to view stdout/stderr, but this worked for me in Windows:

start cmd.exe /c "MyGame --verbose > my_game_out.txt 2>&1"

And open your txt file to view the stdout/stderr.

I hope this helps,
-Shaun

[KaDokta]

It sounds like you need your error message from your crashing release build exported exe (stderr).

Godot allows you to enable logging and stdout and stderr in the project settings.

Run the 'start' command below in the command prompt to start your game and redirect the log output (stdout and stderr if enabled in your project settings when you built your exported exe). There might be easier ways to view stdout/stderr, but this worked for me in Windows:

start cmd.exe /c "MyGame --verbose >my_game_out.txt 2>&1

This indeed shows dome debug information! Sadly, still no error message whatsoever, at least for me.

[asg08dev]

Same for me KaDokta no crash information. The interesting thing is if I export it with the debug option selected nothing crashes I dont understand why debug vs no debug would make a difference in this instance unless the godot engine that the standalone exe is wrapped with is different than the one used to build the game.

[Calinou]

I dont understand why debug vs no debug would make a difference in this instance unless the godot engine that the standalone exe is wrapped with is different than the one used to build the game.

Debug export templates are compiled as console applications, whereas release export templates are compiled as GUI applications (which don't spawn a command window).

[Sajyd]

I'm having the same annoying issue with godot 3.2.1 : App crashes when i uncheck "Export with debug" to have my app as a released version. It happens on both iOS, Android and even PC. When testing on iOS, the crash on Xcode reported something about a variant or/and an object assignement, might be an internal bug. But really frustrating now that I finished my game, I can't even publish it after this long work. Would be a pain in the ass to refactor all to newer godot version

In 3.2.2, this could be due to the dangling variant fix being only applied in debug mode (for performance reasons). In 4.0, a different way is used to solve this problem and will also work in release mode.

It's making it impossible to release games commercially

While not ideal, it's still possible to release a debug build of your game. Many commercial games have actually done this, either by mistake or intentionally 🙂

Well it may be possible on Windows but on Android/iOS stores won't approuve a debug app. Please make this fix for release :)

[Calinou]

Please make this fix for release :)

Unfortunately, this bug can't be fixed in release builds until 4.0 at least. You need to decompose your project into smaller parts, isolate the source of the dangling Variant, and fix it.

[Sajyd]

Ok i found the problem quiet easily by decomposing the project. It was coming from a child node of an instanciated scene
get_node("AnimationPlayer").play("Idle")
Working fine on release now! Thanks for help.

[qarmin]

With 3.4.beta.custom_build. 40b5731
I have this crash when running project with sanitzers

core/math/bvh_split.inc:74:35: runtime error: index 2 out of bounds for type 'float [2]'
core/math/bvh_split.inc:74:62: runtime error: index 2 out of bounds for type 'float [2]'
core/math/bvh_split.inc:74:62: runtime error: load of address 0x7fff11ff0468 with insufficient space for an object of type 'real_t'
0x7fff11ff0468: note: pointer points here
 34 4a 19 42  2a 00 00 00 00 00 00 00  90 eb d1 86 3f 7f 00 00  03 bf af 88 3f 7f 00 00  f2 e6 1d 44
              ^ 
=================================================================
==8412==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fff11ff0468 at pc 0x0000111ecffd bp 0x7fff11ff03a0 sp 0x7fff11ff0390
READ of size 4 at 0x7fff11ff0468 thread T0
    #0 0x111ecffc in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::_split_leaf_sort_groups_simple(int&, int&, unsigned short*, unsigned short*, BVH_ABB<Rect2, Vector2> const*, BVH_ABB<Rect2, Vector2>) core/math/bvh_split.inc:74
    #1 0x111e4928 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::split_leaf_complex(unsigned int, BVH_ABB<Rect2, Vector2> const&) core/math/bvh_split.inc:252
    #2 0x111d3074 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::split_leaf(unsigned int, BVH_ABB<Rect2, Vector2> const&) core/math/bvh_split.inc:186
    #3 0x111c6994 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::_logic_choose_item_add_node(unsigned int, BVH_ABB<Rect2, Vector2> const&) core/math/bvh_logic.inc:212
    #4 0x111cafb8 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::item_set_pairable(BVHHandle const&, bool, unsigned int, unsigned int) core/math/bvh_public.inc:323
    #5 0x111c1c09 in BVH_Manager<CollisionObject2DSW, true, 128, Rect2, Vector2>::set_pairable(BVHHandle const&, bool, unsigned int, unsigned int, bool) core/math/bvh.h:282
    #6 0x111bd7ea in BVH_Manager<CollisionObject2DSW, true, 128, Rect2, Vector2>::set_pairable(unsigned int, bool, unsigned int, unsigned int, bool) core/math/bvh.h:159
    #7 0x111ba436 in BroadPhase2DBVH::set_static(unsigned int, bool) servers/physics_2d/broad_phase_2d_bvh.cpp:46
    #8 0x1122be8f in CollisionObject2DSW::_set_static(bool) servers/physics_2d/collision_object_2d_sw.cpp:148
    #9 0x1112ad98 in Area2DSW::set_monitorable(bool) servers/physics_2d/area_2d_sw.cpp:197
    #10 0x105c2ff1 in Physics2DServerSW::area_set_monitorable(RID, bool) servers/physics_2d/physics_2d_server_sw.cpp:488
    #11 0x1063f38a in Physics2DServerWrapMT::area_set_monitorable(RID, bool) servers/physics_2d/physics_2d_server_wrap_mt.h:165
    #12 0xff4ed18 in MethodBind2<RID, bool>::call(Object*, Variant const**, int, Variant::CallError&) core/method_bind.gen.inc:1523
    #13 0x11b6557b in Object::call(StringName const&, Variant const**, int, Variant::CallError&) core/object.cpp:918
    #14 0x11ded653 in Variant::call_ptr(StringName const&, Variant const**, int, Variant*, Variant::CallError&) core/variant_call.cpp:1173
    #15 0x1e2bfb1 in GDScriptFunction::call(GDScriptInstance*, Variant const**, int, Variant::CallError&, GDScriptFunction::CallState*) modules/gdscript/gdscript_function.cpp:1034
    #16 0x1c5c599 in GDScriptInstance::call(StringName const&, Variant const**, int, Variant::CallError&) modules/gdscript/gdscript.cpp:1151
    #17 0x11b650eb in Object::call(StringName const&, Variant const**, int, Variant::CallError&) core/object.cpp:899
    #18 0x11ded653 in Variant::call_ptr(StringName const&, Variant const**, int, Variant*, Variant::CallError&) core/variant_call.cpp:1173
    #19 0x1e2bfb1 in GDScriptFunction::call(GDScriptInstance*, Variant const**, int, Variant::CallError&, GDScriptFunction::CallState*) modules/gdscript/gdscript_function.cpp:1034
    #20 0x1c5c599 in GDScriptInstance::call(StringName const&, Variant const**, int, Variant::CallError&) modules/gdscript/gdscript.cpp:1151
    #21 0x11b650eb in Object::call(StringName const&, Variant const**, int, Variant::CallError&) core/object.cpp:899
    #22 0x11b6f3dc in Object::emit_signal(StringName const&, Variant const**, int) core/object.cpp:1226
    #23 0x11b71412 in Object::emit_signal(StringName const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&) core/object.cpp:1281
    #24 0xc58442f in Timer::_notification(int) scene/main/timer.cpp:77
    #25 0xc58f08b in Timer::_notificationv(int, bool) scene/main/timer.h:37
    #26 0x11b65a15 in Object::notification(int, bool) core/object.cpp:927
    #27 0xc4dfd0d in SceneTree::_notify_group_pause(StringName const&, int) scene/main/scene_tree.cpp:969
    #28 0xc4cd369 in SceneTree::iteration(float) scene/main/scene_tree.cpp:478
    #29 0x195e28a in Main::iteration() main/main.cpp:2082
    #30 0x183ca64 in OS_X11::run() platform/x11/os_x11.cpp:3638
    #31 0x17a7d8b in main platform/x11/godot_x11.cpp:55
    #32 0x7f3f87c1c564 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x28564)
    #33 0x17a79ad in _start (/usr/bin/godots+0x17a79ad)

Address 0x7fff11ff0468 is located in stack of thread T0 at offset 40 in frame
    #0 0x111ebc0b in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::_split_leaf_sort_groups_simple(int&, int&, unsigned short*, unsigned short*, BVH_ABB<Rect2, Vector2> const*, BVH_ABB<Rect2, Vector2>) core/math/bvh_split.inc:14

  This frame has 5 object(s):
    [32, 40) 'centre' (line 28) <== Memory access at offset 40 overflows this variable
    [64, 72) 'size' (line 29)
    [96, 108) 'order' (line 31)
    [128, 140) 'min_group_size' (line 57)
    [160, 176) 'full_bound' (line 14)
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow core/math/bvh_split.inc:74 in BVH_Tree<CollisionObject2DSW, 2, 128, true, Rect2, Vector2>::_split_leaf_sort_groups_simple(int&, int&, unsigned short*, unsigned short*, BVH_ABB<Rect2, Vector2> const*, BVH_ABB<Rect2, Vector2>)

[akien-mga]

I'm going to close this issue as I can't reproduce the original problem reported, which was specific to physics.

Since the issue title was mistakenly kept too generic, lots of reports here are not related to the original bug and related instead to various other issues which can happen only in release builds if errors haven't been found or fixed when using debug builds. Several of these issues have been fixed over subsequent releases, including a major fix coming in 3.4 (available in 3.4 beta 5).

So to anyone with an issue here, I would suggest to test the latest 3.4 build to confirm if their issue is fixed, and if not, open a new bug report with a minimal reproduction project so that we can assess what the current state is for each specific issue.

[unforgiven1990]

I literally searched weeks for the bug and I FOUND it!

create_tween().set_loops()

I had tween on items for idle animation. When the item is queued free, then the tween will infinitely run and cause the game to free ON exported version. In the editor for some reason does not cause infinite loop.

This was the only reason that caused the game to crash after export, hopeful this is also your root cause!