Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AssertionError: Crash related to ShaderTerrainMesh #499

Closed
thetestgame opened this issue Jan 1, 2019 · 9 comments
Closed

AssertionError: Crash related to ShaderTerrainMesh #499

thetestgame opened this issue Jan 1, 2019 · 9 comments
Labels
bug
Milestone

Comments

@thetestgame
Copy link
Contributor

@thetestgame thetestgame commented Jan 1, 2019

While running tests on a ShaderTerrainMesh we encountered a rather nasty and instant assertion error. I know ahead of time that the heightfield is not a 16-bit image however I suspect this should be failing more gracefully then it currently is. This also may be related to #489 and its not directly related to multithreading as I originally suspected.

Assertion failed: get_ref_count() == 0 at line 128 of c:\buildslave\sdk-windows-amd64\build\panda\src\pgraph\renderState.cxx
Assertion failed: Traceback (most recent call last):
!is_destructing() at line 117 of c:\buildslave\sdk-windows-amd64\build\panda\src\pgraph\renderState.cxx  File "D:\Panda3D-1.10.0-x64\direct\showbase\ShowBase.py", line 1974, in __garbageCollectStates

Assertion failed:     RenderState.garbageCollect()
_node_ref_count != -100 at line 78 of c:\buildslave\sdk-windows-amd64\build\built\include\nodeCachedReferenceCount.IAssertionError: !is_destructing() at line 117 of c:\buildslave\sdk-windows-amd64\build\panda\src\pgraph\renderState.cxx

:task(error): Assertion failed: _cache_ref_count != -100 at line 78 of c:\buildslave\sdk-windows-amd64\build\built\include\cachedTypedWritableReferenceCount.I
Exception occurred in PythonTask garbageCollectStates
Assertion failed: Traceback (most recent call last):
_ref_count != deleted_ref_count at line 87 of c:\buildslave\sdk-windows-amd64\build\built\include\referenceCount.I  File "main.py", line 92, in <module>

    ShaderTerrainDemo().run()
Assertion failed: orig_flag != (AtomicAdjust::Integer)DCF_deleted, file c:\buildslave\sdk-windows-amd64\build\dtool\src\dtoolbase\deletedBufferChain.cxx, line 122
  File "D:\Panda3D-1.10.0-x64\direct\showbase\ShowBase.py", line 3109, in run

shader-terrain-issue.zip

@rdb

This comment has been minimized.

Copy link
Member

@rdb rdb commented Jan 1, 2019

I just tried running it and nothing bad happened so far. Is the issue supposed to happen right away upon launching the sample, or only after some time or after specific actions?

Are there any specific configuration settings active compared to the default Panda config?

@thetestgame

This comment has been minimized.

Copy link
Contributor Author

@thetestgame thetestgame commented Jan 1, 2019

It is the default panda config and it seems like it renders one frame then dies. All I'm doing to trigger it is start the program.

https://i.gyazo.com/07d6a696becdf8380728074fcc41ac55.gif

Example

@thetestgame

This comment has been minimized.

Copy link
Contributor Author

@thetestgame thetestgame commented Jan 1, 2019

While playing with it more it would appear it only does this when its not maximized. If I maximize the window immediately it runs fine. However there are some interesting issues with how the terrain is behaving. Again most likely due to the heightmap I'm using

https://gyazo.com/822c127532409c3aefbafbd3f8d22599

Screenshot from Gyazo

@tcdude

This comment has been minimized.

Copy link
Contributor

@tcdude tcdude commented Feb 8, 2019

I couldn't reproduce a crash on Ubuntu 18.10, @rdb could this be specific to Windows?

The observed terrain behavior is due to the heightmap file you are using. When changing it to the one from the samples folder it behaves almost correct, except for some flickering on the far end of the ShaderTerrainMesh (looks like a culling issue). The flickering only disappears when disabling multi threaded rendering.

I'll try to investigate this further

@JoelStienlet

This comment has been minimized.

Copy link

@JoelStienlet JoelStienlet commented Feb 8, 2019

I don't know if it is related, but when I run the demo with valgrind he complains about unitialized variables:
[stienlet@localhost shader-terrain]$ valgrind python3 main.py
==16526== Memcheck, a memory error detector
==16526== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==16526== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==16526== Command: python3 main.py
==16526==
==16526== Conditional jump or move depends on uninitialised value(s)
==16526== at 0x142343D6: DTOOL_Call_GetPointerThisClass(_object*, Dtool_PyTypedObject*, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool, bool) (in /usr/lib64/python3.7/site-packages/panda3d/core.cpython-37m-x86_64-linux-gnu.so)
==16526== by 0x13E8192F: Dtool_Init_NodePath(_object*, _object*, _object*) (in /usr/lib64/python3.7/site-packages/panda3d/core.cpython-37m-x86_64-linux-gnu.so)
==16526== by 0x4A00938: _PyObject_FastCallKeywords (in /usr/lib64/libpython3.7m.so.1.0)
==16526== by 0x4A4B6DE: _PyEval_EvalFrameDefault (in /usr/lib64/libpython3.7m.so.1.0)
==16526== by 0x498E6F7: _PyEval_EvalCodeWithName (in /usr/lib64/libpython3.7m.so.1.0)
==16526== by 0x49D55F0: _PyFunction_FastCallKeywords (in /usr/lib64/libpython3.7m.so.1.0)
==16526== by 0x4A4B33C: _PyEval_EvalFrameDefault (in /usr/lib64/libpython3.7m.so.1.0)
==16526== by 0x498F6E9: _PyFunction_FastCallDict (in /usr/lib64/libpython3.7m.so.1.0)
==16526== by 0x499F845: _PyObject_Call_Prepend (in /usr/lib64/libpython3.7m.so.1.0)
==16526== by 0x49ED520: ??? (in /usr/lib64/libpython3.7m.so.1.0)
==16526== by 0x4A00938: _PyObject_FastCallKeywords (in /usr/lib64/libpython3.7m.so.1.0)
==16526== by 0x4A4B6DE: _PyEval_EvalFrameDefault (in /usr/lib64/libpython3.7m.so.1.0)

Then again, multiple times (I won't copy the full output here, it's tool long).

@Moguri Moguri added the bug label Mar 3, 2019
@Moguri

This comment has been minimized.

Copy link
Collaborator

@Moguri Moguri commented Mar 3, 2019

I am unable to reproduce the crash on Arch Linux.

@thetestgame

This comment has been minimized.

Copy link
Contributor Author

@thetestgame thetestgame commented Mar 10, 2020

I'm currently working on a new application inside Panda3D and am hitting the same Assertion failed: orig_flag != (AtomicAdjust::Integer)DCF_deleted, file c:\buildslave\sdk-windows-amd64\build\dtool\src\dtoolbase\deletedBufferChain.cxx, line 122 assertion error when using multithreading and a ShaderTerrainMesh instance. Still the same hardware and still Windows 10 same as last year.

@rdb

This comment has been minimized.

Copy link
Member

@rdb rdb commented Mar 10, 2020

I tried again and managed to reproduce it. It seems to be a crash in the state system not directly related to ShaderTerrainMesh. I'll investigate further.

@rdb

This comment has been minimized.

Copy link
Member

@rdb rdb commented Mar 10, 2020

The problem is that the state cache relies on being the only one that can raise a state reference count from 1 to 2, and therefore assumes that if the reference count is 1, it owns it and can safely remove it from the cache and delete it. However, WeakPointerTo.lock() can also make the refcount go from 1 to 2 (which happens in the GL shader implementation), so this assumption is not correct. This doesn't directly have anything to do with ShaderTerrainMesh, it's just that it exposes this bug because it manipulates states in the cull thread.

I think I have an idea how to fix this; the state cache needs to atomically set the state reference count to 0 before attempting to remove the state from the cache. This will work because weak pointers can't be locked when their reference count is 0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.