Elusive and peculiar issue with TreeStore and EntryCompletion

[ghost]

Hello

I'm experiencing a very strange reproducible crash, which seems to be related to TreeStore. Let me sketch my situation: I have a sort of 'file browser' that contains a simple TreeView with TreeStore. I also have a ListStore, that keeps track of the files inside the TreeStore and updates itself (or rather: will update, as this is not implemented yet) whenever the tree is updated. This ListStore is also used by an EntryCompletion as model so you can search through the files of the tree.

At startup, I load in a bunch of folders in my file browser asynchronously (in a separate thread created by D's taskPool, using threadsAddIdle to do the actual GTK manipulations in the main thread). The ListStore is simply built by hooking into the method that adds a node to the tree. This works fine, I can use the EntryCompletion to complete my text successfully. What is very strange is that any modification I do to the original TreeStore (thus, not the model used by the EntryCompletion) causes any completion in the EntryCompletion to crash the entire application. My tree also listens to editing the text in cells and update files on disk accordingly, but even though the ListStore is not modified in any way when this happens, the application still crashes when searching it through the EntryCompletion. In short:

1. Start the application, the ListStore is built at the same time as the TreeStore.
2. Searching the ListStore works.
3. Rename a node in the TreeStore, which does nothing in the ListStore (which is completely independant).
4. Search again, a crash occurs.

I have been searching for a while and tried a dozen different solutions, but nothing seems to work, also using the TreeStore directly in the EntryCompletion (thus bypassing the ListStore) has the same effect: a crash whenever the tree is first modified. Every modification after startup runs through the exact same code as on startup. This crash both happens with DMD 2.066.1 as well as GDC 4.9.2, attaching GDB to the GDC version yields a huge backtrace:

#0  0x000000000083e868 in core.sync.mutex.Mutex.lock() ()
#1  0x00000000008c9dd7 in gc.gc.GC.malloc(ulong, uint, ulong*) ()
#2  0x000000000083230f in _d_newitemT ()
#3  0x000000000083fdf3 in _d_throw ()
#4  0x00000000008c875a in gc.gc.GC.mallocNoSync(ulong, uint, ulong*) ()
#5  0x00000000008c9de8 in gc.gc.GC.malloc(ulong, uint, ulong*) ()
#6  0x000000000083230f in _d_newitemT ()
#7  0x000000000083fdf3 in _d_throw ()
#8  0x00000000008c875a in gc.gc.GC.mallocNoSync(ulong, uint, ulong*) ()
#9  0x00000000008c9de8 in gc.gc.GC.malloc(ulong, uint, ulong*) ()
...
About 120000 times the method calls #6 through #9.
...
#123266 0x000000000083f53f in gc_qalloc ()
#123267 0x0000000000832516 in _d_newarrayiT ()
#123268 0x0000000000549224 in glib.Str.Str.toStringz(immutable(char)[]) (s=...)
    at ../../.dub/packages/gtk-d-2.4.1/src/glib/Str.d:130
#123269 0x000000000055bb20 in gobject.ObjectG.ObjectG.stealData(immutable(char)[]) (this=..., 
    key=...) at ../../.dub/packages/gtk-d-2.4.1/src/gobject/ObjectG.d:927
#123270 0x000000000055a454 in gobject.ObjectG.ObjectG.~this() (this=...)
    at ../../.dub/packages/gtk-d-2.4.1/src/gobject/ObjectG.d:225
#123271 0x0000000000832a9e in rt_finalize2 ()
#123272 0x00000000008c604a in gc.gc.Gcx.fullcollect() ()
#123273 0x00000000008c8df6 in gc.gc.GC.mallocNoSync(ulong, uint, ulong*) ()
#123274 0x00000000008ca2d8 in gc.gc.GC.malloc(ulong, uint, ulong*) ()
#123275 0x000000000083f53f in gc_qalloc ()
#123276 0x0000000000833d79 in _d_arrayappendcTX ()
#123277 0x0000000000833f95 in _d_arrayappendT ()
#123278 0x0000000000834057 in _d_arrayappendcd ()
#123279 0x000000000088778b in _D3std3uni93__T6toCaseS34_D3std3uni12toLowerIndexFNaNbNewZtVi1043S32_D3std3uni10toLowerTabFNaNbNemZwTAyaZ6toCaseFNaNeAyaZAya14__foreachbody2MFKmKwZi14__foreachbody3MFNfKwZi ()
#123280 0x000000000082e624 in _aApplycd1 ()
#123281 0x00000000008876a0 in _D3std3uni93__T6toCaseS34_D3std3uni12toLowerIndexFNaNbNewZtVi1043S32_D3std3uni10toLowerTabFNaNbNemZwTAyaZ6toCaseFNaNeAyaZAya14__foreachbody2MFKmKwZi ()
#123282 0x000000000082eac1 in _aApplycd2 ()
#123283 0x0000000000885ed3 in std.uni.toLower(immutable(char)[]) ()

As you can see, the backtrace claims to crash in 'toLower', which is called during filtering in my EntryCompletion.setMatchFunc() method, but it runs just fine on startup. The end of the back trace is also not always constant: sometimes it claims to crash somewhere else. Could this possibly be a memory corruption of some kind? The other threads in my application are created using D's taskPool and at the time of the backtrace were all finished running.

I wish I could be more specific about what specifically is crashing, but this issue seems to be rather elusive. Any tips or suggestions are welcome. I was hoping that you might have some idea what could be causing this, I'm not sure if this is even a GtkD issue, it could very well also be a Gtk, a GDC/DMD, a Phobos, or even a garbage collector issue. In very rare cases, the crash also doesn't seem to occur at all.

Thanks in advance for any help you can offer

[MikeWey]

Do you have some code for me to look at ?

[ghost]

Thanks for your response. I've uploaded the code that GDB is claming crashes to Pastebin. I've added a comment to the location of the crash, and also noticed the following during yet another debug run: if I remove all the code inside the 'try' block, no crash occurs when searching after a tree modification. However, readding just the first getDObject (TreeIter) call is sufficient to make it happen again, but this time not immediately after modifying the tree; it happens a few searches later, this time GDB is pointing its finger at the readded getDObject call . What however is strange, is that not only the same massive backtrace as above is printed, but the backtrace also goes through the same steps:

1. _D7gobject7ObjectG7ObjectG94__T10getDObjectTC3gtk8TreeIter8TreeIterTC3gtk8TreeIter8TreeIterTPS4gtkc8gtktypes11GtkTreeIterZ10getDObjectFPS4gtkc8gtktypes11GtkTreeIterZC3gtk8TreeIter8TreeIter
2. ...
3. gc.gc.Gcx.fullcollect
4. rt_finalize2
5. gobject.ObjectG.ObjectG.~this
6. gobject.ObjectG.ObjectG.stealData
7. glib.Str.Str.toStringz
8. _d_newarrayiT
9. gc_qalloc
10. Loop a large amount of times with the four calls starting with gc.gc.GC.malloc and eventually run into a segmentation fault (SIGSEGV). Again, no other threads were running at the time.

The same seems to occur if I just use auto iter = new TextIter(rawIter). What seems to happen is that the allocation of a new TextIter causes the garbage collector to do a full garbage collection (so far: seems normal), which causes some ObjectG object to finalize (still normal) and end up in its destructor where the stealData call is performed, which eventually ends up in allocating more memory. One would think that the garbage collector enters some sort of infinite loop (IIRC it collects memory during memory (de)allocation) when finalizing an object during an allocation, another allocation happens in the destructor (the toStringz), which does the same thing over again. However, if this was the case one would expect more references to fullcollect.

[MikeWey]

I think the problem is the stealData call in the ObjectG destructor, it currently allocates memory.

[MikeWey]

Could you check if the issue is still present with commit: e4d4cef ?

[ghost]

I wasn't able to drop in the code as is, as GDC is complaining about not being able to cast a string to a char pointer, but I used the following code instead:

g_object_steal_data(gObject, cast(char*) "GObject");

With this in place, I seem to no longer be able to reproduce the issue; I can happily modify the model without the crash occurring, so this does indeed seem to fix it. I thought that D's garbage collector was aware and prepared for memory allocations during finalization, but this doesn't completely seem to be the case.

Many thanks, this seems to be the second time you've saved the day with a strange bug I was having 👍 . Haven't regretted using GtkD (and D, for that matter) once since I started my project.

[MikeWey]

Thanks.