segfault while gziping file in other thread

[tchaloupka]

When trying to gzip some file in a separate thread it leads to segfault on GC call when it tries to release file handle.
It works with old vibe-d:core.

vibe-core: 1.6.0
vibe-d: 0.8.5-beta.1
os: fedora linux x86_64

Testcase

#!/usr/bin/env dub
/+ dub.sdl:
	name "gzipseg"
	dependency "vibe-d:stream" version="~>0.8.5-beta.1"
+/

import core.thread;
import std.stdio;
import vibe.core.file : FileMode, openFile;
import vibe.stream.wrapper : pipe;
import vibe.stream.zlib : createGzipOutputStream;

enum fname = "foo.txt";

void main()
{
	auto th = new Thread(
		{
			"foo".toFile(fname);
			auto fin = openFile(fname);
			auto fout = openFile(fname ~ ".gz", FileMode.append);
			auto gz = createGzipOutputStream(fout);
			fin.pipe(gz);
			gz.finalize();
			fin.close();
			fout.close();
		}
	);
	th.start();
	th.join();
}

Leads to:

#0  pure nothrow @nogc @safe shared(core.sync.mutex.Mutex) core.atomic.atomicLoad!(3, core.sync.mutex.Mutex).atomicLoad(ref shared(const(core.sync.mutex.Mutex))) (val=<error reading variable>)
    at /usr/include/dmd/druntime/import/core/atomic.d:1133
#1  0x00000000005fb46d in shared nothrow @safe void eventcore.drivers.posix.driver.PosixEventDriverCore!(eventcore.drivers.posix.epoll.EpollEventLoop, eventcore.drivers.timer.LoopTimeoutTimerDriver, eventcore.drivers.posix.events.PosixEventDriverEvents!(eventcore.drivers.posix.epoll.EpollEventLoop, eventcore.drivers.posix.sockets.PosixEventDriverSockets!(eventcore.drivers.posix.epoll.EpollEventLoop).PosixEventDriverSockets).PosixEventDriverEvents).PosixEventDriverCore.runInOwnerThread(void function(long) nothrow @safe*, long) (this=0x0, param=20,
    del=0x5abf44 <nothrow @safe void vibe.core.internal.release.releaseHandle!("files", eventcore.driver.FileFD).releaseHandle(eventcore.driver.FileFD, shared(eventcore.drivers.posix.driver.PosixEventDriver!(eventcore.drivers.posix.epoll.EpollEventLoop).PosixEventDriver)).__lambda4!(long).__lambda4(long)>) at ../../../.dub/packages/eventcore-0.8.40/eventcore/source/eventcore/drivers/posix/driver.d:247
#2  0x00000000005abf33 in nothrow @safe void vibe.core.internal.release.releaseHandle!("files", eventcore.driver.FileFD).releaseHandle(eventcore.driver.FileFD, shared(eventcore.drivers.posix.driver.PosixEventDriver!(eventcore.drivers.posix.epoll.EpollEventLoop).PosixEventDriver)) (drv=0x7ffff0000ec0, handle=...) at ../../../.dub/packages/vibe-core-1.6.0/vibe-core/source/vibe/core/internal/release.d:15
#3  0x00000000005ab586 in vibe.core.file.FileStream.~this() (this=...) at ../../../.dub/packages/vibe-core-1.6.0/vibe-core/source/vibe/core/file.d:449
#4  0x000000000059a139 in @safe void object._destructRecurse!(vibe.core.file.FileStream)._destructRecurse(ref vibe.core.file.FileStream) (s=...) at /usr/include/dmd/druntime/import/object.d:516
#5  0x000000000059a0c1 in @safe void object.destroy!(vibe.core.file.FileStream).destroy(ref vibe.core.file.FileStream) (obj=...) at /usr/include/dmd/druntime/import/object.d:501
#6  0x0000000000597ba0 in nothrow @trusted void vibe.internal.interfaceproxy.InterfaceProxy!(vibe.core.stream.OutputStream).InterfaceProxy.ProxyImpl!(vibe.core.file.FileStream).ProxyImpl._destroy(void[]) (
    this=0x7ffff7acf420, stor=...) at ../../../.dub/packages/vibe-core-1.6.0/vibe-core/source/vibe/internal/interfaceproxy.d:244
#7  0x0000000000595912 in nothrow @safe void vibe.internal.interfaceproxy.InterfaceProxy!(vibe.core.stream.OutputStream).InterfaceProxy.clear() (this=...)
    at ../../../.dub/packages/vibe-core-1.6.0/vibe-core/source/vibe/internal/interfaceproxy.d:123
#8  0x00000000005958a5 in nothrow @safe void vibe.internal.interfaceproxy.InterfaceProxy!(vibe.core.stream.OutputStream).InterfaceProxy.__dtor() (this=...)
    at ../../../.dub/packages/vibe-core-1.6.0/vibe-core/source/vibe/internal/interfaceproxy.d:112
#9  0x00000000005a227d in vibe.stream.zlib.ZlibOutputStream.__fieldDtor() (this=0x7ffff7ad4000) at ../../../.dub/packages/vibe-d-0.8.5-beta.1/vibe-d/stream/vibe/stream/zlib.d:138
#10 0x00000000005a229e in vibe.stream.zlib.ZlibOutputStream.__aggrDtor() (this=0x7ffff7ad4000) at ../../../.dub/packages/vibe-d-0.8.5-beta.1/vibe-d/stream/vibe/stream/zlib.d:138
#11 0x0000000000631505 in rt_finalize2 ()
#12 0x00000000006744ae in rt_finalizeFromGC ()
#13 0x000000000066d5f6 in nothrow ulong gc.impl.conservative.gc.Gcx.sweep() ()
#14 0x000000000066df36 in nothrow ulong gc.impl.conservative.gc.Gcx.fullcollect(bool) ()
#15 0x0000000000670806 in nothrow ulong gc.impl.conservative.gc.ConservativeGC.runLocked!(gc.impl.conservative.gc.ConservativeGC.fullCollectNoStack().go(gc.impl.conservative.gc.Gcx*), gc.impl.conservative.gc.Gcx*).runLocked(ref gc.impl.conservative.gc.Gcx*) ()
#16 0x000000000066a975 in nothrow void gc.impl.conservative.gc.ConservativeGC.fullCollectNoStack() ()
#17 0x000000000066a946 in nothrow void gc.impl.conservative.gc.ConservativeGC.collectNoStack() ()
#18 0x0000000000668dcd in gc_term ()
#19 0x000000000062f287 in rt_term ()
#20 0x000000000062f7c5 in rt.dmain2._d_run_main(int, char**, extern(C) int(char[][]) function).runAll() ()
#21 0x000000000062f6a0 in rt.dmain2._d_run_main(int, char**, extern(C) int(char[][]) function).tryExec(scope void() delegate) ()
#22 0x000000000062f608 in _d_run_main ()
#23 0x000000000059de4a in main ()
#24 0x00007ffff7bf4413 in __libc_start_main () from /usr/lib64/libc.so.6
#25 0x0000000000585b6e in _start ()

[s-ludwig]

The problem that happens here is that the FileStream is freed by the GC at program exit, because the GZipOutputStream is GC allocated. However, the owning thread has already terminated, so the handle cannot be freed there anymore (which is needed for eventcore's current threading model).

There are three fixes for this that should all be implemented:

• Detect that the owner thread is terminated and leak the handle instead of crashing
• Switch ZlibStream to an RC struct
• Let ZlibStream.finalize clear the underlying stream handle

[s-ludwig]

The first issue is resolved by #164. Closing this while moving the other two to a ticket in vibe-d: vibe-d/vibe.d#2325