Thread local variables do not get destroyed upon joinThread calls

[PhilippMDoerner]

Description

Apparently (?) thread-local variables do not get collected/destroyed when a thread is joined with another thread via joinThread.
This was the result of a forum discussion with mratsim and a discord discussion with leorize

A minimal example to demonstrate the general problem:

# Example1
var thread: Thread[void]

type MyType = object

var destructionCounter: int = 0

proc `=destroy`(x: MyType) =
    destructionCounter.inc

var localMyType {.threadVar.}: MyType
    
proc threadProc() =
    localMyType = MyType()

thread.createThread(threadProc)

joinThread(thread)

assert destructionCounter == 2 # Call #1 during assignment, Call #2 when threadProc ends

This is a bit problematic particularly with async, because the global dispatcher from std/asyncdispatch is a thread-local variable. Therefore, when joining a thread that uses async you automatically leak that entire dispatcher (which appears to be around ~58KB in total when empty).

Click this for a second example for that scenario. You can copy paste and run this directly to see the leakage reports, just ensure you run this with the included command (make sure to use clang):

`nim r --cc:clang --mm:arc -d:release --debugger:native -d:useMalloc --passc:-fsanitize=address --passl:-fsanitize=address .nim`

# Example2
import std/asyncdispatch

var thread: Thread[void]

proc stuff() {.async.} =
    await sleepAsync(400)
    echo "End of Stuff"
    
proc threadProc() =
    asyncCheck stuff()
    poll()

thread.createThread(threadProc)

joinThread(thread)
echo "End of Thread"

Nim Version

Nim Compiler Version 2.0.2 [Linux: amd64]
Compiled at 2023-12-15
Copyright (c) 2006-2023 by Andreas Rumpf

git hash: c4c44d1
active boot switches: -d:release

Current Output

# This is for example 1
/home/philipp/dev/playground/src/playground.nim(19) playground
/home/philipp/.choosenim/toolchains/nim-2.0.2/lib/std/assertions.nim(41) failedAssertImpl
/home/philipp/.choosenim/toolchains/nim-2.0.2/lib/std/assertions.nim(36) raiseAssert
/home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/fatal.nim(53) sysFatal
Error: unhandled exception: /home/philipp/dev/playground/src/playground.nim(19, 1) `destructionCounter == 2`  [AssertionDefect]
Error: execution of an external program failed: '/home/philipp/.cache/nim/playground_d/playground_2C5DA2BC2B16D71ABD46BED2876B0C57809E1730'

Expected Output

# For example 1

Possible Solution

Ideally: A proc that cleans up all thread-local variables that gets implicitly called at the end of a thread's life.

Alternatively: Document this behaviour in std/threadtypes that you must clean up your thread-local variables at the end of the thread's life. Either in the module docs or the docs of proc createThread

Additional Information

Not a suggestion from me as I lack experience in these matters, but a suggestion from mratsim in the forum thread was:

I'm not too sure what's the correct fix here.
Maybe having a magic destroyThreadLocalVariables that can be called before joinThread or exiting a program.

Note that if the solution for this is to just provide docs for this behaviour I'm happy to provide a PR for this.

As a general workaround:

You can just "nil" ref variables such as that manually. In the case of the global dispatcher, which is a ref object, just call setGlobalDispatcher(nil)

Edit:
Just stumbled into 2 cases where it is impossible to not leak: The second times is involved (e.g. during logging with chronicles ) you will get values for these thread variables from std/times:

var utcInstance {.threadvar.}: Timezone
var localInstance {.threadvar.}: Timezone

These are module private and have no setters. They can not be destroyed, you are forced to have them leak.

Click here to see below address sanitizier stacktraces for reference

==1956732==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 56 byte(s) in 1 object(s) allocated from:
    #0 0x5605fb3d6a79 in __interceptor_malloc (/home/philipp/.cache/nim/ex_stdinput_customloop_r/ex_stdinput_customloop_B6A6F7BD959A6EC4A1345F3C0AC071454C91DD54+0x117a79) (BuildId: c6e2107289f8929d50310b047fd55b699a76c684)
    #1 0x5605fb42ec99 in allocImpl__system_u1761 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:5:11
    #2 0x5605fb42ec99 in allocSharedImpl /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:34:11
    #3 0x5605fb42ec99 in alignedAlloc__system_u1901 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/memalloc.nim:331:12
    #4 0x5605fb42ec99 in nimNewObjUninit /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/arc.nim:99:343
    #5 0x5605fb445fae in newTimezone__pureZtimes_u1538 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/times.nim:1216:84
    #6 0x5605fb445fae in local__pureZtimes_u1804 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/times.nim:1389:398
    #7 0x5605fb4467dd in local__pureZtimes_u1818 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/times.nim:1407:23
    #8 0x5605fb4467dd in now__pureZtimes_u1821 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/times.nim:1415:82
    #9 0x5605fb462079 in rfcTimestamp__OOZOOZOOZOnimbleZpkgsZchronicles4548O4948O50ZchroniclesZlog95output_u396 /home/philipp/.nimble/pkgs/chronicles-0.10.2/chronicles/log_output.nim:426:2
    #10 0x5605fb462079 in initLogRecord__ex95stdinput95customloop_u303 /home/philipp/.nimble/pkgs/chronicles-0.10.2/chronicles/log_output.nim:438:18
    #11 0x5605fb47d40f in runServerLoop__ex95stdinput95customloop_u3415 /home/philipp/.nimble/pkgs/chronicles-0.10.2/chronicles.nim:298:3
    #12 0x5605fb479a5f in serverProc__ex95stdinput95customloop_u5718 /home/philipp/dev/threadbutler/src/threadButler.nim:84:2
    #13 0x5605fb436fb2 in threadProcWrapDispatch__ex95stdinput95customloop_u5766 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/threadimpl.nim:69:2
    #14 0x5605fb436fb2 in threadProcWrapStackFrame__ex95stdinput95customloop_u5756 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/threadimpl.nim:95:2
    #15 0x5605fb422998 in threadProcWrapper__ex95stdinput95customloop_u5742 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/threadimpl.nim:101:2
    #16 0x7f9c71e0b9ea  (/usr/lib/libc.so.6+0x8c9ea) (BuildId: 8bfe03f6bf9b6a6e2591babd0bbc266837d8f658)

Direct leak of 56 byte(s) in 1 object(s) allocated from:
    #0 0x5605fb3d6a79 in __interceptor_malloc (/home/philipp/.cache/nim/ex_stdinput_customloop_r/ex_stdinput_customloop_B6A6F7BD959A6EC4A1345F3C0AC071454C91DD54+0x117a79) (BuildId: c6e2107289f8929d50310b047fd55b699a76c684)
    #1 0x5605fb42ec99 in allocImpl__system_u1761 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:5:11
    #2 0x5605fb42ec99 in allocSharedImpl /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:34:11
    #3 0x5605fb42ec99 in alignedAlloc__system_u1901 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/memalloc.nim:331:12
    #4 0x5605fb42ec99 in nimNewObjUninit /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/arc.nim:99:343
    #5 0x5605fb445fae in newTimezone__pureZtimes_u1538 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/times.nim:1216:84
    #6 0x5605fb445fae in local__pureZtimes_u1804 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/times.nim:1389:398
    #7 0x5605fb4467dd in local__pureZtimes_u1818 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/times.nim:1407:23
    #8 0x5605fb4467dd in now__pureZtimes_u1821 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/times.nim:1415:82
    #9 0x5605fb462079 in rfcTimestamp__OOZOOZOOZOnimbleZpkgsZchronicles4548O4948O50ZchroniclesZlog95output_u396 /home/philipp/.nimble/pkgs/chronicles-0.10.2/chronicles/log_output.nim:426:2
    #10 0x5605fb462079 in initLogRecord__ex95stdinput95customloop_u303 /home/philipp/.nimble/pkgs/chronicles-0.10.2/chronicles/log_output.nim:438:18
    #11 0x5605fb46e705 in readMsg__ex95stdinput95customloop_u2222 /home/philipp/.nimble/pkgs/chronicles-0.10.2/chronicles.nim:298:4
    #12 0x5605fb477c02 in runServerLoop__ex95stdinput95customloop_u5136 /home/philipp/dev/threadbutler/src/threadButler.nim:58:185
    #13 0x5605fb47956f in serverProc__ex95stdinput95customloop_u5133 /home/philipp/dev/threadbutler/src/threadButler.nim:84:2
    #14 0x5605fb431ee2 in threadProcWrapDispatch__ex95stdinput95customloop_u5536 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/threadimpl.nim:69:2
    #15 0x5605fb431ee2 in threadProcWrapStackFrame__ex95stdinput95customloop_u5526 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/threadimpl.nim:95:2
    #16 0x5605fb4224f8 in threadProcWrapper__ex95stdinput95customloop_u5512 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/threadimpl.nim:101:2
    #17 0x7f9c71e0b9ea  (/usr/lib/libc.so.6+0x8c9ea) (BuildId: 8bfe03f6bf9b6a6e2591babd0bbc266837d8f658)