You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I recently had an issue where my tokio-based application would often crash before exiting with the relatively unhelpful error message:
cannot access a TLS value during or after it is destroyed: AccessError
While my application did use thread-local storage (TLS), it did not use LocalKey.with (which is where this error originates, so I was pretty confused what was causing this. After some gdb debugging, I discovered that the offending with call was located within tokio (fixed in #628). Following some discussion on the tokio gitter channel, @stjepang then pointed out:
@jonhoo: Well, I do store a thing that holds a Runtime inside of a TLS (specifically, in a RefCell). Is that an issue? @stjepang: That is probably it then :) Well, the Tokio runtime heavily uses TLS and cannot be dropped properly while the TLS is already tearing down.
This came as a surprise to me, and I do not believe it is documented anywhere at the moment. My code did indeed have something that amounts to:
Which I set to Some during setup, but never explicitly dropped, instead expecting it to be cleaned up automatically during thread teardown. This is indeed what happened, but apparently tokio doesn't expect to have to do work while thread locals are being destroyed (which is what caused this issue). The fix is pretty straightforward — just explicitly set the thread local to None before the thread exits to trigger a drop — but realizing that this is the issue in the first place requires a lot of debugging.
So, all that to say that I think tokio needs to either make it very clear that tokio and its components cannot be dropped inside of TLS (if so, where do we document this), or we need to fix the tokio components where this is broken (e.g., #593 (comment)) so that this works as expected. While the latter is more appealing as a user, it is also likely a lot more work. I think either is acceptable, since the workaround is also usually quite straightforward.
If we do opt to disallow dropping Runtime (others?) in TLS teardown, then ideally we should also detect it when this happens if possible. I don't know exactly how we'd do that (can you detect that you're in TLS teardown), but if tokio could at that point scream "DON'T DROP ME IN TLS", that would certainly help avoid users spending hours on trying to track down this issue like I had to!
The text was updated successfully, but these errors were encountered:
Ah I concur, I did encounter the same issue and as TLS dtors order is out of control, at the moment at least docs should be updated to state against it.
On other hand there is another option for user.
For example I did worked around this limitation by introducing special struct that cleans up TLS in its dtor:
thread_local!(staticTOKIO: Cell<Option<Runtime>> = Cell::new(None));pubstructGuard{//workaround for unstable !Send_dummy:PhantomData<*mutu8>}implDropforGuard{fndrop(&mutself){TOKIO.with(|rt| rt.replace(None));}}
You must make sure to not let it be Send, but other than that just by creating at the beggining of main would guarantee you that tokio's Runtime goes out of scope before tokio's internal TLS
I recently had an issue where my tokio-based application would often crash before exiting with the relatively unhelpful error message:
While my application did use thread-local storage (TLS), it did not use
LocalKey.with
(which is where this error originates, so I was pretty confused what was causing this. After somegdb
debugging, I discovered that the offendingwith
call was located withintokio
(fixed in #628). Following some discussion on the tokio gitter channel, @stjepang then pointed out:This came as a surprise to me, and I do not believe it is documented anywhere at the moment. My code did indeed have something that amounts to:
Which I set to
Some
during setup, but never explicitly dropped, instead expecting it to be cleaned up automatically during thread teardown. This is indeed what happened, but apparently tokio doesn't expect to have to do work while thread locals are being destroyed (which is what caused this issue). The fix is pretty straightforward — just explicitly set the thread local toNone
before the thread exits to trigger adrop
— but realizing that this is the issue in the first place requires a lot of debugging.So, all that to say that I think
tokio
needs to either make it very clear that tokio and its components cannot be dropped inside of TLS (if so, where do we document this), or we need to fix the tokio components where this is broken (e.g., #593 (comment)) so that this works as expected. While the latter is more appealing as a user, it is also likely a lot more work. I think either is acceptable, since the workaround is also usually quite straightforward.If we do opt to disallow dropping
Runtime
(others?) in TLS teardown, then ideally we should also detect it when this happens if possible. I don't know exactly how we'd do that (can you detect that you're in TLS teardown), but iftokio
could at that point scream "DON'T DROP ME IN TLS", that would certainly help avoid users spending hours on trying to track down this issue like I had to!The text was updated successfully, but these errors were encountered: