Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dropping AsyncCore in the logger thread will cause panic #11

Closed
kennytm opened this issue Apr 7, 2019 · 5 comments
Closed

Dropping AsyncCore in the logger thread will cause panic #11

kennytm opened this issue Apr 7, 2019 · 5 comments

Comments

@kennytm
Copy link
Contributor

kennytm commented Apr 7, 2019

AsyncCore::drop (and AsyncGuard::drop) needs to join the logger thread. When join is called inside the logger thread itself, it will unconditionally panic.

The Drain::log() function is executed in the logger thread. Therefore, if we drop the AsyncCore inside log(), or if log() panics and we have a custom panic hook and we drop the AsyncCore in the hook, it will cause the EDEADLK crash.

Demo:

use slog::{info, o, Discard, Drain, Logger};
use slog_async::Async;
use std::panic::{set_hook, take_hook};
use std::sync::Mutex;
use std::time::Duration;

struct ErrDrain;

impl Drain for ErrDrain {
    type Ok = ();
    type Err = ();

    fn log(&self, _: &slog::Record, _: &slog::OwnedKVList) -> Result<(), ()> {
        Err(())
    }
}

lazy_static::lazy_static! {
    static ref GLOBAL_LOGGER: Mutex<Logger> = {
        let drain = Async::new(ErrDrain.fuse()).thread_name("slogger".to_string()).build().fuse();
        Mutex::new(Logger::root(drain, o!()))
    };
}

fn main() {
    let hook = take_hook();
    set_hook(Box::new(move |pi| {
        hook(pi);
        *GLOBAL_LOGGER.lock().unwrap() = Logger::root(Discard, o!());  // <--- this
    }));

    info!(GLOBAL_LOGGER.lock().unwrap(), "should panic");
    std::thread::sleep(Duration::from_secs(1));
}

The marked line will panic on Linux and macOS.

@dpc
Copy link
Contributor

dpc commented Apr 8, 2019

Yeah. Don't do it. 😁

What are you trying to do exactly? :D

@kennytm
Copy link
Contributor Author

kennytm commented Apr 8, 2019

@dpc As shown in the example there is a global logger for code simplicity.

There is also panic hook is to capture any uncaught panics into the log file before killing the process. During the time between flushing and log and exiting, there might still be some additional logs coming from other threads. We do not want to lose them, and therefore replaced the global logger with the one that logs to stderr.

And that turns out to cause EDEADLK when the panicking thread is the logger thread 🙃.

I'm checking if this issue is the expected behavior before changing the drainer implementation.

@dpc
Copy link
Contributor

dpc commented Apr 8, 2019

Global logger behind a mutex? Oh dear.

Well, dropping AsyncCore will make it wait for the the worker thread to finish ... but how is it going to finish if it's the thread that is waiting? :D

@dpc
Copy link
Contributor

dpc commented Apr 8, 2019

@dpc
Copy link
Contributor

dpc commented Apr 8, 2019

I can't think of any good excuse not to include such "fix". Can you fill a PR? In the meantime I'll think about it a little.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants