EnvFilter does not implement Filter

[lilyball]

Bug Report

Version

tracing-subscriber v0.3.7

Crates

tracing-subscriber

Description

tracing_subscriber::layer::Filter is implemented for most various filter-like things I can think of, including LevelFilter, Targets, FilterFn, DynFilterFn, and the new combinators. But what is notably missing from this list is EnvFilter. I can apply a filter on top of EnvFilter using <EnvFilter as Layer>::with_filter(), but I can't apply an EnvFilter on top of an existing filter. This is a weird limitation and feels like a simple oversight.

[Eugeny]

Workaround:

    let env_filter = Arc::new(EnvFilter::from_default_env());
    let fmt_layer = tracing_subscriber::fmt::layer()
        .with_filter(dynamic_filter_fn(move |m, c| {
            env_filter.enabled(m, c.clone())
        }));

[hawkw]

This is a weird limitation and feels like a simple oversight.

Unfortunately, this is more than a simple oversight. EnvFilter does not implement the Filter trait because it implements dynamic filtering that requires keeping track of the currently entered span scope. At the moment, it does that through its implementation of the Layer trait, which allows it to implement methods like new_span, on_enter, and on_exit hooks.

In order to make EnvFilter implement Filter, it would need to be re-written to fit the Filter interface. Currently, Filters don't have callbacks for recording spans being created, entered, exited, or closed. Instead, the current trait requires that they query the wrapped Subscriber to get the current span scope. We could potentially change the implementation of EnvFilter to do this, but it hasn't happened yet because it's a large rewrite.

Alternatively, we could consider adding optional new_span, on_enter, on_exit, and on_close callbacks to the Filter trait, as well. This might be a good idea, as it would make it easier for Filters to implement dynamic span based filtering without having to ask the wrapped subscriber for the current context.

@Eugeny

Workaround:

    let env_filter = Arc::new(EnvFilter::from_default_env());
    let fmt_layer = tracing_subscriber::fmt::layer()
        .with_filter(dynamic_filter_fn(move |m, c| {
            env_filter.enabled(m, c.clone())
        }));

This workaround will work, _provided that the EnvFIlter is also added as a global filter layer. If it's not, it will not receive the new_span, on_enter, on_exit, and on_close callbacks it requires to perform dynamic filtering based on the span context. But, in that case, it will also perform global filtering, which defeats the point of also using it as a per-layer filter. In order for this workaround to actually work correctly, you would also need to wrap the EnvFilter that's added as a Layer in a wrapper struct that forwards all the Layer methods, but doesn't return false from enabled to disable a span or event globally. That way, you could use this workaround to use EnvFilter as a per-layer filter while still having dynamic span filtering work correctly.

[Eugeny]

@hawkw I can't explain why, but it works without adding it as a filter. Here's my complete setup:

fn init_logging() {
    if std::env::var("RUST_LOG").is_err() {
        std::env::set_var("RUST_LOG", "xxxxx=info")
    }

    let offset =
        UtcOffset::current_local_offset().unwrap_or(UtcOffset::from_whole_seconds(0).unwrap());

    let env_filter = Arc::new(EnvFilter::from_default_env());
    let fmt_layer = tracing_subscriber::fmt::layer()
        .with_timer(OffsetTime::new(
            offset,
            format_description::parse("[day].[month].[year] [hour]:[minute]:[second]").unwrap(),
        ))
        .with_filter(dynamic_filter_fn(move |m, c| {
            env_filter.enabled(m, c.clone())
        }));

    let r = tracing_subscriber::registry();

    #[cfg(debug_assertions)]
    let console_layer = console_subscriber::spawn();

    #[cfg(debug_assertions)]
    let r = r.with(console_layer);

    r.with(fmt_layer).init();
}

I'm using Registry to enable both FmtLayer and tokio-console.

With the above code, I can use RUST_LOG to filter things that go into the fmt layer, while console-subscriber still gets all spans (including trace level which is required for tokio-console to work).

[lilyball]

@Eugeny Does it work for filter strings that depend on span names/fields?

[Eugeny]

I've just started out using tracing, so not sure about that. It works when filtering by module name/prefix if that's what you mean.

[hawkw]

It works when filtering by module name/prefix if that's what you mean.

I would expect this to work with the approach you are using.

It's specifically filters that match on the current spans (anything using the [...] syntax as described here) that will not work with this approach. If you don't need dynamic filtering based on spans, though, you could just use the Targets filter, instead; it implements the same module name prefix-based filtering as EnvFilter, but does not implement dynamic filtering based on spans.

Thus, the workaround does work with basic usage, but not for the features that are EnvFilter-specific. If dynamic filtering based on spans is not needed, though, you can just use Targets, which does implement the Filter trait, so there isn't actually a reason to use this workaround without the additional changes I described.

Does that help clear things up?

[Eugeny]

Yep that explains it, thanks!