Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

lager_error_logger_h dropped X messages #180

Open
Andy-Richards opened this Issue · 5 comments

4 participants

@Andy-Richards

I randomly see warning messages reported from Lager 2.0.X as follows...

[warning] lager_error_logger_h dropped 633 messages in the last second that exceeded the limit of 50 messages/sec

As per this mailing list post http://erlang.org/pipermail/erlang-questions/2013-May/073773.html

Being random its difficult to produce a reproducible test case however I frequently see this warning when starting up an application where I assume there is considerable error_logger/SASL logging going when starting supervisors, gen_servers etc.

Thanks,

Andy.

@Vagabond
Collaborator

Yes, it will happen when something like Riak starts up, because error_logger spams with startup messages. I have occasionally seen possibly spurious messages like that mailing list post describes, but have been unable to track it down.

I'm sort of waiting for more information about the problem to accue, because I don't have any ideas on it right now.

@rlipscombe

I'm seeing this in an Erlang application that's fairly simple -- doesn't use Riak, for example.

I'm seeing "lager_error_logger_h dropped 15 messages in the last second that exceeded the limit of 50 messages/sec". Not sure if the first number is meant to be the excess, but 15 is not more than 50.

@grahamrhay

The problem is that the message is not displayed until an event arrives in a second later than the one in which the hwm was breached. So, if your app startup creates more than 50 messages (quite likely), but they all occur within one second; and the next event is not for a while (because nothing is happening), then you don't get the "dropped X messages" message until some time later. Which makes it look spurious, but it's technically correct, just not referring to the second you think it is.

Unfortunately I don't really have any suggestions to fix it, other than starting a timer, which I'm not sure is a good idea?

@rlipscombe

If the warning could refer to the second for which it dropped the messages, that'd be less confusing, potentially...

@grahamrhay

It also doesn't take into account whether the event would actually have been logged. If ?SHOULD_LOG returns false, then ?LOGFMT (or ?LOGMSG) is effectively a no-op. Wouldn't it make more sense to check that before counting the event towards the high water mark? The only thing I can see that that would affect is the crash log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.