Common problems

It's common practice for library creators to be mindful of the expected behavior of their code. Well designed libraries are intuitive and introduces no surprises to its consumers.

Issues created in this repository indicate that there are some aspects of this sink that aren't intuitive. The intention with this page is to describe these unintuitive behaviors and why they exist.

Problems

Sending log events doesn't seem to work?

Lets back up somewhat and start with Serilog's take on reliability.

Serilog takes the view that, all things considered, logging is a lower priority than other application code and should never avoidably impact the operation of a running application.

This means that Serilog itself, as well as its sinks, shouldn't throw exceptions if it can be avoided. Well, if the sink isn't behaving as expected, how should I debug and diagnose it? SelfLog to the rescue. All sinks log exceptions to SelfLog. Its a bit like the Russian nesting doll, a logger within a logger 😀

Read Debugging and Diagnostics and Reliability on Serilog wiki for more information.

My console application isn't sending the log events to the Loki?

Sending log events over the network is an asynchronous operation. Mainly because I/O is an extremely slow operation, but also since the sink is batching multiple log events into a single HTTP request.

Before terminating the console application, make sure to flush Serilog.

ILogger log = new LoggerConfiguration()
  .WriteTo.GrafanaLoki("http://localhost:3100")
  .CreateLogger();

log.Information("Hello!");

Log.CloseAndFlush();

Read Lifecycle of Loggers on Serilog wiki for more information.

Sink stopped working after error or some log events were missed?

This could happens due to log rejection from Loki side or network troubles.

In case of unsuccessful delivery, which could be caused by Loki rejection or exception in the sink, we are using ExponentialBackoffConnectionSchedule for retries.

So the next situations are possible:

• Logs rejection by Loki. This could be caused by Loki misconfiguration (for example, the allowed length of label is less then sent). The batch of rejected entries will be dropped. Next batch will be delivered after next interval. Loki configuration guide could be useful in this situation.
• Exception raised in the sink. In this case, waiting batch will be kept and retried after an interval. Now we don't provide an unique logic for different exceptions, but this behavior could be changed in the future. This behavior will be useful when you are facing some infrastructure issues and Loki is unavailable.

For both cases SelfLog function will be very useful to determine the source of a problem

High cardinality problems

Please check that you follow label best practices. Also a Configuration pages from documentation could be useful