05 Performance Considerations

JP Toto edited this page Apr 14, 2017 · 9 revisions

THIS IS IMPORTANT

Volatile Data and Buffering

Because log4net.ElasticSearch inherits from the BufferingAppenderSkeleton class, there is a potential "gotcha" that will reduce performance, potentially drastically, in a high volume application.

BufferingAppenderSkeleton stores messages in a buffer before flushing them to the target, in this case Elasticsearch. However, since the messages can sit in the buffer for an unspecified amount of time, some of the field data is considered "volatile" and could potentially change and become inaccurate before the buffer is flushed. For each flush, a FixVolatileData() method is called for each event which confirms the current identity of the computer, thread name, etc. This is a very expensive operation and can drastically slow down performance of your application.

If you don't need the data that is corrected by the FixVolatileData() method, you can have log4net ignore it in your appender configuration by adding:

<Fix value="0"/> <!-- Set Fix flag to NONE -->

That setting will tell log4net not to "fix" any of the volatile data thereby speeding up operations (if they've slowed down for you). Your final appender configuration may look like this:

<appender name="ElasticSearchAppender" type="log4net.ElasticSearch.ElasticSearchAppender, log4net.ElasticSearch">
    <connectionString value="Server=localhost;Index=log;"/>
    <lossy value="false" />
    <evaluator type="log4net.Core.LevelEvaluator">
        <threshold value="ERROR" />
    </evaluator>
    <bufferSize value="100" />
    <Fix value="0"/>
</appender>

This issue isn't documented very well in log4net but for a complete explanation of the options and solutions, please see this excellent StackOverflow post . It details how you may try adding some of the fixed data back in if you need it if it doesn't affect performance.

log4net Calls are Blocking

Calls to log4net are blocking on the log4net side and log4net.Elasticsearch does not use asynchronous or background threads to make http calls in order to maintain .NET 4.0 support (for now). If your Elasticsearch server is not available to receive a log call, you may see significant performance issues.

You probably shouldn't use this in production

log4net.Elasticsearch relies on Elasticsearch to be available during the logging call and if it's not, you could see performance issues due to the above warnings. I don't suggest using this library in a production scenario for that reason. Instead, I would recommend writing logs to a file, then using a utility like Filebeat to send your logs either directly to Elasticsearch or through logstash first so the data may be massaged or enhanced. Filebeat and logstash will both throttle calls to Elasticsearch if it becomes unavailable due to a network segmentation, outage, or some other catastrophe.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.