02 Appender Settings

Appenders in log4net are how you configure logging output. There are built in appenders for logging messages to text files, SMTP, SQL databases, and more. log4net.ElasticSearch is an appender for sending log4net style logs directly to Elasticsearch and the appender is configured accordingly.

Below is an example appender configuring for log4net.Elasticsearch added an an App.config file. This can be added to an App.config file, Web.config, or its own configuration xml file and referenced in code.

<?xml version="1.0"?>
<configuration>
    <configSections>
        <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
    </configSections>
    <startup>
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
    </startup>
    <log4net>
        <appender name="ElasticSearchAppender" type="log4net.ElasticSearch.ElasticSearchAppender, log4net.ElasticSearch">
        <!-- for .NET 40 <appender name="ElasticSearchAppender" type="log4net.ElasticSearch.ElasticSearchAppender, log4net.ElasticSearch.Net40">-->
            <connectionString value="Scheme=https;User=username;Pwd=password;Server=localhost;Index=log;Port=9200;rolling=true"/>
            <lossy value="false" />
            <evaluator type="log4net.Core.LevelEvaluator">
                    <threshold value="ERROR" />
            </evaluator>
            <bufferSize value="100" />
        </appender>
        <root>
            <level value="ALL"/>
            <appender-ref ref="ElasticSearchAppender" />
        </root>
    </log4net>
</configuration>

The important part starts with the <log4net> attribute.

The <appender> node begins the configuration. The name specified is ElasticSearchAppender. This can be called anything you like but I recommend keeping it the default since it needs to be referenced later in your log4net configuration.

The <layout> describes how log4net will output fields and is standard for appenders. You can refer to the log4net documentation for more details.

The <connectionString> block contains a lot of information specific to the log4net.ElasticSearch appender:

• schema: (optional) The http protocol used to send requests to Elasticsearch. May be http or https
• user/password: (optional but must be used together) If your Elasticsearch installation is using a plugin like Shield for security, this is where you'd enter your http credentials to allow the appender to access Elasticsearch
• server: The hostname or IP of your Elasticsearch server
• port: (optional) Set to 9200 which is the default for Elasticsearch or whatever port your ES node is listening on
• rolling: (optional) Set to true or false (default is false) Will cause a new index to be created daily in a "rolling indices" format. The date will be appended to each new index name so that you get one per day. The old ttl field is not supported since the guidance from the Elasticsearch team is to use rolling indices and then a tool like Curator to remove old expired indices.

<lossy value="false" /> log4net.ElasticSearch uses a buffer to collect events and then flush them to the Elasticsearch server on a background thread. Setting this value to true will cause log4net.Elasticsearch to begin discarding events if the buffer is full and has not been flushed. This could happen if the Elasticsearch server becomes unresponsive or goes offline. If application performance is paramount, set this to true so that events are discarded from the buffer but the logging is not blocking application performance. If logging each event is more important than application performance, set this to false. You must have an evaluator block if lossy is set to true.

<bufferSize value="100" /> The bufferSize tells log4net.ElasticSearch how many events to store before flushing to the Elasticsearch server. You should experiment with this figure but generally if you have a high volume application with lots of logging you can set this figure higher. If you log very infrequently, set this number lower so that the buffer is flushed often enough to see the events show up in Elasticsearch. A buffer size set greater than 1 will automatically cause log4net.ElasticSearch to use the _bulk API for more efficient logging.

<evaluator type="log4net.Core.LevelEvaluator">
  <threshold value="ERROR"/>
</evaluator>

The evaluator is a final check for events that are about to be discarded in a lossy setup. If the level of the incoming event is equal to or greater than the evaluator that is set, the buffer will trigger and an attempt will be made to log it to the Elasticsearch server. For example, if your threshold value is set to ALL, every single event will be logged to Elasticsearch each time one is come in and you lose the benefit of the buffer.

<root>
    <level value="ALL"/>
    <appender-ref ref="ElasticSearchAppender" />
</root>

The <root> is where, in log4net, you specify which appenders should be used per log level and per namespace. More information is available in the log4net configuration docs. If you'd like to log all messages to Elasticsearch, you can leave the configuration as shown here.

What about layout and ConversionPattern ?

Since log4net.ElasticSearch adds all the fields to the Elasticsearch logEvent document, formatting the message field becomes redundant. All the fields are available in the document so formatting isn't needed. You may select or search on whatever fields you'd like. Because of this, any kind of <layout> with a ConversionPattern is ignored.