Logging not working on production

[maqeelqureshi]

I need to shift my application logs from file to elk. Logging on elk is working fine on my local environment (without credentials) but its not working on production.

On production, if i just switch appender to RollingFileAppender text file logging works.

Following is the output of internal debugging.

`log4net: log4net assembly [log4net, Version=2.0.8.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a]. Loaded from [C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\root\38ce21d9\71672efd\assembly\dl3\3a61141d\00cb2cb9_2898d201\log4net.dll]. (.NET Runtime [4.0.30319.42000] on Microsoft Windows NT 10.0.17763.0)
log4net: defaultRepositoryType [log4net.Repository.Hierarchy.Hierarchy]
log4net: Creating repository for assembly [MyApp.API, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]
log4net: Assembly [MyApp.API, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] Loaded From [C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\root\38ce21d9\71672efd\assembly\dl3\836bf9fe\56a04992_59c9d501\MyApp.API.dll]
log4net: Assembly [MyApp.API, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] does not have a RepositoryAttribute specified.
log4net: Assembly [MyApp.API, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] using repository [log4net-default-repository] and repository type [log4net.Repository.Hierarchy.Hierarchy]
log4net: Creating repository [log4net-default-repository] using type [log4net.Repository.Hierarchy.Hierarchy]
log4net: configuring repository [log4net-default-repository] using file [C:\Deployment\Feature\web.config] watching for file updates
log4net: configuring repository [log4net-default-repository] using file [C:\Deployment\Feature\web.config]
log4net: configuring repository [log4net-default-repository] using stream
log4net: loading XML configuration
log4net: Configuring Repository [log4net-default-repository]
log4net: Configuration update mode [Merge].
log4net: Logger [root] Level string is [ALL].
log4net: Logger [root] level set to [name="ALL",value=-2147483648].
log4net: Loading Appender [EventsLog1] type: [log4net.ElasticSearch.ElasticSearchAppender,log4net.ElasticSearch]
log4net: Setting Property [ConnectionString] to String value [Scheme=http;Server=[ip];Index=[index name];Port=[port];rolling=true;User=[user];Pwd=[pwd]]
log4net: Setting Property [Lossy] to Boolean value [False]
log4net: Setting Property [BufferSize] to Int32 value [10]
log4net: Setting Property [Fix] to FixFlags value [None]
log4net: Created Appender [EventsLog1]
log4net: Adding appender named [EventsLog1] to logger [root].
log4net: Hierarchy Threshold []

log4net: configuring repository [log4net-default-repository] using .config file section
log4net: Application config file is [C:\Deployment\Feature\web.config]
log4net: Configuring Repository [log4net-default-repository]
log4net: Configuration update mode [Merge].
log4net: Logger [root] Level string is [ALL].
log4net: Logger [root] level set to [name="ALL",value=-2147483648].
log4net: Loading Appender [EventsLog1] type: [log4net.ElasticSearch.ElasticSearchAppender,log4net.ElasticSearch]
log4net: Setting Property [ConnectionString] to String value [Scheme=http;Server=[ip];Index=[index name];Port=[port];rolling=true;User=[user];Pwd=[pwd]]
log4net: Setting Property [Lossy] to Boolean value [False]
log4net: Setting Property [BufferSize] to Int32 value [10]
log4net: Setting Property [Fix] to FixFlags value [None]
log4net: Created Appender [EventsLog1]
log4net: Adding appender named [EventsLog1] to logger [root].
log4net: Hierarchy Threshold []

log4net: Creating repository for assembly [MyApp.Utils, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]
log4net: Assembly [MyApp.Utils, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] Loaded From [C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\root\38ce21d9\71672efd\assembly\dl3\7cea7f47\847a6e5b_58c9d501\MyApp.Utils.dll]
log4net: Assembly [MyApp.Utils, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] does not have a RepositoryAttribute specified.
log4net: Assembly [MyApp.Utils, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] using repository [log4net-default-repository] and repository type [log4net.Repository.Hierarchy.Hierarchy]
log4net: repository [log4net-default-repository] already exists, using repository type [log4net.Repository.Hierarchy.Hierarchy]
log4net: ConfigureAndWatchHandler: Deleted [C:\Deployment\Feature\web.config]
log4net: Shutdown called on Hierarchy [log4net-default-repository]
`

Following is the configuration

<root>
  <level value="ALL" />
  <appender-ref ref="EventsLog" />
</root>
<appender name="EventsLog" type="log4net.ElasticSearch.ElasticSearchAppender, log4net.ElasticSearch">
  <connectionString value="Scheme=http;Server=[ip];Index=[index name];Port=[port];rolling=true;User=[user];Pwd=[pwd]"/>
  <lossy value="false" />
  <!-- Discard packects if srver is not available-->
  <bufferSize value="10" />
  <!-- Buffer batch size-->
  <Fix value="0"/>
  <!--Discard packects those are corrupted -->
</appender>    

<!--<root>
  <level value="ALL"/>
  <appender-ref ref="EventsLog"/>
</root>
<appender name="EventsLog" type="log4net.Appender.RollingFileAppender">
  <file type="log4net.Util.PatternString" value="C:\Log\Feature\EventsLog.xml"/>
  <staticLogFileName value="false"/>
  <preserveLogFileNameExtension value="true"/>
  <appendToFile value="true"/>
  <rollingStyle value="Date"/>
  <datePattern value="_yyyyMMdd"/>
  <layout type="MyApp.Utils.Log.XMLLayout">
    <header>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;LogEntries&gt;</header>
    <footer>&lt;/LogEntries&gt;</footer>
  </layout>
</appender>-->

Note : Browsing to ELK or curl to elasticsearch works fine from production server.

Update: I have also tried setup local elasticsearch docker instance with credentials. Logging works fine on that also. I was thinking that there might be some problem with connection string on production.

Update: When I change the logger name and user/pwd of elasticsearch, sometimes it works. I have no clue what is happening.

[bruno-garcia]

Often production server require a proxy to access the Internet. With .NET Framework you can set it up via app.config or web.config (or even machine.config).
Other than that it's hard to help troubleshoot these issues since it's probably some configuration in your prod environment.

Lastly the sometimes it works is a bit more worrying. Can you reproduce that also with an instance of elastic on docker? Some steps to reproduce would help

[Abouzeid]

I faced a similar issue today. I'm posting here, as it might save someone's time (and sanity).

ElasticSearch: 7.13.2
log4net.Elasticsearch: 2.5.0
Net framework: 4.6.2
Schema: HTTP

Elasticsearch server returned the "401 unauthorized " in the GetResponse method

log4net.ElasticSearch/src/log4net.ElasticSearch/Infrastructure/HttpClient.cs

Line 43 in d5f2b88

using (var httpResponse = (HttpWebResponse) httpWebRequest.GetResponse())

However, the exception doesn't show up in any log.

The main reason

The user, whose credential specified in the connection string, didn't have the correct index privilege permission. See ElasticSearch docs here
Also, don't include any special characters in the password.

Tip: at first, give the user the superuser role, just to see the data on ElasticSearch. After that, narrow the permission down as you want. (try first with no special char in the password)