Configure columnOptionsSection in Json doesn't work

[Luxuride]

I have serilog config in Json. When I'm trying to add/remove columns, nothing happens.

---

Dependencies

Microsoft.AspNetCore.All (2.1.0)
Serilog.AspNetCore (2.1.1)
Serilog.Settings.Configuration (2.6.1)
Serilog.Sinks.MSSqlServer (5.1.2)

---

logsettings.json have Copy always (After config change Clean -> Rebuild)

---

Calling config

IConfiguration config = new ConfigurationBuilder()
        .AddJsonFile("logsettings.json", optional: true, reloadOnChange: true)
        .Build();

Log.Logger = new LoggerConfiguration()
         .ReadFrom.Configuration(config)
         .CreateLogger();

---

Config

{
    "Serilog": {
        "Using": [ "Serilog.Sinks.MSSqlServer" ],
        "MinimumLevel": {
            "Default": "Information",
            "Override": {
                "Microsoft": "Warning"
            }
        },
        "WriteTo": [
            {
                "Name": "MSSqlServer",
                "Args": {
                    "connectionString": "Server=localhost; Database=log; MultipleActiveResultSets=true; User ID=sa; Password=Admin1234",
                    "tableName": "Logs",
                    "autoCreateSqlTable": true,
                    "columnOptionsSection": {
                        "customColumns": [
                            {
                                "ColumnName": "Application",
                                "DataType": "string"
                            },
                            {
                                "ColumnName": "SourceContext",
                                "DataType": "string"
                            },
                            {
                                "ColumnName": "ActionId",
                                "DataType": "string"
                            },
                            {
                                "ColumnName": "ActionName",
                                "DataType": "string"
                            },
                            {
                                "ColumnName": "RequestId",
                                "DataType": "string"
                            },
                            {
                                "ColumnName": "RequestPath",
                                "DataType": "string"
                            },
                            {
                                "ColumnName": "ConnectionId",
                                "DataType": "string"
                            }
                        ], //Custom columns aren't in database
                        "addStandardColumns": [ "LogEvent" ], //LogEvent column still isn't in database
                        "removeStandardColumns": [ "MessageTemplate", "Properties" ], //Properties column is still in database
                        "logEvent": {
                            "columnName": "LogEvent",
                            "excludeAdditionalProperties": true
                        }
                    }
                }
            }
        ]
    }
}

---

Even config template did not work

{
"Serilog": {
    "Using":  ["Serilog.Sinks.MSSqlServer"],
    "MinimumLevel": "Debug",
    "WriteTo": [
    { "Name": "MSSqlServer", 
        "Args": { 
            "connectionString": "Server=localhost; Database=log; MultipleActiveResultSets=true; User ID=sa; Password=Admin1234",
            "tableName": "Logs",
            "autoCreateSqlTable": true,
            "columnOptionsSection": {
            "customColumns": [
                { "ColumnName": "EventType", "DataType": "int", "AllowNull": false },
                { "ColumnName": "Release", "DataType": "varchar", "DataLength": 32 }
            ]
            }
        } 
    }
    ]
}
}

---

In database are always these columns (None is added/removed)

| Id | Message | MessageTemplate | Level | TimeStamp | Exception | Properties |

[kalexx]

I am running in to the same problem.
I was wondering if there is a solution for this problem

[OculiViridi]

I've the same problem too. I'm also using the Serilog.Settings.Configuration 3.0.0-dev-00119, as suggested here, but the problem still remains.

It writes on Console and also creates the default table in SQL Server with the given name, but the columnOptionsSection configuration seems to be totally ignored. Options structure is taken from the GitHub repository of the project.
Instead, by setting the same configuration using the Fluent way, it works as expected.

UPDATE:

By adding the Serilog.Debugging.SelfLog.Enable(Console.Error); line to my application startup code, the following error appears on console:

2018-08-07T14:38:44.2797285Z Unable to write 4 log events to the database due to following error: The given ColumnMapping does not match up with any column in the source or destination.

This is my actual appsettings.json file:

{
  "ConnectionStrings": {
    "DbContext": "Server=..."
  },
  "Serilog": {
    "Enrich": "FromLogContext",
    "MinimumLevel": {
      "Default": "Debug",
      "Override": {
        "Microsoft": "Warning",
        "System": "Warning"
      }
    },
    "Using": [ "Serilog.Sinks.MSSqlServer" ],
    "WriteTo": [
      "Console",
      {
        "Name": "MSSqlServer",
        "Args": {
          "connectionString": "Server=...",
          "tableName": "Logs",
          "columnOptionsSection": {
            "addStandardColumns": [ "LogEvent" ],
            "removeStandardColumns": [ "Properties" ],
            "level": {
              "columnName": "Level",
              "storeAsEnum": true
            }
          },
          "autoCreateSqlTable": true
        }
      }
    ]
  }
}

UPDATE 2018/08/22:

I'm on .NET Core v2.1.

[MV10]

@Luxuride the version you listed (5.1.2) doesn't have Microsoft.Extensions.Configuration support. That support was added last month (PR#123) and is currently only available in dev-release packages (at the moment, 5.1.3-dev00202).

@OculiViridi are you on the SQL release package or the dev package?

[OculiViridi]

@MV10 This is all my packages versions:

• Serilog.AspNetCore v2.1.1
• Serilog.Sinks.Console v3.1.1
• Serilog.Sinks.MSSqlServer v5.1.2
• Serilog.Settings.Configuration v3.0.0-dev-00119

I didn't know that even dev version of Serilog.Sinks.MSSqlServer was necessary. Anyway, what do you mean when you say

the version you listed (5.1.2) doesn't have Microsoft.Extensions.Configuration support

Because I have no problem with the code below:

Log.Logger = new LoggerConfiguration()
    .ReadFrom.Configuration(configuration)
    .CreateLogger();

UPDATE 2018/08/22:

I'm on .NET Core v2.1.

[kalexx]

@MV10 Thanks for the tip. I installed the version you mentioned but using dev-release packages (at the moment, 5.1.3-dev00202) throws the following exception:

Argument value should be of type Action<>.

at Serilog.Settings.Configuration.ConfigurationSectionArgumentValue.ConvertTo(Type toType, IReadOnlyDictionary 2 declaredLevelSwitches) at Serilog.Settings.Configuration.ConfigurationReader.<>c__DisplayClass17_3.<CallConfigurationMethods>b__4(<>f__AnonymousType6 2 <>h__TransparentIdentifier0) at System.Linq.Utilities.<>c__DisplayClass2_0 3.<CombineSelectors>b__0(TSource x) at System.Linq.Enumerable.SelectListPartitionIterator 2.ToList() at System.Linq.Enumerable.ToList[TSource](IEnumerable 1 source) at Serilog.Settings.Configuration.ConfigurationReader.CallConfigurationMethods(ILookup 2 methods, IList 1 configurationMethods, Object receiver, IReadOnlyDictionary 2 declaredLevelSwitches) at Serilog.Settings.Configuration.ConfigurationReader.Configure(LoggerConfiguration loggerConfiguration)
do you know if there is anything new in this Package?

[MV10]

@OculiViridi Versions earlier than 5.1.3 were based upon the .NET Framework's XML-based ConfigurationManager (you can see this in the older readme which preceded my PR#123 in May that added .NET Standard multi-targeting and MS.Ext.Config v2 support).

@kalexx Since that's a new problem I'd suggest a new thread with some of your details -- which .NET you're targeting, your various packages, and your config.

[Luxuride]

@MV10 that error what happened to @kalexx is happening in the the program which is in this issue. All details are on this site.

Update:

Only difference is Serilog.Sinks.MSSqlServer(5.1.3-dev-00202)

[globtech1]

Same configuration, the same problem. When switching to the dev package, get the same error: Argument value should be of type Action<>. Thanks

[paulpce]

Another user having the same issue with Serilog.Sinks.MSSqlServer(5.1.3-dev-00202) using .NET Core v2.1. Configuring in Startup using .WriteTo.MSSqlServer(...) works fine, but when using .ReadFrom.Configuration(Configuration) and the exact snippet from the README page, I get the same error Argument value should be of type Action<>:

{
  "Serilog": {
    "Using":  ["Serilog.Sinks.MSSqlServer"],
    "MinimumLevel": "Debug",
    "WriteTo": [
      { "Name": "MSSqlServer", 
        "Args": { 
            "connectionString": "Data Source=...",
            "tableName": "Logs",
            "columnOptionsSection": {
              "customColumns": [
                { "ColumnName": "EventType", "DataType": "int", "AllowNull": false },
                { "ColumnName": "Release", "DataType": "varchar", "DataLength": 32 }
              ]
            }
        } 
      }
    ]
  }
}

And I might add that the issue that led me here was the original one from the OP: I couldn't get custom columns written to using v5.1.2. The only way I've found that works is using v5.1.3-dev-00202 and .WriteTo.MSSqlServer(...)

[MV10]

Everyone, please update with the .NET you're targeting -- Framework or Core?

[kalexx]

@MV10 I am using .NET Core 2.1

[OculiViridi]

@MV10 Updated my posts. I'm on .NET Core v2.1.

[paulpce]

@MV10 Also .NET Core v2.1

[MV10]

Thanks everyone. I'll try to reproduce this today.

[MV10]

Quick update, a basic .NET Core 2.1 console app is able to use the README configuration to create the database and write log entries correctly. Next I'll set up a simple ASP.NET Core test.

These are the packages I'm using, I think they're equivalent to what everyone has reported above. (The last two are local builds from source pulled from the current dev branches today).

• Microsoft.Extensions.Configuration v2.1.1
• Microsoft.Extensions.Configuration.FileExtensions v2.1.1
• Microsoft.Extensions.Configuration.Json v2.1.1
• Serilog v2.6.0
• Serilog.Sinks.Console v3.1.1
• Serilog.Settings.Configuration v3.0.0-m0001
• Serilog.Sinks.MSSqlServer v5.1.3-m0001

Configuration -- exactly as shown in the README (copied from the post by @paulpce above, actually) except I added Console output, and to demonstrate these other features work, autoCreateSqlTable and removeStandardColumns:

{
  "Serilog": {
    "Using":  ["Serilog.Sinks.MSSqlServer"],
    "MinimumLevel": "Debug",
    "WriteTo": [
      "Console",
      { "Name": "MSSqlServer", 
        "Args": { 
          "connectionString": "Data Source=(LocalDB)\\MSSQLLocalDB;Initial Catalog=SerilogTest;Integrated Security=True;",
            "tableName": "Logs",
          "autoCreateSqlTable": true,
            "columnOptionsSection": {
            "removeStandardColumns": [ "Properties" ],
              "customColumns": [
                { "ColumnName": "EventType", "DataType": "int", "AllowNull": false },
                { "ColumnName": "Release", "DataType": "varchar", "DataLength": 32 }
              ]
            }
        } 
      }
    ]
  }
}  

Console program:

var appConfig = new ConfigurationBuilder()
    .SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile(path: "appsettings.json", optional: false, reloadOnChange: true)
    .Build();

var logConfig = new LoggerConfiguration();
logConfig.ReadFrom.Configuration(appConfig);
var logger = logConfig.CreateLogger();

logger.Information("Release is nullable {EventType} {Release}", 123, "1.0.0");
logger.Information("An Information message {EventType}", 100);
logger.Warning("A Warning message {EventType}", 200);
logger.Error("An Error message {EventType}", 300);
logger.Verbose("A Verbose message {EventType}", 400);
logger.Debug("A Debug message {EventType}", 500);

logger.Dispose();

The database it generated (the custom fields are added and the Properties field is removed):

Rows written to the database: