Release new version of ECS.NET 8.6.0

[Mpdreamz]

• sanity check the process field set (@gregkalapos)
• Extend the code generator to include richer meta information in xml documentation. (@Mpdreamz )
• Merge Further Elastic.Ingest Improvements #110 as it has a lot of integration tests using the new types. (@gregkalapos)
• We will aim for an 8.4.0-alpha first to validate our build tooling. (@Mpdreamz)
• Include direct support for ECS fields in message templates (@Mpdreamz) Set ECS fields from message templates #229
• Utilize labels for value types and metadata for objects. (@Mpdreamz) Set ECS fields from message templates #229
• Do not assume we can safely serialize to metadata [BUG] MetaDataDictionaryConverter - NullReferenceException #219 (@Mpdreamz)
• Bump to 8.5/8.6 if that bump is not complicated, prioritize releasing (@Mpdreamz)
• Breaking changes documentation. Include an paragraph describing what's expected to break with this new major version and why. (@Mpdreamz)

[bonyaroslav]

@Mpdreamz @gregkalapos Any news when we can use it? We're waiting for many months already

[odin568]

Same for me. Eager to use updated version to get rid of some workarounds

[lewinskimaciej]

Is this package still being maintained? We desperately need #167 but it seems that there hasn't been a release in 1,5 years. Is there something we can do to help push this out?

[Mpdreamz]

Still maintained folks! Thanks for the nudges for an update.

I have been on leave for an extended period of time but am back working on getting an update out the door. Any feedback on the upcoming pre-release would be most welcomed!

Will comment on this issue when it goes up on nuget.

[Mpdreamz]

Heads up here that we just shipped version 8.4.0-alpha1 to nuget.

I intend to wait two weeks to collect feedback on this new version before shipping it as a GA release (8.4.0).

A LOT of changes have gone in since we last release 1.6.0-alpha1, you can see the full list of changes here

Here are some highlights:

New integrations

• Elasticsearch.Extensions.Logging an integration with Microsoft.Extensions.Logging that writes logs directly to Elasticsearch or Elastic Cloud (thank you @sgryphon for all the help)
• Elastic.CommonSchema.Serilog.Sink a Serilog sink that writes ecs logs directly to Elasticsearch or Elastic Cloud
• Elastic.CommonSchema.Log4Net Log4Net integration that logs to files in ecs-logging format. (thank you @andreycha!)
• Elastic.Ingest.Elasticsearch.CommonSchema a specialization of Elastic.Ingest.Elasticsearch to push data to datastreams indices that is fully ECS aware and can bootstrap the target datastreams/indices confirming ECS mappings.

Big Updates:

• Completely rewrote the code generator from scratch to accomodate the ECS spec format changes since 1.6 to 8.x.
• Spun off the ingest components into their own seperate libraries: elastic/elastic-ingest-dotnet
• Elastic.CommonSchema was completely rearchitected as we rewritten the code generator from scratch to align with the ECS specification changes.
  • EcsDocument now has an AssignField(path, value) method that can dynamically assign ECS fields based on path. This allows all the structured logging integrations to directly assign ECS fields too.
  • Through CreateNewWithDefaults all logging integrations share a common base default for how ecs documents get created and enriched.
• NLog integration has seen continued improvements from @snakefoot (Can't thank you enough for all your efforts!)
• Moved CI back onto Github Actions (thank you @v1v).

Breaking Changes.

• The root object for ECS documents in Elastic.CommonSchema was renamed from Base to EcsDocument.

[HubDevUser]

Can you please include .netstandard2.0 in the official 8.4.0 release. We need to consume this in fullframework, the alpha1 release only contains .netstandard2.1 assemblies @Mpdreamz

[reydelleon-skuvault]

@Mpdreamz Are there plans to update documentation to show how to put all these together? For example, how to enhance the EcsDocument with custom fields.

I noticed that you're putting forward an alternative to Serilog.Sinks.Elasticsearch. it would be helpful to see some examples of how you would configure your sink to achieve a similar experience (knowing that they don't have the same features) than that of the contrib sink. Many of us have been using that other sink for a long time and would like to make any possible migration as painless as possible.

[Mpdreamz]

@reydelleon-skuvault absolutely, docs are a big prerequisite for the 8.4.0 release.

I tried to capture some of the main differences between the sinks here for now: https://github.com/elastic/ecs-dotnet/tree/main/src/Elastic.CommonSchema.Serilog.Sink#comparison-with-serilogsinkselasticsearch

Any feedback or additional questions you may have are more than welcomed!