Add an experimental configuration which disables the automatic export…

[JeremyBessonElastic]

What:

Add an experimental configuration which disables the automatic export of scope log kv attributes for OTLP

Why:

Now, the scoped log kv attributes are either not exported at all with IncludeScopes=false or are all exported for OTLP.
There is no way to export the 'scoped log kv attributes' in a controlled manner while it is possible to do it with LogRecord attributes.
'scoped log kv attributes' are fully read-only while the LogRecord attributes can be modified.
Especially, the OTL exporter is failing with the lack of log kv attributes de-duplication.

How to solve it for now:

The client can decide to add the scope log attributes into the LogRecord using the IncludeScopes=true, config 'OTEL_DOTNET_EXPERIMENTAL_OTLP_DISABLE_ADDING_SCOPE_LOG'=1 and uses a custom BaseProcessor to deal with scope log kv attributes of the LogRecord: add only some of them to the LogRecord attributes, perform de-duplication, sanitization,...

public class ExampleScopeLogKvAttributeProcessor : BaseProcessor<LogRecord>
{
	public override void OnEnd(LogRecord logRecord)
	{
		var attributes = new Dictionary<string, object?>(logRecord.Attributes?.ToList() ?? new List<KeyValuePair<string, object?>>());

		logRecord.ForEachScope<object?>((scope, _) =>
		{
			if (scope.Scope is not IEnumerable<KeyValuePair<string, object>> scopeKvs) return;

			foreach (var scopeKv in scopeKvs)
			{
				// Sanitization
				if (scopeKv.Key.Equals("UserId", StringComparison.OrdinalIgnoreCase))
				{
					attributes[scopeKv.Key] = "Obfuscated ";
					return;
				}

				// Sanitization
				if (scopeKv.Key.Contains("password", StringComparison.OrdinalIgnoreCase))
				{
					return;
				}

				// Business logic filtering
				if (scopeKv.Key.StartsWith("Internal_"))
				{
					return;
				}

				// Add scope log KV attributes in attributes with some sort of de-duplication
				attributes[scopeKv.Key] = scopeKv.Value;
			}
		}, null);

		logRecord.Attributes = attributes.ToList();
		base.OnEnd(logRecord);
	}
}

Note:

An experimental configuration is used for now until a more solid solution is found/available.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: JeremyBessonElastic (9d86ee6)

[rajkumar-rangaraj]

Disabling scope is already supported via the IncludeScopes property in OpenTelemetryLoggerOptions. There is no need to introduce a new experimental environment variable at the OTLP exporter level for this.

From my understanding, setting IncludeScopes = false should prevent the logger from populating LogRecord.Scope in the first place—meaning there shouldn't be a need to filter it at the exporter layer. Can we validate whether IncludeScopes = false is truly being honored during pipeline construction, and whether LogRecord.Scope is still being set despite it?

If it turns out that IncludeScopes isn’t behaving as expected, that’s something we should fix upstream in logger configuration, not through an OTLP-specific flag.

[stevejgordon]

Hi, @rajkumar-rangaraj. This stems from #4324 and provides a sort of workaround. The issue is that IncludeScopes applies everywhere and stops any data from scopes being added to LogRecord.Scope. Because that data is read-only, there's no way to exclude some scope data, perform redaction or deduping in a processor. There are no hooks to process scopes, unlike attributes added onto the log record.

This would workaround that by allowing IncludeScopes to be true, thereby capturing the data, but allowing consumers to opt out of the direct serialization of scopes. Instead, they can then use a process to extract the scope data they need and add it to attributes, prior to export.

Overall, we feel a more robust solution is needed to refactor scope data so that it's added to the LogRecord attributes (and deduplicated per the spec) before processors run. We felt this experimental option would be a quicker short-term workaround while we focus on a proper fix.