Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Populate feature flag telemetry metadata when feature flag telemetry is enabled #517

Merged
merged 19 commits into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -916,7 +916,7 @@ private void SetData(IDictionary<string, string> data)
continue;
}

IEnumerable<KeyValuePair<string, string>> kvs = await adapter.ProcessKeyValue(setting, _logger, cancellationToken).ConfigureAwait(false);
IEnumerable<KeyValuePair<string, string>> kvs = await adapter.ProcessKeyValue(setting, AppConfigurationEndpoint, _logger, cancellationToken).ConfigureAwait(false);

if (kvs != null)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ public AzureKeyVaultKeyValueAdapter(AzureKeyVaultSecretProvider secretProvider)
/// <summary> Uses the Azure Key Vault secret provider to resolve Key Vault references retrieved from Azure App Configuration. </summary>
/// <param KeyValue ="IKeyValue"> inputs the IKeyValue </param>
/// returns the keyname and actual value
public async Task<IEnumerable<KeyValuePair<string, string>>> ProcessKeyValue(ConfigurationSetting setting, Logger logger, CancellationToken cancellationToken)
public async Task<IEnumerable<KeyValuePair<string, string>>> ProcessKeyValue(ConfigurationSetting setting, Uri endpoint, Logger logger, CancellationToken cancellationToken)
{
KeyVaultSecretReference secretRef;

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
//
using System.Text;
using System;

namespace Microsoft.Extensions.Configuration.AzureAppConfiguration.Extensions
{
internal static class BytesExtensions
{
/// <summary>
/// Converts a byte array to Base64URL string with optional padding ('=') characters removed.
/// Base64 description: https://datatracker.ietf.org/doc/html/rfc4648.html#section-4
/// </summary>
public static string ToBase64Url(this byte[] bytes)
Copy link
Member

@jimmyca15 jimmyca15 Feb 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you put a summary that mentions it converts byte array to b64 URL. Using the trim trailing = characters strategy and also link to b64 spec.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{
string bytesBase64 = Convert.ToBase64String(bytes);

int indexOfEquals = bytesBase64.IndexOf("=");

// Remove all instances of "=" at the end of the string that were added as padding
int stringBuilderCapacity = indexOfEquals != -1 ? indexOfEquals : bytesBase64.Length;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code trims everything after the first occurrence of the =. It doesn't match your comment. You assume the = is at the end of the string. I'm wondering why we don't simply use TrimEnd.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Base64 doesn't use equals as any value other than padding at the end, so it should work effectively the same. I had it as TrimEnd initially but I got lost with this other implementation that was suggested and I didn't reconsider it. I think it makes more sense to be TrimEnd to save iteration and be more clear. Jumped the gun on the merge, so I can make another PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why TrimEnd would be used, it's unnecessary allocation. Otherwise it would just be Convert.ToBase64String(bytes).Replace('+', '-').Replace('/', '_').TrimEnd('='). But that creates copies a bit carelessly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation is fine. The comment can be updated to clarify that we choose to skip the padding (if any) based on the spec.


StringBuilder stringBuilder = new StringBuilder(stringBuilderCapacity);

// Construct Base64URL string by replacing characters in Base64 conversion that are not URL safe
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason the featureFlagId needs to be URL safe?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For use in REST resource IDs (/experiment/{id})

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, sounds like we should have a comment in CalculateFeatureFlagId to explain why we need base64url encoding.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems more like a design doc/specification territory rather than a code comment. The code just implements the spec of a feature flag id, which uses b64url encoding.

for (int i = 0; i < stringBuilderCapacity; i++)
{
if (bytesBase64[i] == '+')
{
stringBuilder.Append('-');
}
else if (bytesBase64[i] == '/')
{
stringBuilder.Append('_');
}
else
{
stringBuilder.Append(bytesBase64[i]);
}
}

return stringBuilder.ToString();
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,8 @@ internal class FeatureManagementConstants
public const string From = "From";
public const string To = "To";
public const string Seed = "Seed";
public const string ETag = "ETag";
public const string FeatureFlagId = "FeatureFlagId";
public const string FeatureFlagReference = "FeatureFlagReference";
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,12 @@
// Licensed under the MIT license.
//
using Azure.Data.AppConfiguration;
using Microsoft.Extensions.Configuration.AzureAppConfiguration.Extensions;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
Expand All @@ -20,7 +23,7 @@ public FeatureManagementKeyValueAdapter(FeatureFilterTracing featureFilterTracin
_featureFilterTracing = featureFilterTracing ?? throw new ArgumentNullException(nameof(featureFilterTracing));
}

public Task<IEnumerable<KeyValuePair<string, string>>> ProcessKeyValue(ConfigurationSetting setting, Logger logger, CancellationToken cancellationToken)
public Task<IEnumerable<KeyValuePair<string, string>>> ProcessKeyValue(ConfigurationSetting setting, Uri endpoint, Logger logger, CancellationToken cancellationToken)
{
FeatureFlag featureFlag;
try
Expand Down Expand Up @@ -197,15 +200,28 @@ public FeatureManagementKeyValueAdapter(FeatureFilterTracing featureFilterTracin

if (telemetry.Enabled)
{
keyValues.Add(new KeyValuePair<string, string>($"{telemetryPath}:{FeatureManagementConstants.Enabled}", telemetry.Enabled.ToString()));
}
if (telemetry.Metadata != null)
{
foreach (KeyValuePair<string, string> kvp in telemetry.Metadata)
{
keyValues.Add(new KeyValuePair<string, string>($"{telemetryPath}:{FeatureManagementConstants.Metadata}:{kvp.Key}", kvp.Value));
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add some comments to explain how the featureFlagId is supposed to be calculated? It's not clear why we are doing some of the operations we are doing here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would also add clarity if we move the b64url into an extension method.

ToBase64UrlString(this bytes[]);

At that point the code will be more self explanatory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, won't mind if we move the whole featureFlagId calculation to a separate method. This function is getting way too long.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'll go with the extension for now since it makes sense to me conceptually, but let me know if you have any other concerns

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will the extension method be reusable somewhere?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just want to point out that the featureFlagId calculation is "by convention". It's not persisted anywhere. Portal and all our other providers will do the same calculation on-fly. Since the .NET provider is the first one doing this, it will make everyone's life easier if we can encapsulate it in one function with detailed comments so others can review/reference.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amerjusupovic Can we please go with b64url extension method and private static method for CalculateFeatureFlagId(string key, string label)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good to me

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhenlan do you think the extension method is unnecessary? I thought it made sense to be an extension just because it's a known conversion, but it's true that we're only using it here.

Copy link
Member

@jimmyca15 jimmyca15 Feb 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend keeping that extension method. b64url is a spec and encapsulates well as utility method. Similar to frameworks System.Convert.ToBase64String


if (telemetry.Metadata != null)
{
foreach (KeyValuePair<string, string> kvp in telemetry.Metadata)
string featureFlagId = CalculateFeatureFlagId(setting.Key, setting.Label);

keyValues.Add(new KeyValuePair<string, string>($"{telemetryPath}:{FeatureManagementConstants.Metadata}:{FeatureManagementConstants.FeatureFlagId}", featureFlagId));

if (endpoint != null)
{
keyValues.Add(new KeyValuePair<string, string>($"{telemetryPath}:{FeatureManagementConstants.Metadata}:{kvp.Key}", kvp.Value));
string featureFlagReference = $"{endpoint.AbsoluteUri}kv/{setting.Key}{(!string.IsNullOrWhiteSpace(setting.Label) ? $"?label={setting.Label}" : "")}";

keyValues.Add(new KeyValuePair<string, string>($"{telemetryPath}:{FeatureManagementConstants.Metadata}:{FeatureManagementConstants.FeatureFlagReference}", featureFlagReference));
}

keyValues.Add(new KeyValuePair<string, string>($"{telemetryPath}:{FeatureManagementConstants.Metadata}:{FeatureManagementConstants.ETag}", setting.ETag.ToString()));

keyValues.Add(new KeyValuePair<string, string>($"{telemetryPath}:{FeatureManagementConstants.Enabled}", telemetry.Enabled.ToString()));
}
}

Expand All @@ -229,5 +245,19 @@ public bool NeedsRefresh()
{
return false;
}

private static string CalculateFeatureFlagId(string key, string label)
{
byte[] featureFlagIdHash;

// Convert the value consisting of key, newline character, and label to a byte array using UTF8 encoding to hash it using SHA 256
using (HashAlgorithm hashAlgorithm = SHA256.Create())
{
featureFlagIdHash = hashAlgorithm.ComputeHash(Encoding.UTF8.GetBytes($"{key}\n{(string.IsNullOrWhiteSpace(label) ? null : label)}"));
}

// Convert the hashed byte array to Base64Url
return featureFlagIdHash.ToBase64Url();
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
// Licensed under the MIT license.
//
using Azure;
using Microsoft.Extensions.Logging;
using System;
using System.Threading;
using System.Threading.Tasks;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// Licensed under the MIT license.
//
using Azure.Data.AppConfiguration;
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
Expand All @@ -10,7 +11,7 @@ namespace Microsoft.Extensions.Configuration.AzureAppConfiguration
{
internal interface IKeyValueAdapter
{
Task<IEnumerable<KeyValuePair<string, string>>> ProcessKeyValue(ConfigurationSetting setting, Logger logger, CancellationToken cancellationToken);
Task<IEnumerable<KeyValuePair<string, string>>> ProcessKeyValue(ConfigurationSetting setting, Uri endpoint, Logger logger, CancellationToken cancellationToken);

bool CanProcess(ConfigurationSetting setting);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ internal class JsonKeyValueAdapter : IKeyValueAdapter
KeyVaultConstants.ContentType
};

public Task<IEnumerable<KeyValuePair<string, string>>> ProcessKeyValue(ConfigurationSetting setting, Logger logger, CancellationToken cancellationToken)
public Task<IEnumerable<KeyValuePair<string, string>>> ProcessKeyValue(ConfigurationSetting setting, Uri endpoint, Logger logger, CancellationToken cancellationToken)
{
if (setting == null)
{
Expand Down
23 changes: 22 additions & 1 deletion tests/Tests.AzureAppConfiguration/FeatureManagementTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
using Azure.Core.Testing;
using Azure.Data.AppConfiguration;
using Azure.Data.AppConfiguration.Tests;
using Azure.Identity;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Configuration.AzureAppConfiguration;
using Microsoft.Extensions.Configuration.AzureAppConfiguration.FeatureManagement;
Expand All @@ -14,6 +15,9 @@
using System.Collections.Generic;
using System.Diagnostics.Tracing;
using System.Linq;
using System.Net;
using System.Security.Cryptography;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Xunit;
Expand Down Expand Up @@ -288,7 +292,7 @@ public class FeatureManagementTests
}
}
",
label: default,
label: "label",
contentType: FeatureManagementConstants.ContentType + ";charset=utf-8",
eTag: new ETag("c3c231fd-39a0-4cb6-3237-4614474b92c1"));

Expand Down Expand Up @@ -1330,13 +1334,30 @@ public void WithTelemetry()
.AddAzureAppConfiguration(options =>
{
options.ClientManager = TestHelpers.CreateMockedConfigurationClientManager(mockClient.Object);
options.Connect(TestHelpers.PrimaryConfigStoreEndpoint, new DefaultAzureCredential());
options.UseFeatureFlags();
})
.Build();

Assert.Equal("True", config["FeatureManagement:TelemetryFeature:Telemetry:Enabled"]);
Assert.Equal("Tag1Value", config["FeatureManagement:TelemetryFeature:Telemetry:Metadata:Tags.Tag1"]);
Assert.Equal("Tag2Value", config["FeatureManagement:TelemetryFeature:Telemetry:Metadata:Tags.Tag2"]);
Assert.Equal("c3c231fd-39a0-4cb6-3237-4614474b92c1", config["FeatureManagement:TelemetryFeature:Telemetry:Metadata:ETag"]);

byte[] featureFlagIdHash;

using (HashAlgorithm hashAlgorithm = SHA256.Create())
{
featureFlagIdHash = hashAlgorithm.ComputeHash(Encoding.UTF8.GetBytes($"{FeatureManagementConstants.FeatureFlagMarker}TelemetryFeature\nlabel"));
}

string featureFlagId = Convert.ToBase64String(featureFlagIdHash)
.TrimEnd('=')
.Replace('+', '-')
.Replace('/', '_');

Assert.Equal(featureFlagId, config["FeatureManagement:TelemetryFeature:Telemetry:Metadata:FeatureFlagId"]);
Assert.Equal($"{TestHelpers.PrimaryConfigStoreEndpoint}kv/{FeatureManagementConstants.FeatureFlagMarker}TelemetryFeature?label=label", config["FeatureManagement:TelemetryFeature:Telemetry:Metadata:FeatureFlagReference"]);
}


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,7 @@ public void DoesNotThrowKeyVaultExceptionWhenProviderIsOptional()
var mockKeyValueAdapter = new Mock<IKeyValueAdapter>(MockBehavior.Strict);
mockKeyValueAdapter.Setup(adapter => adapter.CanProcess(_kv))
.Returns(true);
mockKeyValueAdapter.Setup(adapter => adapter.ProcessKeyValue(_kv, It.IsAny<Logger>(), It.IsAny<CancellationToken>()))
mockKeyValueAdapter.Setup(adapter => adapter.ProcessKeyValue(_kv, It.IsAny<Uri>(), It.IsAny<Logger>(), It.IsAny<CancellationToken>()))
.Throws(new KeyVaultReferenceException("Key vault error", null));
mockKeyValueAdapter.Setup(adapter => adapter.InvalidateCache(null));

Expand Down