[MEAI] Update Aspire to recognize telemetry sources from both Experimental.Microsoft.Extensions.AI and Microsoft.Extensions.AI

[JamesNK]

I think experimental can be removed from the default activity source and meter names: Experimental.Microsoft.Extensions.AI.

extensions/src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs

Line 12 in 77859cb

public const string DefaultSourceName = "Experimental.Microsoft.Extensions.AI";

I don't think the semantic conventions around AI will be stable anytime soon, and leaving experimental in the name forever just turns it into a defacto name that might not be able to be changed in the future because it's baked into libraries and apps that uses it. The longer it stays this name, the more pain there will be in changing it to Microsoft.Extensions.AI in the future. Instead, docs can mention that telemetry isn't stable yet and could change.

For example, imagine Aspire has MEAI integration that automatically enables Experimental.Microsoft.Extensions.AI sources. Telemetry collection will break if MEAI changes the default name in the future and people update MEAI without updating the Aspire integration.

[stephentoub]

@lmolkova, you were a strong advocate for the "Experimental." prefix. What's your current opinion? I see OTEL_SEMCONV_STABILITY_OPT_IN was introduced in the most recent revision; is the intent with that we should now remove "Experimental."?

[lmolkova]

If it's called Microsoft.Extensions.AI then there is no indication that telemetry is experimental and is subject to change. Azure SDKs then solve it with additional opt-in flag to enable experimental features which makes onboarding harder.

With Experimental.* the idea was that we can:

• Explicitly provide stability status and manage expectations
• Recommend to do AddSource("*Microsoft.Extensions.AI") or enable both of them. I think Aspire could do this.
• Once conventions become stable, it should be possible to keep reporting on Experimental source if stable one is disabled (and report only on the stable if it's enabled). It should be possible to report additional experimental features when experimental source is enabled.

The OTEL_SEMCONV_STABILITY_OPT_IN doesn't change much. It's still important to convey that source/meter are experimental and there will be breaking changes. As an external non-OTel hosted instrumentation, MEAI could decide to ignore it and update conventions to the latest version without offering old version support, or support it via extra config.

Having said this, if you'd make a decision to update source name, aligning it with SemConv 1.37.0 support would be the best time - it contains a lot of breaking changes.

[JamesNK]

I like the idea to listen to both Microsoft.Extensions.AI and Experimental.Microsoft.Extensions.AI. Aspire could do that in its integration.