[Question]:

[nadahaninou]

Component

Distro

Question

Hello,

I'm using Microsoft OpenTelemetry Distro to add the governance/observability layer to my agent, which is built with Microsoft Agent Framework.

When I run my agent, I can see that all spans are successfully sent to Agent365 Observability with an HTTP 200 response. Here is the log:

DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:Found 1 identity groups with 4 total spans to export
DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:Exporting 4 spans to endpoint: https://agent365.svc.cloud.microsoft/observabilityService/tenants/<TENANT_ID>/otlp/agents/<AGENT_ID>/traces?api-version=1
DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:Sending chunk 1 of 1 (4 spans, 4334 bytes)
DEBUG:urllib3.connectionpool:https://agent365.svc.cloud.microsoft:443 "POST /observabilityService/tenants/<TENANT_ID>/otlp/agents/<AGENT_ID>/traces?api-version=1 HTTP/1.1" 200 564
DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:HTTP 200 success on attempt 1. Correlation ID: <CORRELATION_ID>. Response: {"results":[{"spanId":"<SPAN_ID>","sinks":{"flashpoint":{"status":"sent"},"sentinel":{"status":"sent"},"esp":{"status":"sent"}}},{"spanId":"<SPAN_ID_2>","sinks":{"flashpoint":{"status":"se...

However, no activity shows up on the Agent 365 side when I access my agent, specifically in the Activity tab.

Please note that my agent is not a conversational agent, it is not published to a channel (like MS Teams) — it's an agent invoked directly through code calls.

Thank you in advance for your help.

Context

No response

[github-actions]

cc @hectorhdzg @JacksonWeber @lzchen @MSNev @rads-1996 @jeremydvoss

[rads-1996]

@nadahaninou Could you share the code for the application you are using? Also if you set enable_console=True in use_microsoft_opentelemetry you might be able to see the logs which can help identify why the spans don't end up in the activity tab. It would be quite helpful if you could share those logs as well.

[nadahaninou]

Hello @rads-1996 , Thanks for the quick reply!

This is my code
import asyncio
import logging
import os
import time

from dotenv import load_dotenv
from msal import ConfidentialClientApplication

from microsoft.opentelemetry import use_microsoft_opentelemetry
from microsoft.opentelemetry.a365.core import (
AgentDetails,
BaggageBuilder,
Channel,
InferenceCallDetails,
InferenceOperationType,
InferenceScope,
InvokeAgentScope,
InvokeAgentScopeDetails,
Request,
ServiceEndpoint,
)

from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient
from azure.identity import AzureCliCredential

logging.basicConfig(level=logging.DEBUG)

load_dotenv()

------------------------------------------------------------------

Entra ID (S2S) - two-step flow for Agentic applications

------------------------------------------------------------------

TENANT_ID = os.getenv("TENANT_ID")
BLUEPRINT_CLIENT_ID = os.getenv("CLIENT_ID")
BLUEPRINT_CLIENT_SECRET = os.getenv("CLIENT_SECRET")

AGENT_ID = "<AGENT_ID>" # Agent identity (agenticAppId)
AGENT_BLUEPRINT_ID = "<AGENT_BLUEPRINT_ID>" # Blueprint app id

OBSERVABILITY_SCOPE = ["api://<OBSERVABILITY_RESOURCE_ID>/.default"]
AZURE_AD_TOKEN_EXCHANGE_SCOPE = ["api://AzureAdTokenExchange/.default"]

_token_cache = {}

def get_access_token(agent_id: str) -> str:
if agent_id != AGENT_ID:
agent_id = AGENT_ID

cached = _token_cache.get(agent_id)
if cached and time.time() < (cached["expiration"] - 60):
    return cached["token"]

authority = f"https://login.microsoftonline.com/{TENANT_ID}"

blueprint_app = ConfidentialClientApplication(
    client_id=BLUEPRINT_CLIENT_ID,
    authority=authority,
    client_credential=BLUEPRINT_CLIENT_SECRET,
)
agent_token_result = blueprint_app.acquire_token_for_client(
    scopes=AZURE_AD_TOKEN_EXCHANGE_SCOPE,
    data={"fmi_path": agent_id},
)
if "access_token" not in agent_token_result:
    raise RuntimeError("Failed to acquire agent token: " + str(agent_token_result))
agent_token = agent_token_result["access_token"]

instance_app = ConfidentialClientApplication(
    client_id=agent_id,
    authority=authority,
    client_credential={"client_assertion": agent_token},
)
result = instance_app.acquire_token_for_client(scopes=OBSERVABILITY_SCOPE)
if "access_token" not in result:
    raise RuntimeError("Failed to acquire instance token: " + str(result))

_token_cache[agent_id] = {
    "token": result["access_token"],
    "expiration": time.time() + int(result["expires_in"]),
}
logging.info("New Agent365 S2S token acquired for agent_id=%s", agent_id)
return result["access_token"]

------------------------------------------------------------------

OpenTelemetry

------------------------------------------------------------------

from agents import set_tracing_disabled
set_tracing_disabled(True)
use_microsoft_opentelemetry(
enable_a365=True,
enable_console=True,
a365_use_s2s_endpoint=True,
a365_token_resolver=lambda agent_id, tenant_id: get_access_token(agent_id),
)

Fallback fix: some auto-instrumented residual spans don't carry the

correct agent_id, so we force-align them to the registered agent id

before export.

import microsoft.opentelemetry.a365.core.exporters.agent365_exporter as _a365_exporter_module
from microsoft.opentelemetry.a365.core.exporters.enriched_span import EnrichedReadableSpan
from microsoft.opentelemetry.a365.core.exporters.utils import (
filter_and_partition_by_identity as _original_filter_and_partition,
)
from microsoft.opentelemetry.a365.constants import GEN_AI_AGENT_ID_KEY, TENANT_ID_KEY

def _patched_filter_and_partition_by_identity(spans):
groups = _original_filter_and_partition(spans)
merged: dict[tuple[str, str], list] = {}
for (tenant_id, agent_id), spans_list in groups.items():
target_key = (tenant_id, AGENT_ID)
if agent_id != AGENT_ID:
spans_list = [
EnrichedReadableSpan(sp, {GEN_AI_AGENT_ID_KEY: AGENT_ID, TENANT_ID_KEY: tenant_id})
for sp in spans_list
]
merged.setdefault(target_key, []).extend(spans_list)
return merged

_a365_exporter_module.filter_and_partition_by_identity = _patched_filter_and_partition_by_identity

------------------------------------------------------------------

Main

------------------------------------------------------------------

async def main():

with (
    BaggageBuilder()
    .tenant_id(TENANT_ID)
    .agent_id(AGENT_ID)
    .conversation_id("conversation-001")
    .build()
):

    client = FoundryChatClient(
        project_endpoint=os.getenv("FOUNDRY_PROJECT_ENDPOINT"),
        model=os.getenv("FOUNDRY_MODEL"),
        credential=AzureCliCredential(),
    )

    agent = Agent(
        client=client,
        name="HelloAgent",
        instructions="You are a friendly assistant. Keep your answers brief.",
    )

    user_message = "What is the capital of France?"

    # --- Manual instrumentation, per official docs, for store validation ---
    agent_details = AgentDetails(
        agent_id=AGENT_ID,
        agent_name="HelloAgent",
        agent_description="A friendly assistant agent",
        agent_blueprint_id=AGENT_BLUEPRINT_ID,
        tenant_id=TENANT_ID,
    )

    request = Request(
        content=user_message,
        session_id="session-001",
        conversation_id="conversation-001",
        channel=Channel(name="console"),
    )

    scope_details = InvokeAgentScopeDetails(
        endpoint=ServiceEndpoint(hostname="localhost", port=443),
    )

    with InvokeAgentScope.start(
        request=request,
        scope_details=scope_details,
        agent_details=agent_details,
    ) as invoke_scope:
        invoke_scope.record_input_messages([user_message])

        inference_details = InferenceCallDetails(
            operationName=InferenceOperationType.CHAT,
            model=os.getenv("FOUNDRY_MODEL"),
            providerName="azure-foundry",
        )

        with InferenceScope.start(
            request=request,
            details=inference_details,
            agent_details=agent_details,
        ) as inference_scope:
            inference_scope.record_input_messages([user_message])
            result = await agent.run(user_message)
            inference_scope.record_output_messages([str(result)])

        invoke_scope.record_output_messages([str(result)])

print(result)

if name == "main":
asyncio.run(main())

and this is the log output after running it with the parameter enable_console=True in use_microsoft_opentelemetry

INFO:microsoft.opentelemetry.a365.core.opentelemetry_scope:Span started: 'invoke_agent HelloAgent' (1207e2741fe70df6)
INFO:microsoft.opentelemetry.a365.core.opentelemetry_scope:Span started: 'Chat <MODEL_NAME>' (e46237658d5b346b)

{
"name": "chat <MODEL_NAME>",
"context": {
"trace_id": "0x<TRACE_ID>",
"span_id": "0x<SPAN_ID>",
"trace_state": "[]"
},
"kind": "SpanKind.INTERNAL",
"parent_id": "0x<PARENT_SPAN_ID>",
"start_time": "2026-07-23T15:38:54.656855Z",
"end_time": "2026-07-23T15:39:03.556906Z",
"status": { "status_code": "UNSET" },
"attributes": {
"microsoft.tenant.id": "<TENANT_ID>",
"gen_ai.conversation.id": "conversation-001",
"gen_ai.agent.id": "<REGISTERED_AGENT_ID>",
"gen_ai.request.choice.count": 1,
"gen_ai.operation.name": "chat",
"gen_ai.provider.name": "azure.ai.foundry",
"server.address": "Unknown",
"gen_ai.request.model": "<MODEL_NAME>",
"gen_ai.response.id": "resp_<RESPONSE_ID>",
"gen_ai.response.model": "<MODEL_VERSION>",
"gen_ai.usage.input_tokens": 29,
"gen_ai.usage.output_tokens": 8,
"gen_ai.usage.cache_read.input_tokens": 0,
"gen_ai.usage.reasoning.output_tokens": 0
},
"events": [],
"links": [],
"resource": {
"attributes": {
"telemetry.sdk.language": "python",
"telemetry.sdk.name": "opentelemetry",
"telemetry.sdk.version": "1.43.0",
"service.instance.id": "<SERVICE_INSTANCE_ID>",
"service.name": "unknown_service"
},
"schema_url": ""
}
}
{
"name": "invoke_agent HelloAgent",
"context": {
"trace_id": "0x<TRACE_ID>",
"span_id": "0x<SPAN_ID>",
"trace_state": "[]"
},
"kind": "SpanKind.INTERNAL",
"parent_id": "0x<CHAT_SPAN_ID>",
"start_time": "2026-07-23T15:38:54.656424Z",
"end_time": "2026-07-23T15:39:03.558967Z",
"status": { "status_code": "UNSET" },
"attributes": {
"microsoft.tenant.id": "<TENANT_ID>",
"gen_ai.conversation.id": "conversation-001",
"gen_ai.request.choice.count": 1,
"gen_ai.operation.name": "invoke_agent",
"gen_ai.provider.name": "microsoft.agent_framework",
"gen_ai.agent.id": "<AUTO_GENERATED_AGENT_ID>",
"gen_ai.agent.name": "HelloAgent",
"gen_ai.request.model": "<MODEL_NAME>",
"gen_ai.usage.input_tokens": 29,
"gen_ai.usage.output_tokens": 8,
"gen_ai.usage.cache_read.input_tokens": 0,
"gen_ai.usage.reasoning.output_tokens": 0
},
"events": [],
"links": [],
"resource": {
"attributes": {
"telemetry.sdk.language": "python",
"telemetry.sdk.name": "opentelemetry",
"telemetry.sdk.version": "1.43.0",
"service.instance.id": "<SERVICE_INSTANCE_ID>",
"service.name": "unknown_service"
},
"schema_url": ""
}
}

INFO:microsoft.opentelemetry.a365.core.opentelemetry_scope:Span ended: 'Chat <MODEL_NAME>' (e46237658d5b346b)
{
"name": "Chat <MODEL_NAME>",
"context": {
"trace_id": "0x<TRACE_ID>",
"span_id": "0x<CHAT_SPAN_ID>",
"trace_state": "[]"
},
"kind": "SpanKind.CLIENT",
"parent_id": "0x<INVOKE_AGENT_SPAN_ID>",
"start_time": "2026-07-23T15:38:54.654515Z",
"end_time": "2026-07-23T15:39:03.559657Z",
"status": { "status_code": "UNSET" },
"attributes": {
"telemetry.sdk.name": "microsoft-opentelemetry",
"telemetry.sdk.language": "python",
"telemetry.sdk.version": "1.3.5",
"gen_ai.agent.id": "<REGISTERED_AGENT_ID>",
"gen_ai.agent.name": "HelloAgent",
"gen_ai.agent.description": "A friendly assistant agent",
"microsoft.a365.agent.blueprint.id": "<BLUEPRINT_ID>",
"microsoft.tenant.id": "<TENANT_ID>",
"gen_ai.conversation.id": "conversation-001",
"gen_ai.operation.name": "Chat",
"gen_ai.request.model": "<MODEL_NAME>",
"gen_ai.provider.name": "azure-foundry",
"microsoft.channel.name": "console",
"gen_ai.input.messages": "[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]",
"gen_ai.output.messages": "[{"role": "assistant", "parts": [{"content": "The capital of France is Paris.", "type": "text"}], "finish_reason": "stop"}]"
},
"events": [],
"links": [],
"resource": {
"attributes": {
"telemetry.sdk.language": "python",
"telemetry.sdk.name": "opentelemetry",
"telemetry.sdk.version": "1.43.0",
"service.instance.id": "<SERVICE_INSTANCE_ID>",
"service.name": "unknown_service"
},
"schema_url": ""
}
}

INFO:microsoft.opentelemetry.a365.core.opentelemetry_scope:Span ended: 'invoke_agent HelloAgent' (1207e2741fe70df6)
{
"name": "invoke_agent HelloAgent",
"context": {
"trace_id": "0x<TRACE_ID>",
"span_id": "0x<INVOKE_AGENT_SPAN_ID>",
"trace_state": "[]"
},
"kind": "SpanKind.CLIENT",
"parent_id": null,
"start_time": "2026-07-23T15:38:54.652194Z",
"end_time": "2026-07-23T15:39:03.560196Z",
"status": { "status_code": "UNSET" },
"attributes": {
"gen_ai.operation.name": "invoke_agent",
"telemetry.sdk.name": "microsoft-opentelemetry",
"telemetry.sdk.language": "python",
"telemetry.sdk.version": "1.3.5",
"gen_ai.agent.id": "<REGISTERED_AGENT_ID>",
"gen_ai.agent.name": "HelloAgent",
"gen_ai.agent.description": "A friendly assistant agent",
"microsoft.a365.agent.blueprint.id": "<BLUEPRINT_ID>",
"microsoft.tenant.id": "<TENANT_ID>",
"microsoft.session.id": "session-001",
"gen_ai.conversation.id": "conversation-001",
"server.address": "localhost",
"microsoft.channel.name": "console",
"gen_ai.input.messages": "[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]",
"gen_ai.output.messages": "[{"role": "assistant", "parts": [{"content": "The capital of France is Paris.", "type": "text"}], "finish_reason": "stop"}]"
},
"events": [],
"links": [],
"resource": {
"attributes": {
"telemetry.sdk.language": "python",
"telemetry.sdk.name": "opentelemetry",
"telemetry.sdk.version": "1.43.0",
"service.instance.id": "<SERVICE_INSTANCE_ID>",
"service.name": "unknown_service"
},
"schema_url": ""
}
}

DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:Found 1 identity groups with 4 total spans to export
DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:Exporting 4 spans to endpoint: https://agent365.svc.cloud.microsoft/observabilityService/tenants/<TENANT_ID>/otlp/agents/<AGENT_ID>/traces?api-version=1
DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:Sending chunk 1 of 1 (4 spans, 4334 bytes)
DEBUG:urllib3.connectionpool:https://agent365.svc.cloud.microsoft:443 "POST /observabilityService/tenants/<TENANT_ID>/otlp/agents/<AGENT_ID>/traces?api-version=1 HTTP/1.1" 200 564
DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:HTTP 200 success on attempt 1. Correlation ID: <CORRELATION_ID>. Response: {"results":[{"spanId":"<SPAN_ID>","sinks":{"flashpoint":{"status":"sent"},"sentinel":{"status":"sent"},"esp":{"status":"sent"}}},{"spanId":"<SPAN_ID_2>","sinks":{"flashpoint":{"status":"se...

[rads-1996]

@nadahaninou I see the issue, you need to enable the A365 exporter explicitly to actually let the exporter export to the A365 endpoint. From the code I see you are not using the enable_a365_observability_exporter=True in use_microsoft_opentelemetry. Have you by any chance set the environment variable instead - ENABLE_A365_OBSERVABILITY_EXPORTER? You need to set either to actually export the spans to the endpoint. Can you try doing that and see if that works?

[nadahaninou]

Thanks for looking into this! But I already have ENABLE_A365_OBSERVABILITY_EXPORTER=true set in my .env file

[rads-1996]

@nadahaninou Is this the first time trying out this setup? Have you earlier been able to see the traces? Also, you mentioned that you don't see any activity under the Activity tab, are you referring to the Microsoft A365 admin center portal? Have you tried using an agent in teams?

[nadahaninou]

Hello @rads-1996 ,thank you for you answer, below my answers to your questions:
Is this the first time trying out this setup? → Yes
Have you earlier been able to see the traces? → No
You mentioned that you don't see any activity under the Activity tab, are you referring to the Microsoft A365 admin center portal? → Yes
Have you tried using an agent in Teams? → My agents will only be used via code, not via channels (like MS Teams).
My purpose with this work is to export telemetry from pro-code agents to Microsoft Agent 365. My agents will not be used through a channel (like MS Teams) — they are invoked directly via code.

[nikhilNava]

Hi @nadahaninou - for the spans emitted ensure all the required attributes are present.
https://learn.microsoft.com/en-us/microsoft-agent-365/developer/observability?tabs=python#invokeagentscope-attributes
refer to the guide above.
This should help with the activity tab.

[nadahaninou]

Hello @nikhilNava Thanks for the pointer to the attribute reference! I added the missing required attributes (microsoft.agent.user.id, microsoft.agent.user.email via AgentDetails.agentic_user_id/agentic_user_email, and user.id, user.email, client.address via CallerDetails(user_details=UserDetails(...))). All required attributes from the reference table now appear on the invoke_agent span — see attached log.

{
"name": "invoke_agent HelloAgent",
"context": {
"trace_id": "0x<TRACE_ID>",
"span_id": "0x<SPAN_ID>",
"trace_state": "[]"
},
"kind": "SpanKind.CLIENT",
"parent_id": null,
"start_time": "2026-07-24T12:42:34.824370Z",
"end_time": "2026-07-24T12:42:49.545880Z",
"status": {
"status_code": "UNSET"
},
"attributes": {
"gen_ai.operation.name": "invoke_agent",
"telemetry.sdk.name": "microsoft-opentelemetry",
"telemetry.sdk.language": "python",
"telemetry.sdk.version": "1.3.5",
"gen_ai.agent.id": "<AGENT_ID>",
"gen_ai.agent.name": "HelloAgent",
"gen_ai.agent.description": "A friendly assistant agent",
"microsoft.agent.user.id": "<AGENT_ID>",
"microsoft.agent.user.email": "<AGENT_ID>@<TENANT_ID>.onmicrosoft.com",
"microsoft.a365.agent.blueprint.id": "<BLUEPRINT_ID>",
"microsoft.tenant.id": "<TENANT_ID>",
"microsoft.session.id": "session-001",
"gen_ai.conversation.id": "conversation-001",
"server.address": "localhost",
"microsoft.channel.name": "console",
"user.id": "system",
"user.email": "system@localhost",
"client.address": "127.0.0.1",
"gen_ai.input.messages": "[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]",
"gen_ai.output.messages": "[{"role": "assistant", "parts": [{"content": "The capital of France is Paris.", "type": "text"}], "finish_reason": "stop"}]"
},
"events": [],
"links": [],
"resource": {
"attributes": {
"telemetry.sdk.language": "python",
"telemetry.sdk.name": "opentelemetry",
"telemetry.sdk.version": "1.43.0",
"service.instance.id": "<SERVICE_INSTANCE_ID>",
"service.name": "unknown_service"
},
"schema_url": ""
}
}

DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:Found 1 identity groups with 4 total spans to export
DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:Exporting 4 spans to endpoint: https://agent365.svc.cloud.microsoft/observabilityService/tenants/<TENANT_ID>/otlp/agents/<AGENT_ID>/traces?api-version=1
DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:Sending chunk 1 of 1 (4 spans, 4789 bytes)
DEBUG:urllib3.connectionpool:https://agent365.svc.cloud.microsoft:443 "POST /observabilityService/tenants/<TENANT_ID>/otlp/agents/<AGENT_ID>/traces?api-version=1 HTTP/1.1" 200 564
DEBUG:microsoft.opentelemetry.a365.core.exporters.agent365_exporter:HTTP 200 success on attempt 1. Correlation ID: <CORRELATION_ID>. Response: {"results":[{"spanId":"<SPAN_ID>","sinks":{"flashpoint":{"status":"sent"},"sentinel":{"status":"sent"},"esp":{"status":"sent"}}},{"spanId":"<SPAN_ID_2>","sinks":{"flashpoint":{"status":"se...

Export still succeeds with 200 and all spans confirmed "sent" on every sink.But no activity appears in tab activity of Agent 365 Admin center.

Do you have any idea how i can solve this problem? thank you very much for your help.

[nikhilNava]

@nadahaninou can you share the correlation ID so I see if all the sinks are receiving your data? if possible please share the tenant id as well.

The observability service receives the payload and sent to collectors downstream. The Admin center is one such collector.
Ideally, you'll have to create an ICM so the relevant team can look into it.