fix(ai): Azure stream text/object otel fix

[jacobkerber]

Background

Many OTEL compatible sinks that monitor LLMs, like Langfuse, observe the generic telemetry endpoints for their data. There is a current issue where completed streamed text and objects do not have a value for the gen_ai.response.model and ai.response.model span attributes. This seems to be caused by the response-metadata case check where the chunk.id and chunk.modelId are given empty strings. The empty string evaluates as True but need to be evaluated as false.

Langfuse span attributes

{
  'operation.name': 'ai.streamText.doStream',
  'ai.operationId': 'ai.streamText.doStream',
  'ai.model.provider': 'azure-openai.chat',
  'ai.model.id': 'gpt-4.1-mini',
  'ai.settings.maxRetries': 2,
  'ai.telemetry.metadata.langfuseTraceId': 'ph7fgtd67bl24xu0c6j8owkr',
  'ai.telemetry.metadata.userId': 'example@email.com',
  'ai.telemetry.metadata.sessionId': '5507bea6-fd53-46e8-b36c-8b49727bb0f5',
  'ai.prompt.format': 'messages',
  'ai.prompt.messages': `[]`,
  'gen_ai.request.model': 'gpt-4.1-mini',
  'ai.response.msToFirstChunk': 339.0682999999999,
  'ai.response.msToFinish': 657.3819000000003,
  'ai.response.avgCompletionTokensPerSecond': 25.860158303719636,
  'ai.response.finishReason': 'stop',
  'ai.response.text': 'Hey there! How’s it going? What can I help you with today?',
  'ai.response.id': '',
  'ai.response.model': '',
  'ai.response.timestamp': '1970-01-01T00:00:00.000Z',
  'ai.usage.promptTokens': 873,
  'ai.usage.completionTokens': 17,
  'gen_ai.response.finish_reasons': [ 'stop' ],
  'gen_ai.response.id': '',
  'gen_ai.response.model': '',
  'gen_ai.usage.input_tokens': 873,
  'gen_ai.usage.output_tokens': 17
}

Summary

I added another check in the response-metadata case statement to check for an empty string case. I am sure there is an upstream issue causing this behavior, but this was an easy fix.

case 'response-metadata': {
    stepResponse = {
        id:
            chunk.id && chunk.id !== ''
                ? chunk.id
                : stepResponse.id,
        timestamp: chunk.timestamp ?? stepResponse.timestamp,
        modelId:
            chunk.modelId && chunk.modelId !== ''
                ? chunk.modelId
                : stepResponse.modelId,
        };
    break;
}

Verification

I installed this package locally and made the adjustment. I then used pnpm link to link it to a local LLM app we have developed. I logged the attributes sent to Langfuse and this is the result wit hthe proper modelId and id present for the response.

Langfuse attributes {
  'operation.name': 'ai.streamText.doStream',
  'ai.operationId': 'ai.streamText.doStream',
  'ai.model.provider': 'azure-openai.chat',
  'ai.model.id': 'gpt-4.1-mini',
  'ai.settings.maxRetries': 2,
  'ai.telemetry.metadata.langfuseTraceId': 'nvyf2xx6vj9snftu4t2ugl71',
  'ai.telemetry.metadata.userId': 'example@email.com',
  'ai.telemetry.metadata.sessionId': 'ab7a0bca-ea86-43cb-8352-49f9a0238b7d',
  'ai.prompt.format': 'messages',
  'ai.prompt.messages': `[]`,
  'gen_ai.system': 'azure-openai.chat',
  'gen_ai.request.model': 'gpt-4.1-mini',
  'ai.response.msToFirstChunk': 310.3084999999992,
  'ai.response.msToFinish': 1058.8732999999993,
  'ai.response.avgCompletionTokensPerSecond': 20.77680115269694,
  'ai.response.finishReason': 'stop',
  'ai.response.text': 'Awesome! If you want me to take a look or help test it out, just let me know.',
  'ai.response.id': 'aitxt-THy0GJ6G0SOYNO3J9k4YOwB6',
  'ai.response.model': 'gpt-4.1-mini',
  'ai.response.timestamp': '1970-01-01T00:00:00.000Z',
  'ai.usage.promptTokens': 937,
  'ai.usage.completionTokens': 22,
  'gen_ai.response.finish_reasons': [ 'stop' ],
  'gen_ai.response.id': 'aitxt-THy0GJ6G0SOYNO3J9k4YOwB6',
  'gen_ai.response.model': 'gpt-4.1-mini',
  'gen_ai.usage.input_tokens': 937,
  'gen_ai.usage.output_tokens': 22
}

Tasks

• Tests have been added / updated (for bug fixes / features)
• Documentation has been added / updated (for bug fixes / features)
• A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
• Formatting issues have been fixed (run pnpm prettier-fix in the project root)

Future Work

Related Issues

Fixes bug: Model name not set when using Vercel AI SDK and Azure OpenAI provider #5286

[jacobkerber]

I know this is a minor change, but this fixes the telemetry in Langfuse which is somewhat important. When can we get this merged?

[Josh-Zirena]

I am also having the same issue as @jacobkerber and this PR should fix the issue for us as well.