[BUG]: LLMObs for ai-sdk not capturing trace

[prescottprue]

Tracer Version(s)

5.63.0, latest

Node.js Version(s)

22.20.0

Bug Report

#5858 adds automatic tracing for Vercel's ai-sdk, but I'm not seeing LLM traces be captured. Other traces are captured just fine so setup of dd-trace is working.

Reproduction Code

NextJS endpoint which uses AI SDK:

import { openai } from '@ai-sdk/openai';
import {
  streamText,
  type UIMessage,
  convertToModelMessages,
} from 'ai';
import { tracer } from 'dd-trace';

tracer.init({
  service: 'chat-ui',
  logInjection: true,
  profiling: true,
  plugins: true,
  env: 'stage',
  version: '1.0.0',
  runtimeMetrics: true,
  hostname: '<hostname for agent>',
  llmobs: {
    mlApp: 'chat-app',
  },
});

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: openai('gpt-5'),
    messages: convertToModelMessages(messages),
    // enabling the following has no effect either:
    // experimental_telemetry: { isEnabled: true },
  });

  return result.toUIMessageStreamResponse();
}

I've also tried initializing dd-trace within instrumentation.ts file as described by Next as method of enabling open telemetry:

export async function register() {
  if (process.env.NEXT_RUNTIME === 'nodejs') {
    const { tracer } = await import('dd-trace');
    tracer.init({
      service: 'chat-ui',
      logInjection: true,
      profiling: true,
      plugins: true,
      env: 'stage',
      version: '1.0.0',
      runtimeMetrics: true,
      hostname: '<hostname for agent>',
      llmobs: {
        mlApp: 'chat-app',
      },
    });
    tracer.use('next');
  }

Error Logs

No response

Tracer Config

{
  service: 'chat-ui',
  logInjection: true,
  profiling: true,
  plugins: true,
  env: 'stage',
  version: '1.0.0',
  runtimeMetrics: true,
  hostname: '<hostname for agent>',
  llmobs: {
    mlApp: 'chat-app',
  },
}

Operating System

Darwin mbp-GR2MG2LHD7 24.6.0 Darwin Kernel Version 24.6.0

Bundling

Next.js

[sabrenner]

Hi @prescottprue! This is an issue we're coincidentally have already been looking into, and are trying a few things out - I believe the issue is how our patching interacts with a Next.js environment.

Would love to get some feedback on if this solution I was able to get working locally for my own Next.js app using ai:

export async function register() {
  if (process.env.NEXT_RUNTIME === 'nodejs') {
    const initializeImportName = 'dd-trace/initialize.mjs';
    await import(/* webpackIgnore: true */ initializeImportName as 'dd-trace/initialize.mjs')
  }
}

while defining your configuration in an .env or .env.local, e.g.

DD_LLMOBS_ENABLED=true
DD_LLMOBS_ML_APP=<YOUR-ML-APP>
DD_SERVICE=<YOUR-SERVICE>
DD_API_KEY=<YOUR-DD-API-KEY>
# ... all other configs

and this should be the only bit needed, no mention of dd-trace in your route.ts file should be required. this will make sure our ESM loader hooks are registered correctly (which is. curious if this resolves it for you, thanks!

[prescottprue]

@sabrenner I appreciate the fast response! When I do what you show above I get the following error:

[Error: An error occurred while loading instrumentation hook: Cannot find module '/app/node_modules/dd-trace/loader-hook.mjs' imported from /app/node_modules/dd-trace/initialize.mjs]

Which is weird given that I see the file in node_modules

To test further, I also did this same implementation with AI sdk within one of our fastify services - we still see standard APM traces including calls to openai + mcp client, but we do not see anything show up in the LLM Observability traces list. Even when using @anthropic-ai/sdk directly in our fastify service (instead of AI SDK), I'm still not seeing any LLM Observability trace even though I see an APM trace

I don't believe it matters, but wanted to callout that we are using an agent instead of the api key approach

[sabrenner]

Interesting, thanks for sharing an update! Can you confirm that the llmobs environment variables are being set? There should be a set of logs like these

Configuring LLMObsEvalMetricsWriter to http://127.0.0.1:8126/evp_proxy/v2/api/intake/llm-obs/v1/eval-metric
Configuring LLMObsSpanWriter to http://127.0.0.1:8126/evp_proxy/v2/api/v2/llmobs
[LLMObs] Enabled LLM Observability with configuration: { agentlessEnabled: false, enabled: true, mlApp: <YOUR-ML-APP> }

or wherever you have your agent configured if it's not. If those are showing up but then LLM Observability traces are still not showing up, then I'll have to approach this from a different angle.

As for the Cannot find module '/app/node_modules/dd-trace/loader-hook.mjs' imported from /app/node_modules/dd-trace/initialize.mjs] error... that one's even more strange. I'll try and reproduce that myself, since I have not seen that show up before.

Is it only showing up in production deployments? Or in dev runs too either with next dev --turbo or next start? And, if in prod, is it deployed via Vercel? If not, maybe NODE_OPTIONS="--import dd-trace/initialize.mjs" would be better.

[prescottprue]

Yes I can confirm that the LLMmobs environment variables are being set both on hosted and locally. That said, since the initialize file isn't working on hosted, I've been initializing the tracer manually with a configuration objects instead of needing to depend on the environment variables (even though those are still set).

On local I see the LLMobs log showing there is a error calling to our datadog agent, but that is because of networking (our DD agent is only available via our VPC):

[Error: Cannot send LLM Observability data without a running agent or without both a Datadog API key and site.
[next] Ensure these configurations are set before running your application.]

However I don't see any logs on the hosted version.

It is only showing up in production deployments - development runs with next dev --turbo load the initialize file just fine. Deployed version is deployed via Google Cloud Run.

I tried NODE_OPTIONS="--import dd-trace/initialize.mjs" as well, but I'm getting the same issue of not being able to load.

I'll keep trying to debug and see if I can gather any more info

[prescottprue]

Really appreciate the help debugging. I still haven't been able to get the automatic tracing of ai-sdk working, but I was able to get traces to record to LLM Observability by doing llmobs.wrap around my call - this proves that the initialization of dd-trace is enabling the LLMobs stuff, but for some reason isn't automatically wrapping ai-sdk. The wrap call doesn't capture any data though

Wanted to callout that I'm on ai-sdk version 5.0.72 and dd-trace version 5.73.0 (also had tried 5.72.0)

I opened a formal support request to link it to our datadog account: https://help.datadoghq.com/hc/en-us/requests/2321575

[sabrenner]

Hi @prescottprue - thanks for so much detail! This helps a lot 😄

I forgot to mention this, but could you add ai (and any other packages you wish to be patched) to the serverExternalPackages section in your next.config.ts, e.g.

import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  ...
  serverExternalPackages: ["ai"], // additionally, "anthropic", "openai", etc. if you are importing these packages as well
};

export default nextConfig;

[prescottprue]

@sabrenner Makes sense - added, but still the same thing, no LLM Observability traces being captured

I wanted to callout though that I'm not using type: "module" in our package.json, so I'm wondering if that plays a role into why dd-trace/initialize.mjs is having an issue loading dd-trace/loader-hook.mjs

When looking at our startup logs for dd-trace I noticed that it doesn't seem that the integration for ai-sdk is loaded (although I don't see it in the list of integrations to enable via tracer.use, so maybe it works different than others?):

{"date":"2025-10-27T22:53:17.991Z","os_name":"Linux","os_version":"4.4.0","architecture":"x64","version":"5.73.0","lang":"nodejs","lang_version":"22.20.0","env":"stage","service":"identity-ui","agent_url":"http://datadog-agent-url:8126","debug":false,"sampling_rules":[],"tags":{"service":"identity-ui","env":"stage","version":"pr-96fa003","runtime-id":"05429f14-2a1c-4cc7-9fb4-ea3a5f362be3","_dd.rc.client_id":"9773a388-fa6e-412c-bab9-cd286330cb72"},"dd_version":"pr-96fa003","log_injection_enabled":true,"runtime_metrics_enabled":true,"profiling_enabled":true,"integrations_loaded":["dns","net","http","pino"],"appsec_enabled":false}

I tried manually enabling llmobs via tracer.llmobs.enable, and when running that I do see logs confirming that it is configured, but still not seeing any LLM Observability traces:

[sabrenner]

I wanted to callout though that I'm not using type: "module" in our package.json, so I'm wondering if that plays a role into why dd-trace/initialize.mjs is having an issue loading dd-trace/loader-hook.mjs

This is also the case for my ai-sdk Next.js test app as well, although I don't see any issues.

I do wonder if externalizing dd-trace as well might help with this issue

serverExternalPackages: ["ai", "dd-trace"]

or

const nextConfig = {
  webpack: (config, { isServer }) => {
    if (isServer) {
      config.externals.push('ai');
      config.externals.push('dd-trace');
    }
    return config;
  },
};

since the initialize.mjs is requiring loader-hook.mjs as a relative path in a Module.register call. unsure if this will solve it since i'm not seeing the issue still on my side but might be worth a try. thanks for your patience so far in troubleshooting this!

[mangersmiovision]

TypeScript types for dd-trace don't include the 'ai' plugin yet. The cast to any works around this - the plugin exists at runtime.

(tracer as any).use('ai', true);

[sabrenner]

TypeScript types for dd-trace don't include the 'ai' plugin yet. The cast to any works around this - the plugin exists at runtime.

(tracer as any).use('ai', true);

ahhh thank you for calling this out, i will have this fixed momentarily 😅

@mangersmiovision are you saying you were also seeing the same issue as @prescottprue, but that calling tracer.use('ai', true) resolved it (and, maybe in addition to some of the other changes i had suggested, or maybe not)?

[mangersmiovision]

@sabrenner I'm not sure if I had the exact same problem, but this is the only way that I was able to enable tracing for ai-sdk.

[neil-gebbie-smarterley]

I've been working through this problem for a few days, and I have it working now (locally, will deploy to a remote environment shortly).

@sabrenner I've noted in your examples your using Webpack - Is that just because your "testing app" is Webpack or is it recommended over Turbopack?

A few things to note:
I'm using the agent as a docker container (latest)
Ai 5.0.86
dd-trace 5.73.0
next 15.5.4
@ai-sdk/azure 2.0.45 (not sure if the model makes a difference because of the data returned from azure...)

Here's how I got nextjs and ai sdk traces to come through:
Anything that needed to be wrapped here I had to change the import style.
So

import { generateObject, streamText, etc } from "ai";

would now be:

const { generateObject, streamText, etc } = require("ai");

As per above in next.config.ts I added:
serverExternalPackages: ["ai", "dd-trace"]

In package.json (you can call tracer.mjs anything you want...)

    "dev": "node --import ./tracer.mjs --enable-source-maps node_modules/.bin/next dev --turbopack -p 3010",
    "start": "node --import ./tracer.mjs --enable-source-maps node_modules/.bin/next start",

tracer.mjs:

import tracer from "dd-trace";

// Sampler function to filter out noisy traces
const sampler = (span) => {
  const resourceName = span.get("resource.name") || "";

  // Filter out Next.js static files
  if (resourceName.includes("GET /_next/static/")) {
    return false;
  }

  // Filter out favicon requests
  if (resourceName.includes("GET /favicon.ico")) {
    return false;
  }

  // Filter out telemetry requests to next.js
  if (resourceName.includes("POST") && resourceName.includes("telemetry.nextjs.org")) {
    return false;
  }

  return true;
};

const agentHostname = process.env.DD_AGENT_HOSTNAME || "localhost";
const DD_SITE = "datadoghq.eu";
process.env.DD_LLMOBS_ENABLED = process.env.DD_LLMOBS_ENABLED || "1";
process.env.DD_LLMOBS_ML_APP = process.env.DD_LLMOBS_ML_APP || "your-service";

tracer.init({
  debug: false,
  startupLogs: true,
  logInjection: true,
  service: "your-service",
  hostname: agentHostname,
  port: 8126,
  // Custom sampler to filter out noisy traces
  sampler,
  llmobs: {
    mlApp: process.env.DD_LLMOBS_ML_APP,
    // Agentless disabled - we use the Docker agent to forward traces to Datadog
    agentlessEnabled: false,
  },
  site: DD_SITE,
});

Create a datadog agent with these environment variables (some might not be needed, I'll need to check)

DD_SITE: datadoghq.eu/or .com if your not in europe
DD_ENV: dev
DD_LOGS_ENABLED: "true"
DD_APM_ENABLED: "true"
DD_TRACE_ENABLED: "true"
DD_LLMOBS_ENABLED: "true"
DD_LLMOBS_ML_APP: "your-service"
DD_SERVICE: "your-service"
DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL: "true"