Skip to content

feat(ai): add telemetry interface and registry#12881

Merged
aayush-kapoor merged 8 commits intomainfrom
aayush/tel-registry
Mar 3, 2026
Merged

feat(ai): add telemetry interface and registry#12881
aayush-kapoor merged 8 commits intomainfrom
aayush/tel-registry

Conversation

@aayush-kapoor
Copy link
Collaborator

@aayush-kapoor aayush-kapoor commented Feb 26, 2026

Background

we needed an interface that could be implemented by telemetry providers for capturing all the data that will be propagated via the callbacks

Summary

  • TelemetryHandler : interface to create custom telemetry integrations
  • expand-handlers.ts : file turns handlers: [handlerA, handlerB, handlerC] into a single object with one callback per lifecycle event, where each callback fans out to every handler that needs info about that event
  • added new option handlers to the telemetry-settings for handlers that receive lifecycle events during generation

Future Work

make the integrations option work behind the scenes, ie without a need for it to be in experimental_telemetry

Checklist

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

@tigent tigent bot added ai/telemetry feature New feature or request labels Feb 26, 2026
Copy link
Collaborator Author

@aayush-kapoor aayush-kapoor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the dx for the providers using this will be

import type { TelemetryHandler } from 'ai';
import { bindTelemetryHandler } from 'ai';

class DevToolsTelemetryHandler implements TelemetryHandler {
  private runId: string | undefined;
  private stepCounter = 0;
  private stepTimers = new Map<number, number>();

  async onStart(event) {
    ...
  }

  async onStepStart(event) {
    ...
  }

  async onToolCallStart(event) {
    // Log tool invocation to devtools UI
  }

  async onToolCallFinish(event) {
    // Log tool result (or error) with duration
  }

  async onStepFinish(event) {
    ...
  }

  async onFinish(event) {
    // Final summary: total usage, all steps, etc.
  }
}

// the public API - what end users import and pass to `handlers`
export function devToolsHandler(): TelemetryHandler {
  return bindTelemetryHandler(new DevToolsTelemetryHandler());
}

TOOLS extends ToolSet = ToolSet,
OUTPUT extends Output = Output,
> {
onStart?(event: OnStartEvent<TOOLS, OUTPUT>): PromiseLike<void> | void;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wonder if we can use / extract types such as OnStartListener - maybe in follow up prs

@lgrammel
Copy link
Collaborator

lgrammel commented Mar 2, 2026

the global listener would need to be called from generateText and streamText

@aayush-kapoor aayush-kapoor merged commit 2a4f512 into main Mar 3, 2026
18 checks passed
@aayush-kapoor aayush-kapoor deleted the aayush/tel-registry branch March 3, 2026 00:47
@gr2m gr2m added ai/core core functions like generateText, streamText, etc. Provider utils, and provider spec. and removed ai/telemetry labels Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai/core core functions like generateText, streamText, etc. Provider utils, and provider spec. feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants