Skip to content

feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption#1270

Merged
jsonbailey merged 3 commits intofeat/ai-sdk-next-releasefrom
jb/aic-2207/update-ai-sdks-billing-spec
Apr 15, 2026
Merged

feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption#1270
jsonbailey merged 3 commits intofeat/ai-sdk-next-releasefrom
jb/aic-2207/update-ai-sdks-billing-spec

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

@jsonbailey jsonbailey commented Apr 14, 2026

Summary

  • Each tracker now carries a runId (UUIDv4) included in all emitted events, scoping every metric to a single execution
  • At-most-once semantics enforce that trackDuration, trackTokens, trackSuccess/trackError, trackFeedback, and trackTimeToFirstToken can only fire once per tracker instance; subsequent calls are dropped with a warning
  • tracker property on config objects replaced by a createTracker() factory — each call returns a new tracker with a fresh runId, making repeated invoke() / evaluate() calls fully independent
  • TrackedChat and Judge call createTracker() internally at the start of each invocation rather than holding a shared tracker
  • resumptionToken getter on the tracker encodes runId, configKey, variationKey, and version as URL-safe Base64 JSON, enabling cross-process event correlation
  • LDAIConfigTrackerImpl.fromResumptionToken() and LDAIClient.createTracker() reconstruct a tracker from a token with the original runId preserved — useful for associating deferred user feedback with the originating execution

Breaking changes

  • tracker property removed from all config types; replaced by createTracker?: () => LDAIConfigTracker
  • TrackedChat and Judge constructors no longer accept a tracker argument
  • getTracker() removed from both TrackedChat and Judge
  • getTrackData() return type now includes runId: string as the first field

Test plan

  • yarn workspace @launchdarkly/server-sdk-ai test — all 110 tests pass
  • yarn workspace @launchdarkly/server-sdk-ai lint — no lint errors
  • Verify resumptionToken round-trips correctly through fromResumptionToken
  • Verify duplicate metric calls on the same tracker produce warnings and are dropped

🤖 Generated with Claude Code


Note

Medium Risk
Breaking API change replaces tracker with createTracker() and changes how metrics are emitted; incorrect adoption could drop or misattribute analytics events. Adds new runId/resumption-token parsing and at-most-once guards that may affect existing metric call patterns.

Overview
Refactors AI config tracking to be per-execution. Config objects no longer expose a shared tracker; they now provide createTracker() which returns a fresh tracker (new runId) each invocation, and TrackedChat/Judge construct a tracker internally at the start of invoke()/evaluate().

Adds run correlation and tracker resumption. LDAIConfigTracker now includes runId in getTrackData(), LDAIConfigTrackerImpl can emit/parse a URL-safe Base64 resumptionToken, and LDAIClientImpl adds createTracker(token, context) to reconstruct a tracker for deferred tracking across processes.

Enforces at-most-once metric emission per tracker instance. Duplicate calls to trackDuration, trackTokens, trackSuccess/trackError, trackFeedback, and trackTimeToFirstToken are dropped with warnings. Tests and examples are updated to the new createTracker() flow and constructor signatures (removing getTracker() and tracker constructor args).

Reviewed by Cursor Bugbot for commit e2b5362. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
Copy link
Copy Markdown
Contributor

@launchdarkly/js-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 25623 bytes
Compressed size limit: 29000
Uncompressed size: 125843 bytes

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 14, 2026

@launchdarkly/js-client-sdk size report
This is the brotli compressed size of the ESM build.
Compressed size: 31655 bytes
Compressed size limit: 34000
Uncompressed size: 112792 bytes

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 14, 2026

@launchdarkly/browser size report
This is the brotli compressed size of the ESM build.
Compressed size: 179375 bytes
Compressed size limit: 200000
Uncompressed size: 829982 bytes

@github-actions
Copy link
Copy Markdown
Contributor

@launchdarkly/js-client-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 37169 bytes
Compressed size limit: 38000
Uncompressed size: 204305 bytes

@jsonbailey jsonbailey marked this pull request as ready for review April 14, 2026 17:10
@jsonbailey jsonbailey requested a review from a team as a code owner April 14, 2026 17:10
Comment thread packages/sdk/server-ai/src/api/chat/TrackedChat.ts
Copy link
Copy Markdown
Contributor

@andrewklatzke andrewklatzke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had one question but overall LGTM

@jsonbailey jsonbailey changed the base branch from main to ai-sdk-next-release April 14, 2026 21:46
jsonbailey and others added 2 commits April 14, 2026 17:00
…ess tracker resumption

- Each tracker now carries a `runId` (UUIDv4) included in all emitted
  events, scoping every metric to a single execution
- At-most-once semantics: duplicate calls to trackDuration, trackTokens,
  trackSuccess/trackError, trackFeedback, and trackTimeToFirstToken on
  the same tracker are dropped with a warning
- `tracker` property on config objects replaced by `createTracker()` factory;
  each call returns a new tracker with a fresh runId so repeated invocations
  are independent
- `TrackedChat` and `Judge` now call `createTracker()` internally per
  invocation rather than accepting a pre-created tracker
- `resumptionToken` getter encodes runId, configKey, variationKey, and
  version as URL-safe Base64 JSON for cross-process correlation
- `LDAIConfigTrackerImpl.fromResumptionToken()` and
  `LDAIClient.createTracker()` reconstruct a tracker from a token,
  preserving the original runId (useful for associating deferred feedback
  with the originating execution)

BREAKING CHANGE: `tracker` property removed from config types in favour of
`createTracker?: () => LDAIConfigTracker`. `TrackedChat` and `Judge`
constructors no longer accept a tracker argument. `getTracker()` methods
removed from both classes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the removed tracker property with the new createTracker() factory
pattern in bedrock, openai, and vercel-ai examples.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jsonbailey jsonbailey force-pushed the jb/aic-2207/update-ai-sdks-billing-spec branch from 197c366 to 7fc3a54 Compare April 14, 2026 22:05
@jsonbailey jsonbailey merged commit d640d8e into feat/ai-sdk-next-release Apr 15, 2026
43 checks passed
@jsonbailey jsonbailey deleted the jb/aic-2207/update-ai-sdks-billing-spec branch April 15, 2026 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants