feat(eve): split cache-aware token usage on local trace spans (OTel GenAI semconv) by chadhietala · Pull Request #1391 · vercel/eve

chadhietala · 2026-07-30T01:54:50Z

Description

Follow-up from the cost-attribution investigation (#1388): model and agent.step spans now split token usage when the provider reports details — cached tokens price differently from plain input, so totals alone can't be cost-attributed exactly.

New attributes, named for the OTel GenAI semantic conventions (gen_ai.usage.cache_read.input_tokens / gen_ai.usage.cache_creation.input_tokens) so a future OTel visualizer speaks the same vocabulary:

gen_ai.usage.cache_read.input_tokens — input served from the provider-managed cache
gen_ai.usage.cache_creation.input_tokens — input written to the provider-managed cache

Only present when the provider reports detailed usage — providers without details emit only the agent.usage.input_tokens / agent.usage.output_tokens totals (unchanged). The split reconciles exactly: cacheRead + cacheCreation + uncached = input. (gen_ai.usage.reasoning.output_tokens is the adjacent semconv addition if we ever want the reasoning split too.)

Verification

Provider test asserts the semconv attributes land on both the model span and the step span.
End-to-end against the real gateway (weather fixture): cold step writes the cache (cache_creation: 5917), later steps read it back (cache_read: 5917 / 6027 + small incremental writes).
Full suite green: fmt, lint, typecheck, unit, integration, guard:invariants, docs:check.

Independent of #1388 (gateway cost attributes) — merges cleanly on its own; together they give computed-cost and gateway-reported-cost in the spool.

vercel · 2026-07-30T01:54:57Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
eve-docs	Building	Preview	Jul 30, 2026 2:12pm
eve-docs-4759	Ready	Preview, v0	Jul 30, 2026 2:12pm

github-actions · 2026-07-30T01:58:12Z

Bundle + Package Summary: `apps/fixtures/weather-agent`

Key takeaways

No notable deltas vs main (33ea372).

Delta vs `main (33ea372)`

Area	Metric	Baseline	Current	Delta
Package	Packed tarball	7.62 MB	7.62 MB	+56 B ⚠️
Package	Unpacked publish size	28.79 MB	28.79 MB	+727 B ⚠️
Package	Installed footprint	91.26 MB	91.26 MB	+727 B ⚠️
Package	Published files	2875	2875	0
Package	Installed files	6664	6664	0
Runtime	Unique function payloads	2	2	0
Runtime	Total function bytes	16.80 MB	16.80 MB	-40 B ✅
Runtime	Public routes	11	11	0

Changed function payloads vs main (33ea372) (2)

Function	Status	Baseline	Current	Delta	Route changes
`functions/__server.func`	changed	8.40 MB	8.40 MB	-20 B ✅	none
`functions/.well-known/workflow/v1/flow.func`	changed	8.40 MB	8.40 MB	-20 B ✅	none

`eve init` install

Metric	Baseline	Current	Delta
Installed footprint	129.67 MB	129.67 MB	+727 B ⚠️
Installed packages	130	130	0
dependencies	4	4	0
devDependencies	2	2	0
Dependency package bytes	43.12 MB	43.12 MB	+727 B ⚠️
devDependency package bytes	5.04 MB	5.04 MB	0 B ➖

Build Metadata

Preset: vercel
Nitro: nitro@3.0.260610-beta
Output directory: apps/fixtures/weather-agent/.vercel/output
Build metadata timestamp: 2026-07-30T14:13:22.453Z
Route aliases: 11 public, 1 internal (12 total aliases)
Vercel routes in config: 14
Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small

Package Drill-Down

Package Details

Package: eve@0.27.13
Package directory: packages/eve
Tarball: 7.62 MB (eve-0.27.13.tgz)
Unpacked payload: 28.79 MB across 2875 published files
Installed footprint: 91.26 MB across 6664 installed files
Installed root package: 27.43 MB
Installed dependencies: 63.83 MB
Runtime dependencies: 2
Peer dependencies: 5 (4 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.

Heavy installed dependencies

eve: 27.43 MB (30.1%)
@rolldown/binding-linux-x64-gnu: 19.28 MB (21.1%)
@rolldown/binding-wasm32-wasi: 10.66 MB (11.7%)
@napi-rs/wasm-runtime: 6.56 MB (7.2%)
ai: 6.53 MB (7.2%)

Publish payload breakdown

Published file size
🔴 dist/src/compiled/shadcn-registry/index.js       [########################] 13.15 MB 45.7%
🟠 dist/src/compiled/experimental-ai-sdk-code-mo... [###.....................] 1.51 MB 5.2%
🟡 dist/src/compiled/@vercel/sandbox/index.js       [#.......................] 632.5 kB 2.2%
🟡 dist/src/compiled/_chunks/workflow/undici-DWL... [#.......................] 502.4 kB 1.7%
🟡 dist/src/compiled/@chat-adapter/slack/index.js   [#.......................] 440.5 kB 1.5%
🔴 Other published files                            [#######################.] 12.56 MB 43.6%

Installed footprint breakdown

Installed package size
🔴 eve                             [########################] 27.43 MB 30.1%
🔴 @rolldown/binding-linux-x64-gnu [#################.......] 19.28 MB 21.1%
🔴 @rolldown/binding-wasm32-wasi   [#########...............] 10.66 MB 11.7%
🔴 @napi-rs/wasm-runtime           [######..................] 6.56 MB 7.2%
🔴 ai                              [######..................] 6.53 MB 7.2%
🔴 zod                             [####....................] 5.07 MB 5.6%
🔴 Other installed packages        [##############..........] 15.73 MB 17.2%

Runtime dependencies (2)

Package	Range	Notes
`nitro`	`3.0.260610-beta`
`undici`	`8.9.0`

Peer dependencies (5)

Package	Range	Notes
`@opentelemetry/api`	`^1.0.0`	optional peer
`ai`	`catalog:`
`braintrust`	`^3.0.0`	optional peer
`just-bash`	`^3.0.0`	optional peer
`microsandbox`	`^0.5.0`	optional peer

eve init install drill-down

`eve init` install details

Command: eve init my-agent
Package manager: npm
Installed footprint: 129.67 MB across 8532 installed files
Installed packages: 130 total (124 transitive-only)
dependencies: 4 direct packages totaling 43.12 MB
devDependencies: 2 direct packages totaling 5.04 MB
Other transitive package files: 81.50 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

@typescript/typescript-linux-x64: 27.95 MB (21.6%)
eve: 27.43 MB (21.2%)
@rolldown/binding-linux-x64-gnu: 19.28 MB (14.9%)
@rolldown/binding-wasm32-wasi: 10.66 MB (8.2%)
zod: 9.02 MB (7.0%)

Installed footprint breakdown

Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 21.6%
🔴 eve                              [########################] 27.43 MB 21.2%
🔴 @rolldown/binding-linux-x64-gnu  [#################.......] 19.28 MB 14.9%
🔴 @rolldown/binding-wasm32-wasi    [#########...............] 10.66 MB 8.2%
🔴 zod                              [########................] 9.02 MB 7.0%
🔴 @napi-rs/wasm-runtime            [######..................] 6.56 MB 5.1%
🔴 ai                               [######..................] 6.53 MB 5.0%
🔴 Other installed packages         [###################.....] 22.23 MB 17.1%

dependencies (4)

Package	Range	Installed size	Share
`@vercel/connect`	`0.4.2`	135.8 kB	0.1%
`ai`	`^7.0.38`	6.53 MB	5.0%
`eve`	`file:eve-0.27.13.tgz`	27.43 MB	21.2%
`zod`	`4.4.3`	9.02 MB	7.0%

devDependencies (2)

Package	Range	Installed size	Share
`@types/node`	`24.x`	2.54 MB	2.0%
`typescript`	`7.0.2`	2.50 MB	1.9%

Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 8.40 MB 50.0%
🔴 functions/__server.func                         [########################] 8.40 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 8.40 MB

Metric	Value
Public routes	`/.well-known/workflow/v1/flow`
Runtime	`nodejs24.x`
Handler	`index.mjs`
Payload	8.40 MB
Function files	8.40 MB across 44 files
Traced dependencies	0 B
Signal	🟠 Bundled file `index.mjs` is 2.30 MB (27.4%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                       [########################] 2.30 MB 27.4%
🟠 _chunks/runtime-artifacts.mjs   [################........] 1.59 MB 18.9%
🟡 _libs/undici.mjs                [##########..............] 980.5 kB 11.7%
🟡 _chunks/sandbox.mjs             [########................] 768.8 kB 9.2%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.2%
🟠 Other bundled files             [########################] 2.33 MB 27.7%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 10 public routes, 1 internal alias • 8.40 MB

Metric	Value
Public routes	`/` `/eve/v1/callback/[token]` `/eve/v1/connections/[name]/callback/[token]` `/eve/v1/health` `/eve/v1/info` `/eve/v1/session` `/eve/v1/session/[sessionId]` `/eve/v1/session/[sessionId]/cancel` `/eve/v1/session/[sessionId]/stream` `/eve/v1/session/reset`
Internal aliases	`/__server`
Runtime	`nodejs24.x`
Handler	`index.mjs`
Payload	8.40 MB
Function files	8.40 MB across 44 files
Traced dependencies	0 B
Signal	🟠 Bundled file `index.mjs` is 2.30 MB (27.4%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                       [########################] 2.30 MB 27.4%
🟠 _chunks/runtime-artifacts.mjs   [################........] 1.59 MB 18.9%
🟡 _libs/undici.mjs                [##########..............] 980.5 kB 11.7%
🟡 _chunks/sandbox.mjs             [########................] 768.8 kB 9.2%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.2%
🟠 Other bundled files             [########################] 2.32 MB 27.7%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: `e2e/fixtures/agent-tools-sandbox`

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

Build pipeline: 2.05 s -> 2.84 s (+790.7 ms) vs main (33ea372).
Timing is informational: shared GitHub runners are too variable for a hard timing budget.

Detailed phase timings vs `main (33ea372)`

Phase	Baseline	Current	Delta
`extension.check`	6.6 ms	1.0 ms	-5.6 ms
`project.resolve`	3.5 ms	0.6 ms	-2.9 ms
`workspace.create`	1.2 ms	0.7 ms	-0.5 ms
`host.prepare`	179.4 ms	149.3 ms	-30.1 ms
`vercel.service-prefix.resolve`	2.4 ms	2.3 ms	-0.1 ms
`nitro.create`	199.6 ms	216.2 ms	+16.6 ms
`sandbox.prewarm`	307.4 ms	1.15 s	+844.5 ms
`nitro.cache.prepare`	0.2 ms	0.2 ms	0.0 ms
`nitro.prepare`	0.9 ms	0.8 ms	-0.1 ms
`nitro.public-assets`	0.7 ms	0.8 ms	+0.1 ms
`nitro.prerender`	0.5 ms	0.5 ms	0.0 ms
`nitro.bundle`	1.31 s	1.28 s	-30.6 ms
`nitro.cache.write`	0.3 ms	0.4 ms	+0.1 ms
`vercel.workflow-function.materialize`	23.4 ms	22.6 ms	-0.8 ms
`agent-summary.emit`	0.5 ms	0.5 ms	0.0 ms
`nitro.close`	0.1 ms	0.1 ms	0.0 ms
`output.publish`	3.5 ms	3.5 ms	0.0 ms
`workspace.remove`	2.2 ms	2.2 ms	0.0 ms

Model and step spans now record gen_ai.usage.cache_read.input_tokens and gen_ai.usage.cache_creation.input_tokens (OTel GenAI semantic convention names) alongside the input/output totals when the provider reports detailed usage. Cached tokens price differently, so the split makes cost attribution exact; providers without detailed usage emit only the totals. Signed-off-by: Chad Hietala <chadhietala@gmail.com>

vercel Bot deployed to Preview – eve-docs-4759 July 30, 2026 01:56 View deployment

vercel Bot deployed to Preview – eve-docs July 30, 2026 01:56 View deployment

chadhietala changed the title ~~feat(eve): split cache-aware token usage on local trace spans~~ feat(eve): split cache-aware token usage on local trace spans (OTel GenAI semconv) Jul 30, 2026

vercel Bot deployed to Preview – eve-docs-4759 July 30, 2026 12:23 View deployment

vercel Bot deployed to Preview – eve-docs July 30, 2026 12:24 View deployment

AndrewBarba approved these changes Jul 30, 2026

View reviewed changes

chadhietala force-pushed the token-split-tracing branch from e27f9ec to 2326973 Compare July 30, 2026 14:10

vercel Bot deployed to Preview – eve-docs-4759 July 30, 2026 14:12 View deployment

vercel Bot deployed to Preview – eve-docs July 30, 2026 14:12 View deployment

chadhietala merged commit dbc8eae into main Jul 30, 2026
181 of 188 checks passed

chadhietala deleted the token-split-tracing branch July 30, 2026 14:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(eve): split cache-aware token usage on local trace spans (OTel GenAI semconv) - #1391

feat(eve): split cache-aware token usage on local trace spans (OTel GenAI semconv)#1391
chadhietala merged 1 commit into
mainfrom
token-split-tracing

chadhietala commented Jul 30, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Jul 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 30, 2026 •

edited

Loading

Package Details

`eve init` install details

Payload Size Graph

Top Function Payloads

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

chadhietala commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Verification

Uh oh!

vercel Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bundle + Package Summary: apps/fixtures/weather-agent

Delta vs main (33ea372)

eve init install

Package Details

eve init install details

Payload Size Graph

Top Function Payloads

Build Timing: e2e/fixtures/agent-tools-sandbox

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chadhietala commented Jul 30, 2026 •

edited

Loading

vercel Bot commented Jul 30, 2026 •

edited

Loading

github-actions Bot commented Jul 30, 2026 •

edited

Loading

Bundle + Package Summary: `apps/fixtures/weather-agent`

Delta vs `main (33ea372)`

`eve init` install

`eve init` install details

Build Timing: `e2e/fixtures/agent-tools-sandbox`