Skip to content

feat(context): track context tokens#2009

Merged
lizradway merged 3 commits intostrands-agents:mainfrom
lizradway:messages-token-size
Apr 1, 2026
Merged

feat(context): track context tokens#2009
lizradway merged 3 commits intostrands-agents:mainfrom
lizradway:messages-token-size

Conversation

@lizradway
Copy link
Copy Markdown
Member

@lizradway lizradway commented Mar 30, 2026

Description

Add a latest_context_size property to EventLoopMetrics and a context_size convenience property on AgentResult that surface the most recent context window size (in tokens) as reported by the model.

Strands currently has no visibility into how full the context window is. The LLM already reports inputTokens on every call, but that data was buried in per-cycle usage metrics. These properties expose it as a simple, top-level read:

agent = Agent(tools=[...])
result = agent("Do something with tools")

# Via AgentResult
result.context_size

# Via EventLoopMetrics
agent.event_loop_metrics.latest_context_size

This is a lagging indicator (post-call), not a pre-call estimate. It enables downstream features like compression and externalization to make threshold-based decisions between invocations.

  • Zero overhead: reuses inputTokens already returned by every LLM call
  • Zero latency impact: no additional API calls
  • Provider agnostic: all providers normalize to the Usage TypedDict with inputTokens
  • Fully backward compatible: adds read-only properties, no existing behavior changes

Related Issues

Closes #1197

Documentation PR

N/A — no user-facing documentation changes needed for this property addition.

Type of Change

New feature

Testing

  • I ran hatch run prepare
  • Added unit tests for EventLoopMetrics.latest_context_size: no invocations, no cycles, multiple cycles, multiple invocations, missing inputTokens key
  • Added unit tests for AgentResult.context_size: delegates to metrics, returns None when no data

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@lizradway lizradway added the area-context Session or context related label Mar 30, 2026
@lizradway lizradway temporarily deployed to manual-approval March 30, 2026 20:48 — with GitHub Actions Inactive
@lizradway lizradway changed the title feat(context): track agent.messages token size feat(context): track context tokens Mar 30, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions
Copy link
Copy Markdown

Assessment: Approve ✅

Clean, minimal implementation that addresses issue #1197 with zero overhead. The tests are comprehensive and cover all edge cases well.

Review Details
  • API Design: Read-only property that follows existing patterns (latest_agent_invocation); intuitive naming
  • Testing: Good coverage including no-invocations, no-cycles, multi-cycle, multi-invocation, and defensive provider-bug scenarios
  • Documentation: PR description is thorough with clear usage example

Nice work keeping this focused and backward-compatible! 🎉

@github-actions
Copy link
Copy Markdown

Thanks for addressing the docstring feedback! The Returns section looks good.

Assessment: ✅ Approve - Ready to merge.

@kevmyung
Copy link
Copy Markdown
Contributor

Hi @lizradway, Thanks for this PR! I've been hoping for this kind of context token tracking in Strands for a while.

One concern: this works well when the agent instance stays in memory, but when the agent is restored from a session (e.g., via RepositorySessionManager), EventLoopMetrics is re-initialized from scratch — so latest_context_tokens returns 0 until the next LLM call completes. This could cause issues for smart compaction strategies that rely on this value to make threshold-based decisions immediately after restoration.

To address this, I think we'd also need per-message metrics metadata that gets persisted and restored alongside the messages themselves. I opened #1532 with a proposal for adding a _metadata field to messages for this purpose — it could complement this PR nicely.

@lizradway lizradway force-pushed the messages-token-size branch from 3771ebd to 516837d Compare April 1, 2026 16:02
@github-actions github-actions bot added size/s and removed size/s labels Apr 1, 2026
@lizradway lizradway deployed to manual-approval April 1, 2026 16:02 — with GitHub Actions Active
@lizradway lizradway added the needs-api-review Makes changes to the public API surface label Apr 1, 2026
@lizradway lizradway requested a review from zastrowm April 1, 2026 16:07
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 1, 2026

Assessment: Approve (pending API review) ✅

The PR has evolved well based on review feedback. Key improvements since last review:

Changes Summary
Change Before After
Property name latest_context_tokens latest_context_size
Return type int (0 for no data) int | None (None for no data)
API surface EventLoopMetrics only + AgentResult.context_size

Code Quality: ✅ Clean implementation, comprehensive tests, proper type hints

Documentation: With AgentResult.context_size now being a user-facing property, consider whether a brief docs addition is warranted (see thread reply).

Awaiting API reviewer sign-off per the needs api review flag.

@lizradway lizradway enabled auto-merge (squash) April 1, 2026 16:40
@lizradway lizradway merged commit 635edbc into strands-agents:main Apr 1, 2026
41 of 48 checks passed
@lizradway lizradway deleted the messages-token-size branch April 1, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-context Session or context related needs-api-review Makes changes to the public API surface size/s

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CONTEXT] [FEATURE] Track context token size

3 participants