Skip to content

chore: add LangGraph node_metrics isolation regression test#176

Merged
jsonbailey merged 1 commit intomainfrom
jb/langgraph-node-metrics-test
May 7, 2026
Merged

chore: add LangGraph node_metrics isolation regression test#176
jsonbailey merged 1 commit intomainfrom
jb/langgraph-node-metrics-test

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

@jsonbailey jsonbailey commented May 7, 2026

Summary

Adds the missing regression test that should have shipped alongside the LangGraph runner migration in #156. Mirrors the OpenAI version added in #155.

Bug class being guarded

The graph-runner pattern uses a callback-style accumulator (LDMetricsCallbackHandler for LangGraph, _node_* state dicts for OpenAI) to collect per-node metrics during a run(). If that accumulator is held on the runner instance across run() calls, the second invocation's node_metrics ends up containing stale entries from the first run.

#155 fixed this for the OpenAI runner and shipped a regression test:

  • packages/ai-providers/server-ai-openai/tests/test_openai_agent_graph_runner.py::test_openai_agent_graph_runner_run_resets_node_metrics_between_runs

#156 migrated the LangGraph runner to the same AgentGraphRunnerResult shape but did not include the equivalent regression test. This PR adds it.

What this PR does

Adds test_langgraph_runner_run_resets_node_metrics_between_runs in packages/ai-providers/server-ai-langchain/tests/test_langgraph_agent_graph_runner.py. It calls runner.run() twice on the same LangGraphAgentGraphRunner instance, fires callback events on the handler each time, and asserts that the second result's node_metrics:

  • contains only the second invocation's nodes (not a union with the first)
  • holds a fresh LDAIMetrics instance, not the same object returned on the first run

No fix to the runner itself was needed: LangGraphAgentGraphRunner.run() already constructs a fresh LDMetricsCallbackHandler per call, so state is isolated by construction. The test locks that behaviour in.

Test plan

  • make test-langchain passes (86 tests)
  • make test passes across all packages (server-ai 200, langchain 86, openai 44)
  • make lint passes (mypy, isort, pycodestyle)
  • New test passes both in isolation and as part of the full suite

Note

Low Risk
Low risk: test-only change that tightens coverage around metrics collection across repeated run() calls, with no production logic modified.

Overview
Adds a new async regression test in test_langgraph_agent_graph_runner.py that calls LangGraphAgentGraphRunner.run() twice on the same instance and asserts the second result’s metrics.node_metrics and metrics.path are fresh per invocation (no stale/cumulative state from the first run).

The test bypasses graph compilation by injecting a mocked compiled graph whose ainvoke triggers LangChain callback events, specifically guarding against accidental reuse of the metrics callback handler/state across runs.

Reviewed by Cursor Bugbot for commit 637ee98. Bugbot is set up for automated code reviews on this repo. Configure here.

@jsonbailey jsonbailey marked this pull request as ready for review May 7, 2026 17:31
@jsonbailey jsonbailey requested a review from a team as a code owner May 7, 2026 17:31
Mirrors the OpenAI regression test added in PR #155. Verifies that
LangGraphAgentGraphRunner.run() produces a fresh node_metrics dict on
each invocation so callback-handler state cannot leak across runs on
the same runner instance.
@jsonbailey jsonbailey force-pushed the jb/langgraph-node-metrics-test branch from bd61c31 to 637ee98 Compare May 7, 2026 17:35
@jsonbailey jsonbailey merged commit 5898959 into main May 7, 2026
43 checks passed
@jsonbailey jsonbailey deleted the jb/langgraph-node-metrics-test branch May 7, 2026 21:40
@jsonbailey jsonbailey changed the title test: add LangGraph node_metrics isolation regression test chore: add LangGraph node_metrics isolation regression test May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants