docs(simulator): updated tool_simulator docs

[poshinchen]

Description

Added toolSimulator related docs

Related Issues

N/A

Type of Change

• New content
• Content update/revision

Checklist

• I have read the CONTRIBUTING document
• My changes follow the project's documentation style
• I have tested the documentation locally using npm run dev
• Links in the documentation are valid and working

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

Documentation Preview Ready

Your documentation preview has been successfully deployed!

Preview URL: https://d3ehv1nix5p99z.cloudfront.net/pr-cms-752/docs/user-guide/quickstart/overview/

Updated at: 2026-04-22T18:36:04.490Z

[github-actions]

Assessment: Comment

Good addition to the evals SDK documentation. The ToolSimulator docs are well-structured, follow the established patterns from the user_simulation.mdx page, and provide clear code examples with progressive complexity (basic → shared state → experiment integration → troubleshooting).

Review Details

• Accuracy: The word "deterministic" appears in both files when describing LLM-powered simulation — since responses are LLM-generated, this is misleading and should be softened to "controllable" (see inline comments).
• Code Examples: The "Integration with Experiments" example is missing from pydantic import BaseModel, Field imports (see inline comment on tool_simulation.mdx L187). Other code examples are complete and well-structured.
• Structure & Consistency: The page follows the same section pattern as user_simulation.mdx (Overview → Key Features → Basic Usage → Integration → Best Practices → Troubleshooting → Related Docs). The comparison table in index.mdx is a nice touch.
• Links: All internal links (index.md, user_simulation.md, quickstart.md, goal_success_rate_evaluator.md) reference files that exist in the repo.

Nice comprehensive documentation with the shared state and troubleshooting sections — those will save users a lot of debugging time.

[ybdarrenwang]

There are a couple of useful tips for tool simulator that we're about to share in an upcoming post:

1. Running independent simulator instances

You can create multiple ToolSimulator instances side by side. Each instance maintains its own tool registry and state, so you can run parallel experiment configurations in the same codebase:

simulator_a = ToolSimulator()
simulator_b = ToolSimulator()
# Each instance has an independent tool registry and state --
# ideal for comparing agent behavior across different tool setups.

2. Seeding state from real data

Because initial_state_description accepts natural language, you can get creative with how you seed context. For tools that interact with tabular data, use a DataFrame.describe() call to generate statistical summaries and pass those statistics directly as the state description. ToolSimulator will generate responses that reflect realistic data distributions, without ever accessing the actual data.

Might worth adding them here as well?

[github-actions]

Documentation Preview Failed

The documentation deployment encountered an error. Please check the deployment logs for more details.

[github-actions]

Documentation Preview Failed

The documentation deployment encountered an error. Please check the deployment logs for more details.

[github-actions]

Issue: Typo — "retrived" should be "retrieved".

Suggestion: ...and the tool can be retrieved and passed to a Strands agent.

[github-actions]

Issue: This code block reads as a self-contained example (it starts with a fresh tool_simulator = ToolSimulator()), but is missing imports for ToolSimulator and Agent (used on lines 100 and 137 respectively). The other self-contained blocks on this page (e.g., "Registering a Tool" at line 51, "Integration with Experiments" at line 158) all include full imports.

Suggestion: Add the missing imports at the top of the block:

from strands import Agent
from strands_evals.simulation.tool_simulator import ToolSimulator
from enum import Enum
from pydantic import BaseModel, Field

[github-actions]

Assessment: Comment

The docs look good — previous review feedback (the "deterministic" wording, the missing pydantic import, and ybdarrenwang's two suggestions) has all been addressed. Two small remaining items flagged inline.

Review Details

• Typo: "retrived" → "retrieved" on line 49
• Missing imports: The Shared State code block (line 96) is self-contained but omits ToolSimulator and Agent imports — inconsistent with other self-contained blocks on the page that include full imports
• Structure: Clean and well-organized; follows the same patterns as user_simulation.mdx and aligns with the evaluator pages
• Links: All internal links verified to exist
• Previous feedback: Fully addressed