diff --git a/docs/toolhive/guides-ui/playground.mdx b/docs/toolhive/guides-ui/playground.mdx new file mode 100644 index 00000000..19b1ddc9 --- /dev/null +++ b/docs/toolhive/guides-ui/playground.mdx @@ -0,0 +1,239 @@ +--- +title: Test MCP servers with playground +description: + Use the playground feature to test and validate MCP servers directly in the + ToolHive UI with AI model providers. +--- + +import useBaseUrl from '@docusaurus/useBaseUrl'; +import ThemedImage from '@theme/ThemedImage'; + +ToolHive's playground lets you test and validate MCP servers directly in the UI +without requiring additional client setup. This streamlined testing environment +helps you quickly evaluate functionality and behavior before deploying MCP +servers to production environments. + +## Key capabilities + +### Instant testing of MCP servers + +Enter an AI model API key, select your MCP servers and tools, and begin testing +immediately in the desktop app. The playground eliminates the friction of +setting up external AI clients just to validate that your MCP servers work +correctly. + +### Detailed interaction logs + +See tool details, parameters, and execution results directly in the UI, ensuring +full visibility into tool performance and responses. Every interaction is +logged, making it easy to understand exactly what your MCP servers are doing and +how they respond to requests. + +### Integrated ToolHive management + +The playground includes a built-in MCP server that lets you manage your other +MCP servers directly through natural language commands. You can list servers, +check their status, start or stop them, and perform other management tasks using +conversational AI. + +## Getting started + +To start using the playground: + +1. **Access the playground**: Click the **Playground** tab in the ToolHive UI + navigation bar. + +2. **Configure API keys**: Click **Configure your API Keys** to set up access to + AI model providers: + - **OpenAI**: Enter your OpenAI API key to use GPT models + - **Anthropic**: Add your Anthropic API key for Claude models + - **Google**: Configure Google AI API key for Gemini models + - **xAI**: Set up xAI API key for Grok models + - **OpenRouter**: Add OpenRouter API key for access to multiple model + providers + +3. **Select MCP tools**: Click the tools icon to manage which MCP servers and + tools are available in the playground. + + + +- View all your running MCP servers +- Enable or disable specific tools from each server +- Search and filter tools by name or functionality +- The `toolhive mcp` server is included by default, providing management + capabilities + +4. **Start testing**: Begin chatting with your chosen AI model. The model will + have access to all enabled MCP tools and can execute them based on your + requests. + +## Using the playground + +### Testing MCP server functionality + +Use the playground to validate that your MCP servers work as expected: + +```text +Can you list all my MCP servers and show their current status? +``` + +The AI will use the `list_servers` tool from the ToolHive MCP server to provide +a comprehensive overview of your server status. + + + +Or test that an individual MCP tool is working as expected: + +```text +Use the GitHub MCP server to search for recent issues in the microsoft/vscode repository +``` + +If you have the GitHub MCP server running, the AI will execute the appropriate +GitHub API calls and return formatted results. + +### Managing servers through conversation + +The ToolHive desktop app automatically starts a dedicated MCP server +(`toolhive mcp`) that orchestrates ToolHive operations through natural language +commands. This approach provides several key benefits: + +- **Unified interface**: Manage your MCP infrastructure using the same + conversational AI interface you use for testing +- **Contextual operations**: The AI understands your current server state and + can make intelligent decisions about which servers to start, stop, or + troubleshoot +- **Reduced complexity**: No need to switch between the chat interface and + traditional UI controls—everything can be done through conversation +- **Audit trail**: All management operations are logged in the same transparent + way as tool executions, providing clear visibility into what actions were + taken + +Take advantage of these integrated ToolHive management tools: + +```text +Start the fetch MCP server for me +``` + +```text +Stop all unhealthy MCP servers +``` + +```text +Show me the logs for the meta-mcp server +``` + +### Validating tool responses + +The playground shows detailed information about each tool execution: + +- **Tool name and description**: What tool was called and its purpose +- **Input parameters**: The exact parameters passed to the tool +- **Execution status**: Whether the tool succeeded or failed +- **Response data**: The complete response from the tool +- **Timing information**: How long the tool took to execute + +This visibility helps you understand exactly how your MCP servers are behaving +and identify any issues with tool implementation or configuration. + +## Recommended practices + +### API key security + +- Use dedicated API keys for testing that have appropriate rate limits +- Regularly rotate API keys used in development environments +- Consider using API keys with restricted permissions for testing purposes + +### Server management + +- Start only the MCP servers you need for testing to improve performance +- Use the playground to validate new server configurations before connecting + them to production AI clients +- Test different combinations of tools to understand how they work together + +### Testing workflow + +1. **Isolated testing**: Test individual MCP servers one at a time to validate + their functionality +2. **Integration testing**: Enable multiple servers to test how they work + together +3. **Performance validation**: Monitor tool execution times and responses under + different loads +4. **Error handling**: Intentionally trigger error conditions to validate proper + error handling + +## Troubleshooting + +### Common issues + +
+API key not working + +If your API key isn't working: + +1. Verify the API key is correct and has proper permissions +2. Check that you have sufficient quota or credits with the provider +3. Ensure the API key hasn't expired or been revoked +4. Try creating a new API key from the provider's dashboard + +
+ +
+MCP tools not appearing + +If your MCP server tools aren't showing up: + +1. Verify the MCP server is running (check the **MCP Servers** page) +2. Click the tools icon and ensure the server's tools are enabled +3. Try restarting the MCP server if it shows as unhealthy +4. Check the server logs for any error messages + +
+ +
+Tool execution failing + +If tools are failing to execute: + +1. Check the tool's parameter requirements in the audit log +2. Verify any required secrets or environment variables are configured +3. Ensure the MCP server has necessary permissions (network access, file system + access) +4. Review the server logs for detailed error information + +
+ +## Next steps + +- Learn about [client configuration](./client-configuration.mdx) to connect + ToolHive to external AI applications +- Set up [secrets management](./secrets-management.md) for secure handling of + API keys and tokens +- Explore [network isolation](./network-isolation.mdx) for enhanced security + when testing untrusted MCP servers +- Browse the [registry](./registry.mdx) to discover new MCP servers to test in + the playground + +## Related information + +- [Run MCP servers](./run-mcp-servers.mdx) +- [Install ToolHive](./install.mdx) diff --git a/docs/toolhive/guides-ui/run-mcp-servers.mdx b/docs/toolhive/guides-ui/run-mcp-servers.mdx index efdd5898..e5c0561d 100644 --- a/docs/toolhive/guides-ui/run-mcp-servers.mdx +++ b/docs/toolhive/guides-ui/run-mcp-servers.mdx @@ -453,3 +453,5 @@ automatically the next time you launch the application. [client configuration guide](./client-configuration.mdx). - Learn more about [secrets management](./secrets-management.md) to securely manage API tokens and other sensitive data. +- Test your MCP servers using the [playground](./playground.mdx) to validate + functionality and behavior with different models. diff --git a/sidebars.ts b/sidebars.ts index e626ce54..672d1ed2 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -52,6 +52,7 @@ const sidebars: SidebarsConfig = { 'toolhive/guides-ui/network-isolation', 'toolhive/guides-ui/secrets-management', 'toolhive/guides-ui/client-configuration', + 'toolhive/guides-ui/playground', ], }, diff --git a/static/img/toolhive/toolhive-ui-playground-chat-response-dark.webp b/static/img/toolhive/toolhive-ui-playground-chat-response-dark.webp new file mode 100644 index 00000000..56529af0 Binary files /dev/null and b/static/img/toolhive/toolhive-ui-playground-chat-response-dark.webp differ diff --git a/static/img/toolhive/toolhive-ui-playground-chat-response-light.webp b/static/img/toolhive/toolhive-ui-playground-chat-response-light.webp new file mode 100644 index 00000000..699553b5 Binary files /dev/null and b/static/img/toolhive/toolhive-ui-playground-chat-response-light.webp differ diff --git a/static/img/toolhive/toolhive-ui-playground-thv-mcp-dark.webp b/static/img/toolhive/toolhive-ui-playground-thv-mcp-dark.webp new file mode 100644 index 00000000..e275dacd Binary files /dev/null and b/static/img/toolhive/toolhive-ui-playground-thv-mcp-dark.webp differ diff --git a/static/img/toolhive/toolhive-ui-playground-thv-mcp-light.webp b/static/img/toolhive/toolhive-ui-playground-thv-mcp-light.webp new file mode 100644 index 00000000..a9e550c0 Binary files /dev/null and b/static/img/toolhive/toolhive-ui-playground-thv-mcp-light.webp differ