Independent project. This is not an official Microsoft repository or supported service — it is built on public Microsoft APIs under the MIT license. It is an evolving work in progress: an agentic companion tool for Copilot Studio makers and developers, provided as-is.
Agentic capabilities for Microsoft Copilot Studio maker evaluations. A remote MCP server puts test sets, graders, and evaluation runs in the hands of any agentic client; a skill teaches the model to drive them well; client plugins package both for specific hosts. Every call executes as the signed-in user via Entra On-Behalf-Of, so the platform's own permissions always apply.
The server (mcp/) is an independent product: a stateless remote MCP server that any client able to attach an Entra bearer token can use. The tools drive Copilot Studio's built-in agent evaluation capability through its official REST API, documented by Microsoft at learn.microsoft.com.
Hosted instance — fastest path. A ready-to-use instance runs at https://mcp.makereval.com/mcp. A tenant admin consents once; no infrastructure, app registration, or certificate needed. Walkthrough: mcp/docs/hosted-instance.md.
Self-host. Run the identical code in your own tenant for full control over identity and the data path — the natural fit for data-residency and compliance boundaries. Start with mcp/README.md.
As of mcp-v1.13.4, the public tool profile exposes 17 tools:
| Group | Tools |
|---|---|
| Identity and discovery | whoami, list_environments, list_agents, get_agent |
| Test sets | list_test_sets, get_test_set, create_test_set, update_test_set, delete_test_set |
| Test cases | add_test_cases, update_test_case, remove_test_case, add_multiturn_case |
| Runs and graders | start_evaluation_run, list_runs, get_run, list_graders |
Details and tool profiles: mcp/README.md.
client-plugins/ holds per-client integrations. Each packages the server connection together with a skill — the model-facing instructions for driving the evaluation tools well on that host.
The only plugin today, published as MakerEval.
Try it yourself first — no admin needed (if your organization allows custom plugin uploads): download makereval-hosted.zip from the latest cowork-v* release, then in Cowork: + → Customize → Plugins → Upload plugin → choose the zip → in the Share dialog keep "Only you" and Apply → start a new task. (Full tool functionality additionally needs the one-time tenant consent; until then the plugin installs and responds, but evaluation calls fail.)
Roll it out: upload the same package to your tenant catalog per the hosted-instance guide. To build against your own server, see client-plugins/cowork/README.md.
- MCP server — tagged
mcp-v*(current:mcp-v1.13.4) - Cowork plugin — tagged
cowork-v*(current:cowork-v0.19.6); the bundled skill is versioned in lockstep
Each tag has a GitHub Release; changes are recorded in CHANGELOG.md. Version-sensitive documentation states the release it describes.
MIT — see LICENSE. Contributions: CONTRIBUTING.md · Security: SECURITY.md.