Can We "Train" MCP Tools and Agent Skills Through Expert Demonstrations and Iterative Feedback?

[mianmianlingqi]

Hi everyone!

I'm an independent AI systems developer, and I've been exploring an idea I'd love to discuss with people working on DSPy, agent optimization, and self-improving AI systems.

What if we treated the development of MCP tools and Agent Skills as something closer to a training process, rather than a one-time engineering task?

Background

I've built an MCP-based system for generating educational electrical circuit diagrams, with a particular focus on strict Chinese educational conventions.

One of the biggest challenges I've encountered is that a diagram can be visually appealing and electrically correct, yet still be unsuitable for education if it doesn't follow the conventions students are expected to understand.

This has led me to explore a different approach to agent development.

Instead of relying entirely on manually designed prompts, tools, and workflows, I'm interested in whether an agent could progressively learn better engineering strategies from high-quality human demonstrations.

In my case, these demonstrations are primarily PNG/JPG images of circuit diagrams created by human experts, without access to their original drawing processes.

The Idea: Training-Like Optimization for MCPs and Skills

I'm imagining a system that could:

1. Analyze expert demonstrations and identify reusable design principles.
2. Convert those principles into executable rules, Agent Skills, or harness configurations.
3. Evaluate the resulting agent behavior against domain-specific requirements.
4. Use structured feedback to propose improvements to its instructions, tool-use policies, and validation workflows.
5. Retain improvements that generalize to unseen tasks.

The underlying foundation model would remain fixed.

Rather than training the model's weights, we would be optimizing the engineering system surrounding the model.

Where DSPy Might Fit

I'm particularly interested in whether DSPy's modular programming approach and optimizers such as GEPA could help implement part of this process.

A few questions I'd love to hear your thoughts on:

1. What would be the right abstraction for making Agent Skills and MCP-based workflows optimizable using DSPy?

2. How could we translate expert-produced visual artifacts into useful optimization feedback, especially when execution trajectories are unavailable?

3. Could this approach eventually support harness-level self-improvement, where the agent improves not only its task performance but also its mechanisms for learning from future demonstrations?

I'm interested in both practical engineering approaches and the broader research implications.

The core implementation of my current MCP system is proprietary, but I'm happy to share high-level architecture, examples, and evaluation ideas.

I'd especially appreciate relevant research references, alternative architectures, or critical perspectives on whether this approach makes sense.

Would love to brainstorm with anyone exploring similar ideas!

Thanks for reading!