Skip to content

MCP guard + engine fix: enforce category-scoped operator listing, fix… - #79

Merged
HeRunming merged 1 commit into
OpenDCAI:skills-agentfrom
HeRunming:skills-agent-emnlp
Jun 2, 2026
Merged

MCP guard + engine fix: enforce category-scoped operator listing, fix…#79
HeRunming merged 1 commit into
OpenDCAI:skills-agentfrom
HeRunming:skills-agent-emnlp

Conversation

@HeRunming

Copy link
Copy Markdown
Contributor

… run/init param plumbing

Backend

  • list_operators (operation_id="list_operators", route /by_category) now REQUIRES a category argument; missing/invalid arg returns a structured ValidationBizError with valid_categories + did_you_mean (difflib) + a three-step recovery hint, instead of returning the full ~145-operator catalog and overflowing the agent context.
  • list_operator_categories now returns {count, use_for, not_for, examples} per category (new app/services/operator_category_guide.py), distilled from observed agent failure modes (e.g. text_sft.not_for explicitly routes QA-generation to core_text). Methods without any construction skill inherit operator-selection guidance directly from this response.
  • Frontend operator palette keeps unrestricted access via a new list_operators_all (route /, NOT MCP-whitelisted).
  • dataflow_engine: drop run/init params the operator's signature can't accept (e.g. agent-supplied system_prompt on operators that don't take it) instead of crashing with TypeError; unwrap llm_serving={"id": ...} dict to its scalar id (was crashing with unhashable-type).

Skills

  • Promote .claude/skills/{dataflow-dev,dataflow-operator-builder, generating-dataflow-pipeline,prompt-template-builder} from the stale "* copy/" siblings (renames, not duplicates).
  • .gitignore now excludes .claude/skills/* copy/ to prevent recurrence.

Evaluation impact (12 tasks x 3 reps x 4 agent configs, see emnlp paper):

  • Mean operator errors: 1.87 -> 0.24 across the four agents (7.8x).
  • Context-overflow failure rate: 5.7-21.7% -> 0% on all four conditions.
  • End-to-end execution success: best agent rises from ~16% to 47.2%.

… run/init param plumbing

Backend
- list_operators (operation_id="list_operators", route /by_category) now
  REQUIRES a `category` argument; missing/invalid arg returns a structured
  ValidationBizError with valid_categories + did_you_mean (difflib) + a
  three-step recovery hint, instead of returning the full ~145-operator
  catalog and overflowing the agent context.
- list_operator_categories now returns {count, use_for, not_for, examples}
  per category (new app/services/operator_category_guide.py), distilled
  from observed agent failure modes (e.g. text_sft.not_for explicitly
  routes QA-generation to core_text). Methods without any construction
  skill inherit operator-selection guidance directly from this response.
- Frontend operator palette keeps unrestricted access via a new
  list_operators_all (route /, NOT MCP-whitelisted).
- dataflow_engine: drop run/init params the operator's signature can't
  accept (e.g. agent-supplied system_prompt on operators that don't take
  it) instead of crashing with TypeError; unwrap llm_serving={"id": ...}
  dict to its scalar id (was crashing with unhashable-type).

Skills
- Promote .claude/skills/{dataflow-dev,dataflow-operator-builder,
  generating-dataflow-pipeline,prompt-template-builder} from the stale
  "* copy/" siblings (renames, not duplicates).
- .gitignore now excludes .claude/skills/* copy/ to prevent recurrence.

Evaluation impact (12 tasks x 3 reps x 4 agent configs, see emnlp paper):
- Mean operator errors: 1.87 -> 0.24 across the four agents (7.8x).
- Context-overflow failure rate: 5.7-21.7% -> 0% on all four conditions.
- End-to-end execution success: best agent rises from ~16% to 47.2%.
@HeRunming
HeRunming merged commit 7680549 into OpenDCAI:skills-agent Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant