Skip to content

fix(skills): enforce capabilities.skills in conductor validate - #351

Merged
jrob5756 merged 1 commit into
mainfrom
fix/skills-capability-validation
Jul 30, 2026
Merged

fix(skills): enforce capabilities.skills in conductor validate#351
jrob5756 merged 1 commit into
mainfrom
fix/skills-capability-validation

Conversation

@jrob5756

Copy link
Copy Markdown
Collaborator

Follow-up to #215 (merged). Closes the validator gap flagged in review there.

Problem

ProviderCapabilities.skills documents this contract:

Workflows that declare runtime.skills or per-agent skills: against a provider with skills=False fail validation.

That check was never implemented. Every other capability has a cross-check in config/validator.pymcp_tools, workflow_tools_passthrough, reasoning_effort, max_session_seconds, working_dir, concurrent_safe — but skills had none.

Verified on main before this change, using the aca provider (which declares skills=False because skill directories are host paths its in-sandbox runner cannot read):

workflow:
  runtime:
    provider: { name: aca, pool_endpoint: https://example.../sessions }
    skills: [conductor]
agents:
  - name: worker
    skills: [conductor]
╭─── Validation Successful ───╮

It passed, then silently dropped the skill content at run time — precisely the regression the descriptor exists to prevent. The blast radius is wider than aca: any third-party or future provider that omits skills=True inherits the safe-looking False default and hits the same silent drop.

After this change:

- Agent 'worker' declares skills=['conductor'] but provider 'aca' does not
  support skills (capabilities.skills=False). Remove the skills, opt out with
  'skills: []', or override the agent to a skill-aware provider.

Changes

  • Per-agent check in _check_agent_capabilities. This covers top-level agents and for_each inline agents, since the inline path already routes through the same helper (Extend per-agent capability validation to for_each inline agents #270) — no separate for-each branch needed. An empty list is an explicit opt-out, so only a non-empty skills: errors.
  • Workflow-level check over all_llm_agents for inherited runtime.skills, skipping any agent whose own skills: (including []) overrides it. Mirrors the existing max_session_seconds / working_dir inheritance checks.
  • AGENTS.md: record that skills is not an allowed experimental carve-out. A provider reaches skills=True either natively (supports_native_skills=True) or through AgentExecutor's eager preamble injection, which is provider-agnostic — so False is only correct when neither path can work.

Tests

11 new tests in tests/test_config/test_validator_capabilities.py:

  • per-agent skills: against a skills=False provider → error
  • inherited runtime.skills against a skills=False provider → error
  • skills: [] opt-out → no error, both standalone and overriding a runtime default
  • per-agent provider override in both directions (to and from a skill-blind provider)
  • for_each inline agent, both explicit and inherited
  • cross-checks against the real AcaRuntimeProvider descriptor, so a future accidental widening of its capabilities is caught here rather than passing silently

_caps() gains skills: True in its defaults so the helper keeps describing a fully-capable stable provider.

Full suite: 4598 passed, 27 skipped. make check clean.

Related

The ProviderCapabilities.skills docstring promised that "workflows that
declare runtime.skills or per-agent skills: against a provider with
skills=False fail validation", but config/validator.py never checked the
flag. Every other capability has a cross-check; skills did not.

Verified before the fix: a workflow setting both runtime.skills and a
per-agent skills: on the aca provider (which declares skills=False)
validated cleanly, then silently dropped the skill content at run time —
exactly the regression the descriptor exists to prevent. Any third-party
provider omitting skills=True inherits the safe-looking False default and
hits the same silent drop.

- Per-agent check in _check_agent_capabilities, so top-level agents and
  for_each inline agents are both covered (the inline path already routes
  through this helper). An empty list is an explicit opt-out, so only a
  non-empty skills list errors.
- Workflow-level check over all_llm_agents for inherited runtime.skills,
  skipping agents whose own skills: (including []) overrides it.
- AGENTS.md: state that skills is not an allowed experimental carve-out,
  since eager preamble injection is provider-agnostic.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jrob5756
jrob5756 merged commit c8fad5c into main Jul 30, 2026
10 checks passed
@jrob5756
jrob5756 deleted the fix/skills-capability-validation branch July 30, 2026 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant