Skip to content

PoC: Add executable-doc based agent skills with context-aware command/guidance responses#64029

Draft
KatalKavya96 wants to merge 3 commits intoapache:mainfrom
KatalKavya96:poc/agent-skills-decision-layer
Draft

PoC: Add executable-doc based agent skills with context-aware command/guidance responses#64029
KatalKavya96 wants to merge 3 commits intoapache:mainfrom
KatalKavya96:poc/agent-skills-decision-layer

Conversation

@KatalKavya96
Copy link
Copy Markdown
Contributor

Summary

This PR introduces a small PoC for modeling Airflow contributor workflows as structured, executable agent skills, with context-aware command selection.

The goal is to explore a narrow vertical slice of the problem described in #62500 , focusing on workflow semantics rather than broad coverage.

What is implemented

1. Structured workflow blocks (Executable-doc style)

Two workflows are defined in a markdown file using structured blocks:

  • run_targeted_tests

    • local-first (uv run --project ... pytest)
    • Breeze fallback (breeze run pytest) when system dependencies are missing
  • setup_breeze_environment

    • host-only workflow
    • returns guidance when invoked inside Breeze instead of suggesting incorrect commands

2. Extraction pipeline

A lightweight extractor converts structured blocks into machine-readable skill definitions:

Markdown → Extractor → skills.json

3. Context-aware decision engine

A simple decision engine consumes skills.json and returns the appropriate response based on execution context:

  • detects host vs breeze
  • applies local-first vs fallback logic
  • prevents invalid operations across contexts

4. Command vs Guidance response model

Instead of returning only commands, the PoC introduces a response contract:

{ "mode": "command", "command": "...", "reason": "..." }

{ "mode": "guidance", "message": "...", "next_action": "..." }

This allows modeling workflows where the correct action is not execution, but a context-aware transition.

Example:

  • running setup_breeze_environment inside Breeze returns guidance instead of suggesting breeze shell

5. Tests

Unit tests cover:

  • skill extraction
  • context detection
  • decision logic
  • command vs guidance behavior

Why this approach

This PoC explores whether:

agent-facing interfaces should return structured workflow-aware responses (command or guidance), rather than only raw commands.

This may better reflect real contributor workflows where:

  • some actions are executable
  • others require environment/context adjustments first

Scope

This is intentionally limited to:

  • 2 workflows
  • basic context detection
  • minimal extraction pipeline

It does not yet include:

  • full contributor-doc integration
  • drift detection / prek hooks
  • multiple workflow categories

Related: #62500

…n engine

- define structured workflow block for targeted test execution
- implement extractor (markdown → JSON)
- add context detection (host vs Breeze)
- implement decision engine for command selection
- connect decision engine to extracted skills (single source of truth)
- add unit tests for extraction, context detection, and decision logic
@boring-cyborg boring-cyborg bot added area:dev-tools backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch labels Mar 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant