Skip to content

Comments

Clarify shared workflow component errors and document 100% standalone compilation rate#8110

Merged
pelikhan merged 4 commits intomainfrom
copilot/audit-workflows-improve-validation
Dec 29, 2025
Merged

Clarify shared workflow component errors and document 100% standalone compilation rate#8110
pelikhan merged 4 commits intomainfrom
copilot/audit-workflows-improve-validation

Conversation

Copy link
Contributor

Copilot AI commented Dec 29, 2025

All 48 "failing" workflows are intentional reusable components in .github/workflows/shared/ that lack on: triggers by design. Actual standalone workflow compilation rate is 127/127 (100%).

Changes

Enhanced error messages (pkg/workflow/compiler_orchestrator.go):

  • Detect shared workflow components (files in shared/ directory missing on: trigger)
  • Provide actionable guidance explaining they're meant to be imported, not compiled directly
  • Show example import syntax for standalone workflows

Documented workflow architecture (DEVGUIDE.md):

  • Added "Workflow Compilation" section distinguishing standalone vs shared workflows
  • Documented 100% compilation success rate for standalone workflows
  • Explained shared component pattern and import mechanism

Example

Before:

✗ missing property 'on'

After:

✗ missing property 'on'

ℹ️  This appears to be a shared workflow component in the 'shared/' directory.
Shared workflows are reusable components meant to be imported by other workflows using the 'imports:' field.
They should not be compiled directly.

To use this shared workflow:
1. Import it in another workflow with: imports: [.github/workflows/shared/genaiscript.md]
2. Compile the workflow that imports it instead

Audit Results

Category Count Status
Standalone workflows 127 100% compile ✓
Shared components 48 Intentionally uncompilable (by design)
Deprecated workflows 0 N/A

Existing validation error messages already provide excellent actionable guidance with examples for MCP configs, engine features, and filter conflicts.

Original prompt

This section details on the original issue you should resolve

<issue_title>[plan] Audit failing workflows and improve validation messages</issue_title>
<issue_description>## Objective

Investigate why 46 out of 172 workflows (26.7%) fail compilation and improve validation error messages to be more actionable.

Context

The current 73.3% compilation success rate suggests either intentional test fixtures, validation issues, or unclear error messaging. Improving this will directly impact developer iteration speed.

Approach

  1. Run compilation audit on all 172 workflows
  2. Categorize each failure into:
    • Intentional test fixtures
    • Deprecated/archived workflows
    • Validation errors needing better messages
    • Actual bugs
  3. Move intentional test fixtures to dedicated test directory or mark clearly
  4. Archive or remove deprecated workflows
  5. Identify top 5 most common validation error patterns
  6. Enhance error messages with "how to fix" guidance
  7. Document expected compilation success rate in DEVGUIDE.md

Files to Modify

  • .github/workflows/*.md (failing workflows - categorize and organize)
  • pkg/workflow/validation.go (enhance error messages)
  • pkg/workflow/compiler.go (improve error context)
  • DEVGUIDE.md (document expected success rate)

Example Error Enhancement

Before: invalid network configuration

After: invalid network configuration: 'allowed' must be a list of domains. Try: allowed: ['api.github.com']

Acceptance Criteria

  • All 46 failing workflows categorized with documented reasons
  • Intentional test fixtures moved to dedicated test directory or clearly marked
  • Deprecated workflows archived or removed
  • Top 5 validation error patterns identified
  • Error messages for top 5 patterns enhanced with actionable recovery guidance
  • Expected compilation success rate documented in DEVGUIDE.md
  • Compilation success rate improved (target: >90%)

AI generated by Planning Assistant for #7631
Related to #8056

AI generated by Plan Command for discussion #7631

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 3 commits December 29, 2025 17:57
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Add helpful error messages when compiling shared workflow components that lack 'on' triggers.
Shared workflows in .github/workflows/shared/ are reusable components meant to be imported,
not compiled directly. Updated DEVGUIDE.md to document workflow types and expected compilation rates.

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Copilot AI changed the title [WIP] Improve validation messages and audit failing workflows Clarify shared workflow component errors and document 100% standalone compilation rate Dec 29, 2025
Copilot AI requested a review from mnkiefer December 29, 2025 18:16
@pelikhan pelikhan marked this pull request as ready for review December 29, 2025 20:46
@pelikhan pelikhan merged commit 1f54462 into main Dec 29, 2025
4 checks passed
@pelikhan pelikhan deleted the copilot/audit-workflows-improve-validation branch December 29, 2025 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[plan] Audit failing workflows and improve validation messages

3 participants