From 48e91f59f5b9b247f3edd51f9d3f2f8a80bcc80f Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Tue, 14 Oct 2025 13:37:35 +0200 Subject: [PATCH 01/34] =?UTF-8?q?=F0=9F=9A=80[Feature]:=20Enhance=20workfl?= =?UTF-8?q?ow=20orchestration=20with=20detailed=20Get-Settings=20phase=20d?= =?UTF-8?q?escription=20and=20context-aware=20execution?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .specify/memory/constitution.md | 23 +++++++++++++-- README.md | 52 +++++++++++++++++++++++++++++---- 2 files changed, 67 insertions(+), 8 deletions(-) diff --git a/.specify/memory/constitution.md b/.specify/memory/constitution.md index 4bc59ab8..08993045 100644 --- a/.specify/memory/constitution.md +++ b/.specify/memory/constitution.md @@ -458,11 +458,23 @@ Pull requests MUST use labels to determine release behavior: ### Workflow Conditional Execution -The Process-PSModule workflow uses **dynamic conditions** to determine job execution: +The Process-PSModule workflow uses **dynamic conditions** to determine job execution, controlled by the **Get-Settings** phase. + +#### Get-Settings Phase: Workflow Orchestration + +The **Get-Settings** phase serves as the orchestration layer that: + +- **Collects Configuration**: Loads settings from `.github/PSModule.yml` (or JSON/PSD1) +- **Analyzes Context**: Examines GitHub event type, PR state, labels, and repository structure +- **Makes Smart Decisions**: Determines which workflow jobs should execute based on collected data +- **Prepares Test Matrices**: Generates dynamic test suite configurations for parallel execution +- **Optimizes Execution**: Skips unnecessary jobs to reduce CI/CD runtime and resource usage + +The Get-Settings phase embodies the principle of **context-aware workflow execution**, ensuring that each workflow run performs only the necessary tasks based on the current state and configuration. #### Always Execute (All PR States) -- **Get-Settings** - Configuration loading +- **Get-Settings** - Configuration loading and workflow orchestration - **Build-Module** - Module compilation - **Build-Docs** - Documentation generation - **Build-Site** - Static site generation @@ -667,7 +679,12 @@ jobs: ### Workflow Execution Order The standard execution order for Process-PSModule workflows MUST be: -1. **Get-Settings** - Reads configuration and prepares test matrices +1. **Gather** - Reads configuration, analyzes context, and prepares workflow execution plan + - Loads settings from `.github/PSModule.yml` (or JSON/PSD1) + - Examines GitHub event type, PR state, and labels + - Generates dynamic test matrices for parallel execution + - Determines which jobs should execute based on configuration and context + - Provides intelligent deployment orchestration for CI/CD workflows 2. **Build-Module** - Compiles source into module 3. **Test-SourceCode** - Parallel matrix testing of source code standards 4. **Lint-SourceCode** - Parallel matrix linting of source code diff --git a/README.md b/README.md index fa4d33d1..786e7468 100644 --- a/README.md +++ b/README.md @@ -24,10 +24,14 @@ Depending on the labels in the pull requests, the workflow will result in differ ![Process diagram](./media/Process-PSModule.png) -- [Get settings](./.github/workflows/Get-Settings.yml) - - Reads the settings file from a file in the module repository to configure the workflow. - - Gathers tests and creates test configuration based on the settings and the tests available in the module repository. - - This includes the selection of what OSes to run the tests on. +- [Get-Settings](./.github/workflows/Get-Settings.yml) + - **Workflow orchestration phase** that analyzes context and makes smart execution decisions + - **Collects Configuration**: Reads the settings file from `.github/PSModule.yml` (or JSON/PSD1) to configure the workflow + - **Analyzes Context**: Examines GitHub event type, PR state, labels, and repository structure + - **Makes Smart Decisions**: Determines which workflow jobs should execute based on collected data + - **Prepares Test Matrices**: Generates dynamic test suite configurations for parallel execution across OSes + - **Optimizes Execution**: Skips unnecessary jobs to reduce CI/CD runtime and resource usage + - This phase embodies context-aware workflow execution, ensuring each run performs only necessary tasks - [Build module](./.github/workflows/Build-Module.yml) - Compiles the module source code into a PowerShell module. - [Test source code](./.github/workflows/Test-SourceCode.yml) @@ -68,6 +72,44 @@ Depending on the labels in the pull requests, the workflow will result in differ - Publishes the module to the PowerShell Gallery. - Creates a release on the GitHub repository. +## Gather Phase: Workflow Orchestration + +The **Gather** phase is the cornerstone of Process-PSModule's workflow execution, providing context-aware orchestration for CI/CD pipelines. + +### Philosophy + +The Gather phase embodies the principle of **deployment orchestration**: + +- **Discovers the Environment**: Analyzes the GitHub context (event type, PR state, labels, branch) +- **Loads Configuration**: Reads settings from `.github/PSModule.yml` (or JSON/PSD1 format) +- **Makes Context-Aware Decisions**: Determines which jobs should execute based on collected data +- **Optimizes Resource Usage**: Skips unnecessary steps to reduce CI/CD runtime and costs +- **Prepares Execution Plan**: Generates dynamic test matrices for parallel execution across platforms + +### What Gather Does + +1. **Configuration Loading**: Imports settings with fallback to defaults +2. **Context Analysis**: Examines whether the workflow is running on: + - Open/Updated PR (build and test for validation) + - Merged PR (publish release and deploy site) + - Abandoned PR (cleanup only) + - Manual/Scheduled run (validation only) +3. **Test Matrix Generation**: Creates OS-specific test configurations based on: + - Available test files in the repository + - Skip flags in settings + - Test type requirements (SourceCode, PSModule, Module) +4. **Decision Output**: Produces structured outputs that control downstream job execution + +### Key Benefits + +- **Consistency**: Same logic across all consuming repositories +- **Efficiency**: Only runs necessary jobs for each scenario +- **Flexibility**: Easy to customize via settings file +- **Transparency**: Clear decision-making visible in workflow logs +- **Maintainability**: Centralized orchestration logic + +This approach ensures that each workflow run is optimized for its specific context, reducing unnecessary work while maintaining comprehensive validation when needed. + ## Usage To use the workflow, create a new file in the `.github/workflows` directory of the module repository and add the following content. @@ -202,7 +244,7 @@ This table shows when each job runs based on the trigger scenario: | Job | Open/Updated PR | Merged PR | Abandoned PR | Manual Run | |-----|-----------------|-----------|--------------|------------| -| **Get-Settings** | ✅ Always | ✅ Always | ✅ Always | ✅ Always | +| **Gather** | ✅ Always | ✅ Always | ✅ Always | ✅ Always | | **Lint-Repository** | ✅ Yes | ❌ No | ❌ No | ❌ No | | **Build-Module** | ✅ Yes | ✅ Yes | ❌ No | ✅ Yes | | **Build-Docs** | ✅ Yes | ✅ Yes | ❌ No | ✅ Yes | From 72425c4de539a6fd5fe25492e0021c10afde710d Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Tue, 14 Oct 2025 14:38:20 +0200 Subject: [PATCH 02/34] feat: Complete planning phase for settings-driven workflow configuration - Add comprehensive research findings on configuration patterns - Define data model for hierarchical configuration system - Create configuration schema contract with JSON validation - Develop quickstart guide for configuration setup - Update GitHub Copilot context with new technical details - Mark planning phases as complete in implementation plan This implements Phase 0 (research) and Phase 1 (design) of the settings-driven workflow configuration feature, establishing the foundation for centralized configuration management in Process-PSModule. --- .github/copilot-instructions.md | 23 +- .../contracts/configuration-schema.md | 326 +++++++++++++++ .../data-model.md | 200 ++++++++++ specs/001-settings-driven-workflow/plan.md | 301 ++++++++++++++ .../quickstart.md | 376 ++++++++++++++++++ .../001-settings-driven-workflow/research.md | 207 ++++++++++ specs/001-settings-driven-workflow/spec.md | 169 ++++++++ 7 files changed, 1591 insertions(+), 11 deletions(-) create mode 100644 specs/001-settings-driven-workflow/contracts/configuration-schema.md create mode 100644 specs/001-settings-driven-workflow/data-model.md create mode 100644 specs/001-settings-driven-workflow/plan.md create mode 100644 specs/001-settings-driven-workflow/quickstart.md create mode 100644 specs/001-settings-driven-workflow/research.md create mode 100644 specs/001-settings-driven-workflow/spec.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 858eab7b..588a1c48 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -2,17 +2,18 @@ ## Terminal Commands -When executing terminal commands (using `run_in_terminal` or similar tools): - - Prefer MCP server calls over command-line tools when possible. -- **ALWAYS** send commands into `pwsh -Command` to ensure proper execution. - - These commands must be enclosed in single quotes. - - Escape any single quotes within the command by doubling them (e.g., `It's` becomes `It''s`). - - Use double quotes for string with variables or expressions inside the single-quoted command. +- When running scripts within the [scripts](../.specify/scripts/) folder, just run them directly (shell is default PowerShell). +- For other commands, send them into `pwsh -Command` to ensure proper execution. + +### Quoting in PowerShell + +Proper quoting is essential in PowerShell to prevent parsing errors and ensure correct command execution. + +- **Direct script execution**: Scripts run directly in PowerShell use standard PowerShell quoting rules. Double quotes expand variables and expressions, while single quotes are literal. No additional shell escaping is needed. + +- **Via `pwsh -Command`**: Commands are passed as strings to PowerShell. Enclose the entire command in single quotes to treat it as a literal string. Escape single quotes within the command by doubling them (e.g., `It's` becomes `It''s`). Use double quotes within the command for variable expansion, but ensure the outer single quotes protect the string from shell interpretation. -## Other instructions +For arguments containing single quotes, prefer double-quoting the argument inside the command string. -| Tech | Instruction file | -|------|------------------| -| PowerShell | [pwsh.instructions.md](./instructions/pwsh.instructions.md) | -| Markdown | [md.instructions.md](./instructions/md.instructions.md) | +Example: `pwsh -Command 'Write-Host "I''m Groot"'` diff --git a/specs/001-settings-driven-workflow/contracts/configuration-schema.md b/specs/001-settings-driven-workflow/contracts/configuration-schema.md new file mode 100644 index 00000000..a5f80d6b --- /dev/null +++ b/specs/001-settings-driven-workflow/contracts/configuration-schema.md @@ -0,0 +1,326 @@ +# Configuration Schema Contract + +## Overview + +This contract defines the schema for Process-PSModule configuration files. The schema ensures type safety, validation, and consistent structure across all consuming repositories. + +## Schema Definition + +### Root Schema + +```json +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://raw.githubusercontent.com/PSModule/Process-PSModule/main/schemas/configuration.schema.json", + "title": "Process-PSModule Configuration", + "description": "Configuration schema for Process-PSModule workflows", + "type": "object", + "properties": { + "version": { + "type": "string", + "pattern": "^\\d+\\.\\d+\\.\\d+$", + "description": "Configuration schema version (semantic versioning)" + }, + "workflows": { + "type": "object", + "description": "Workflow-specific configurations", + "patternProperties": { + "^[a-zA-Z][a-zA-Z0-9_-]*$": { + "$ref": "#/definitions/WorkflowConfig" + } + } + }, + "environments": { + "type": "object", + "description": "Environment-specific overrides", + "patternProperties": { + "^[a-zA-Z][a-zA-Z0-9_-]*$": { + "$ref": "#/definitions/EnvironmentConfig" + } + } + }, + "defaults": { + "type": "object", + "description": "Global default values", + "properties": { + "timeout": { + "type": "integer", + "minimum": 1, + "maximum": 3600, + "description": "Default step timeout in minutes" + }, + "retries": { + "type": "integer", + "minimum": 0, + "maximum": 10, + "description": "Default retry count for failed steps" + }, + "concurrency": { + "type": "integer", + "minimum": 1, + "maximum": 100, + "description": "Default concurrency limit" + } + } + } + }, + "required": ["version"], + "additionalProperties": false, + "definitions": { + "WorkflowConfig": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean", + "description": "Whether this workflow is enabled", + "default": true + }, + "triggers": { + "type": "array", + "items": { + "type": "string", + "enum": ["push", "pull_request", "pull_request_target", "schedule", "workflow_dispatch", "release"] + }, + "description": "GitHub events that trigger this workflow" + }, + "matrix": { + "type": "object", + "description": "Test matrix configuration", + "properties": { + "os": { + "type": "array", + "items": { + "type": "string", + "enum": ["ubuntu-latest", "windows-latest", "macos-latest"] + } + }, + "powershell": { + "type": "array", + "items": { + "type": "string", + "pattern": "^7\\.[4-9]\\.\\d+$" + } + } + } + }, + "steps": { + "type": "array", + "items": { + "$ref": "#/definitions/StepConfig" + }, + "description": "Custom step configurations" + } + }, + "additionalProperties": false + }, + "EnvironmentConfig": { + "type": "object", + "properties": { + "variables": { + "type": "object", + "description": "Environment-specific variables", + "patternProperties": { + "^[A-Z][A-Z0-9_]*$": { + "type": "string" + } + } + }, + "secrets": { + "type": "object", + "description": "References to GitHub secrets", + "patternProperties": { + "^[A-Z][A-Z0-9_]*$": { + "type": "string" + } + } + }, + "matrix": { + "type": "object", + "description": "Environment-specific matrix overrides" + } + }, + "additionalProperties": false + }, + "StepConfig": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "Step identifier" + }, + "action": { + "type": "string", + "description": "GitHub Action reference (owner/repo@version)", + "pattern": "^[^@]+@[^@]+$" + }, + "inputs": { + "type": "object", + "description": "Action input parameters" + }, + "conditions": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Conditional execution expressions" + }, + "timeout": { + "type": "integer", + "minimum": 1, + "maximum": 3600, + "description": "Step timeout in minutes" + } + }, + "required": ["name", "action"], + "additionalProperties": false + } + } +} +``` + +## API Contract + +### Configuration Loading API + +**Endpoint:** `Get-Settings` Action + +**Input Parameters:** +- `config-file`: Path to configuration file (default: `.github/PSModule.yml`) +- `environment`: Target environment (default: auto-detected) +- `strict-validation`: Enable strict schema validation (default: true) + +**Output Parameters:** +- `config-json`: Complete configuration as JSON string +- `config-version`: Configuration schema version +- `environment-name`: Detected/active environment +- `validation-errors`: JSON array of validation errors (if any) + +**Environment Variables:** +All configuration values exported as `PSMODULE_CONFIG_*` prefixed variables + +### Configuration Validation API + +**Endpoint:** `Test-Configuration` Action + +**Input Parameters:** +- `config-file`: Path to configuration file +- `schema-file`: Path to JSON schema file +- `output-format`: Validation output format (json, text, junit) + +**Output Parameters:** +- `is-valid`: Boolean indicating configuration validity +- `error-count`: Number of validation errors +- `warning-count`: Number of validation warnings +- `validation-report`: Detailed validation report + +## Error Response Contract + +### Validation Error Format + +```json +{ + "errors": [ + { + "type": "schema", + "field": "workflows.build.steps[0].timeout", + "message": "Value must be between 1 and 3600", + "severity": "error", + "line": 45, + "column": 12 + } + ], + "warnings": [ + { + "type": "deprecated", + "field": "workflows.build.legacySetting", + "message": "This setting is deprecated and will be removed in v2.0.0", + "severity": "warning" + } + ], + "summary": { + "totalErrors": 1, + "totalWarnings": 1, + "isValid": false + } +} +``` + +### Runtime Error Format + +```json +{ + "error": { + "code": "CONFIG_LOAD_FAILED", + "message": "Failed to load configuration file", + "details": { + "file": ".github/PSModule.yml", + "reason": "File not found or inaccessible" + }, + "suggestions": [ + "Ensure .github/PSModule.yml exists in the repository", + "Check file permissions and repository access" + ] + } +} +``` + +## Version Compatibility + +### Schema Version Matrix + +| Schema Version | PSModule Version | Breaking Changes | Migration Guide | +|----------------|------------------|------------------|-----------------| +| 1.0.0 | v4.x | Initial release | N/A | +| 1.1.0 | v4.1+ | Added environment overrides | [Migration Guide](migration-1.0-to-1.1.md) | +| 2.0.0 | v5.x | Removed deprecated fields | [Migration Guide](migration-1.x-to-2.0.md) | + +### Backward Compatibility Rules + +- Minor version increments: Additive changes only +- Major version increments: Breaking changes allowed +- Deprecated fields: Warned for one major version before removal +- Migration tools: Provided for complex schema changes + +## Testing Contract + +### Configuration Test Scenarios + +**Valid Configurations:** +- Minimal configuration with required fields +- Full configuration with all optional fields +- Environment-specific overrides +- Matrix configurations for all platforms + +**Invalid Configurations:** +- Missing required fields +- Invalid data types +- Malformed YAML/JSON +- Invalid GitHub Actions references +- Circular references + +**Edge Cases:** +- Empty configuration files +- Very large configuration files (>1MB) +- Special characters in field names +- Unicode content +- Comments in YAML files + +### Validation Test Contract + +**Test Case Format:** +```yaml +test-case: + name: "Valid minimal configuration" + input: ".github/test-config.yml" + expected: + valid: true + errors: [] + warnings: [] + environment: "development" +``` + +**Coverage Requirements:** +- 100% schema field coverage +- All validation rules tested +- Error message accuracy +- Performance benchmarks met \ No newline at end of file diff --git a/specs/001-settings-driven-workflow/data-model.md b/specs/001-settings-driven-workflow/data-model.md new file mode 100644 index 00000000..b2d1a7cd --- /dev/null +++ b/specs/001-settings-driven-workflow/data-model.md @@ -0,0 +1,200 @@ +# Data Model: Settings-Driven Workflow Configuration + +## Overview + +The settings-driven workflow configuration system introduces a hierarchical configuration model that centralizes all Process-PSModule workflow settings into a single, authoritative source. This model supports environment-specific overrides and provides type-safe configuration validation. + +## Core Entities + +### Configuration File + +**Entity: ConfigurationFile** +- **Purpose:** Root container for all workflow settings +- **Format:** YAML (primary), JSON, or PowerShell Data (PSD1) +- **Location:** `.github/PSModule.yml` (or `.json`/`.psd1`) +- **Validation:** Schema-based validation with clear error messages + +**Properties:** +- `version`: String (semantic version of configuration schema) +- `workflows`: Object (workflow-specific settings) +- `environments`: Object (environment-specific overrides) +- `defaults`: Object (fallback values) + +### Workflow Configuration + +**Entity: WorkflowConfig** +- **Purpose:** Settings for a specific workflow execution +- **Scope:** Applied to individual workflow runs +- **Inheritance:** Can inherit from global defaults + +**Properties:** +- `name`: String (workflow identifier) +- `enabled`: Boolean (whether workflow is active) +- `triggers`: Array (GitHub event triggers) +- `matrix`: Object (test matrix configuration) +- `steps`: Array (workflow step settings) + +### Environment Override + +**Entity: EnvironmentOverride** +- **Purpose:** Environment-specific configuration values +- **Scope:** Applied based on deployment environment +- **Precedence:** Overrides base configuration values + +**Properties:** +- `name`: String (environment identifier: development, staging, production) +- `variables`: Object (environment-specific variables) +- `secrets`: Object (references to GitHub secrets) +- `matrix`: Object (environment-specific test matrix) + +### Step Configuration + +**Entity: StepConfig** +- **Purpose:** Configuration for individual workflow steps +- **Scope:** Applied to specific GitHub Actions steps +- **Flexibility:** Supports conditional execution and parameterization + +**Properties:** +- `name`: String (step identifier) +- `action`: String (GitHub Action reference) +- `inputs`: Object (action input parameters) +- `conditions`: Array (execution conditions) +- `timeout`: Number (step timeout in minutes) + +## Data Relationships + +### Configuration Hierarchy + +``` +ConfigurationFile (root) +├── defaults (global fallbacks) +├── workflows[] (workflow-specific settings) +│ └── steps[] (step configurations) +└── environments[] (environment overrides) + ├── variables (env-specific values) + ├── secrets (secure references) + └── matrix (env-specific testing) +``` + +### Inheritance Rules + +1. **Base Configuration:** Default values from `defaults` section +2. **Workflow Override:** Workflow-specific settings override defaults +3. **Environment Override:** Environment settings override workflow settings +4. **Runtime Override:** Command-line or trigger inputs override all above + +### Validation Rules + +**Required Fields:** +- `version`: Must be valid semantic version +- At least one workflow configuration +- Valid GitHub Actions syntax in step configurations + +**Type Constraints:** +- `enabled`: Must be boolean +- `timeout`: Must be positive integer +- `triggers`: Must be valid GitHub event names +- `matrix`: Must follow GitHub Actions matrix syntax + +**Cross-Reference Validation:** +- Referenced actions must exist in marketplace or repository +- Secret references must exist in GitHub Secrets +- Environment names must be valid deployment targets + +## Data Flow + +### Configuration Loading Process + +1. **Discovery:** Locate configuration file in `.github/` directory +2. **Parsing:** Parse YAML/JSON/PSData into PowerShell objects +3. **Validation:** Apply schema validation and business rules +4. **Environment Selection:** Determine active environment based on context +5. **Merge:** Combine base config with environment overrides +6. **Export:** Make configuration available as environment variables + +### Runtime Data Flow + +``` +Repository Context → Configuration Loader → Validation → Environment Merge → Workflow Execution + ↓ ↓ ↓ ↓ ↓ + Branch/PR Info YAML Parser Schema Check Override Logic Step Inputs +``` + +## Error Handling + +### Configuration Errors + +**Invalid Syntax:** +- **Detection:** YAML/JSON parsing failures +- **Response:** Fail workflow with syntax error details +- **Recovery:** Suggest configuration fixes + +**Missing Required Fields:** +- **Detection:** Schema validation failures +- **Response:** List all missing required fields +- **Recovery:** Provide configuration template + +**Invalid References:** +- **Detection:** Cross-reference validation +- **Response:** Identify invalid action/secret references +- **Recovery:** Suggest valid alternatives + +### Runtime Errors + +**Environment Variable Conflicts:** +- **Detection:** Duplicate environment variable names +- **Response:** Warn and use last-defined value +- **Recovery:** Document override behavior + +**Type Conversion Failures:** +- **Detection:** Invalid type conversions +- **Response:** Fail with type error details +- **Recovery:** Validate configuration types + +## Security Model + +### Data Protection + +**Sensitive Data Handling:** +- Configuration files: Public, no secrets allowed +- Secrets: Referenced via GitHub Secrets mechanism +- Logging: Sensitive values masked in output + +**Access Control:** +- Configuration files: Readable by all repository collaborators +- Secrets: Controlled by repository permissions +- Validation: Runs in unprivileged workflow context + +### Audit Trail + +**Change Tracking:** +- Configuration changes tracked in git history +- Validation results logged in workflow runs +- Override applications documented in debug logs + +## Migration Path + +### Backward Compatibility + +**Legacy Support:** +- Workflows without configuration files continue to work +- Default values provided for all optional settings +- Gradual migration with feature flags + +**Migration Steps:** +1. Create configuration file with current hardcoded values +2. Replace hardcoded values with configuration references +3. Test configuration loading and validation +4. Remove legacy hardcoded values + +### Versioning Strategy + +**Configuration Schema Versions:** +- Semantic versioning for breaking changes +- Backward compatibility within major versions +- Migration guides for major version upgrades + +**Deprecation Policy:** +- Deprecated settings warned in validation +- Removal announced in release notes +- Migration tools provided for complex changes \ No newline at end of file diff --git a/specs/001-settings-driven-workflow/plan.md b/specs/001-settings-driven-workflow/plan.md new file mode 100644 index 00000000..1e360e76 --- /dev/null +++ b/specs/001-settings-driven-workflow/plan.md @@ -0,0 +1,301 @@ +# Implementation Plan: Settings-Driven Workflow Configuration + +**Branch**: `001-settings-driven-workflow` | **Date**: October 14, 2025 | **Spec**: [spec.md](spec.md) +**Input**: Feature specification from `/specs/001-settings-driven-workflow/spec.md` + +## Execution Flow (/plan command scope) + +1. Load feature spec from Input path + → If not found: ERROR "No feature spec at {path}" +2. Fill Technical Context (scan for NEEDS CLARIFICATION) + → Detect Project Type from file system structure or context (web=frontend+backend, mobile=app+API) + → Set Structure Decision based on project type +3. Fill the Constitution Check section based on the content of the constitution document. +4. Evaluate Constitution Check section below + → If violations exist: Document them in Complexity Tracking + → If no justification is possible: ERROR "Simplify approach first" + → Update Progress Tracking: Initial Constitution Check +5. Execute Phase 0 → research.md + → If NEEDS CLARIFICATION remain: ERROR "Resolve unknowns" +6. Execute Phase 1 → contracts, data-model.md, quickstart.md, agent-specific template file (e.g., `CLAUDE.md` for Claude Code, `.github/copilot-instructions.md` for GitHub Copilot, `GEMINI.md` for Gemini CLI, `QWEN.md` for Qwen Code or `AGENTS.md` for opencode). +7. Re-evaluate Constitution Check section + → If new violations: Refactor design, return to Phase 1 + → Update Progress Tracking: Post-Design Constitution Check +8. Plan Phase 2 → Describe task generation approach (DO NOT create tasks.md) +9. STOP - Ready for /tasks command + +**IMPORTANT**: The /plan command STOPS at step 7. Phases 2-4 are executed by other commands: + +- Phase 2: /tasks command creates tasks.md +- Phase 3-4: Implementation execution (manual or via tools) + +## Summary + +Implement a centralized configuration system for Process-PSModule workflows that eliminates scattered hardcoded values and reduces the complexity of conditional if: statements by at least 50%. The system will provide a single authoritative configuration file that makes settings available throughout the workflow via environment variables or step inputs, supporting hierarchical configuration with environment-specific overrides. + +## Technical Context + +| Aspect | Details | +|--------|---------| +| **Language/Version** | PowerShell 7.4+ | +| **Primary Dependencies** | GitHub Actions, PSModule framework actions (Build-PSModule, Test-ModuleLocal, etc.) | +| **Storage** | Configuration files (YAML/JSON/PSData) in `.github/PSModule.yml` | +| **Testing** | Pester for workflow validation, PSScriptAnalyzer for code quality | +| **Target Platform** | GitHub Actions execution environment (Linux, macOS, Windows runners) | +| **Project Type** | Workflow framework (Process-PSModule) | +| **Performance Goals** | Configuration loading within 10 seconds, workflow execution efficiency | +| **Constraints** | Must work within GitHub Actions syntax, support YAML/JSON/PSData formats, maintain backward compatibility | +| **Scale/Scope** | Single repository configuration, extensible to multiple consuming repositories | + +## Constitution Check + +*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.* + +### I. Workflow-First Design (NON-NEGOTIABLE) + +- [x] Feature is implemented as reusable GitHub Actions workflow(s) +- [x] Workflows have clearly defined inputs and outputs +- [x] Workflows follow single responsibility principle +- [x] Matrix strategies used for parallel execution where appropriate +- [x] Workflows are independently testable via CI validation workflow +- [x] Logic delegated to reusable GitHub Actions (PSModule organization) +- [x] Inline PowerShell code avoided; action-based scripts used instead +- [x] Actions referenced by specific versions/tags + +### II. Test-Driven Development (NON-NEGOTIABLE) + +- [x] Tests will be written before implementation +- [x] Initial tests will fail (Red phase documented) +- [x] Implementation plan includes making tests pass (Green phase) +- [x] Refactoring phase planned while maintaining tests +- [x] PSScriptAnalyzer validation included +- [x] Manual testing documented if needed +- [x] CI validation workflow tests included + +### III. Platform Independence with Modern PowerShell + +- [x] PowerShell 7.4+ constructs used exclusively +- [x] Matrix testing across Linux, macOS, Windows included +- [x] Platform-specific behaviors documented +- [x] Skip mechanisms justified if platform-specific tests needed +- [x] No backward compatibility with PowerShell 5.1 required + +### IV. Quality Gates and Observability + +- [x] Test results captured in structured JSON format +- [x] Code coverage measurement included +- [x] Linting results captured and enforced +- [x] Quality gate thresholds defined +- [x] Clear error messages planned +- [x] Debug mode support included + +### V. Continuous Delivery with Semantic Versioning + +- [x] Version bump strategy documented (labels, SemVer) +- [x] Release automation compatible with existing workflow +- [x] Documentation updates included +- [x] GitHub Pages publishing considered if docs changes + +## Project Structure + +### Documentation (this feature) + +```plaintext +specs/001-settings-driven-workflow/ +├── plan.md # This file (/plan command output) +├── research.md # Phase 0 output (/plan command) +├── data-model.md # Phase 1 output (/plan command) +├── quickstart.md # Phase 1 output (/plan command) +├── contracts/ # Phase 1 output (/plan command) +└── tasks.md # Phase 2 output (/tasks command - NOT created by /plan) +``` + +### Source Code (Process-PSModule framework) + +This feature modifies the existing Process-PSModule framework structure by adding configuration management capabilities: + +```plaintext +.github/ +├── workflows/ +│ ├── workflow.yml # Main workflow (MODIFIED: add configuration loading) +│ └── ci.yml # CI validation (MODIFIED: add config validation) +└── PSModule.example.yml # Example configuration file (NEW) + +.specify/ +├── scripts/ +│ └── powershell/ +│ ├── Get-Settings.ps1 # NEW: Configuration loading action +│ └── Test-Configuration.ps1 # NEW: Configuration validation action +└── templates/ + └── PSModule.yml # NEW: Configuration template + +src/ # Framework actions (MODIFIED) +├── actions/ +│ ├── Get-Settings/ # NEW: Configuration management action +│ │ ├── action.yml +│ │ └── Get-Settings.ps1 +│ └── Test-Configuration/ # NEW: Configuration validation action +│ ├── action.yml +│ └── Test-Configuration.ps1 +└── workflows/ + ├── workflow.yml # MODIFIED: Integrate configuration loading + └── ci.yml # MODIFIED: Add configuration validation + +tests/ # Framework tests (MODIFIED) +├── Get-Settings.Tests.ps1 # NEW: Configuration loading tests +├── Test-Configuration.Tests.ps1 # NEW: Configuration validation tests +└── workflow-tests/ # MODIFIED: Update workflow integration tests +``` + +**Structure Decision**: This feature extends the existing Process-PSModule framework by adding centralized configuration management. The implementation follows the established framework patterns with new actions for configuration loading and validation, integrated into the main workflow. + +## Phase 0: Outline & Research + +1. **Extract unknowns from Technical Context** above: + - For each NEEDS CLARIFICATION → research task + - For each dependency → best practices task + - For each integration → patterns task +2. **Generate and dispatch research agents**: + ```plaintext + For each unknown in Technical Context: + Task: "Research {unknown} for {feature context}" + For each technology choice: + Task: "Find best practices for {tech} in {domain}" + ``` +3. **Consolidate findings** in `research.md` using format: + - Decision: [what was chosen] + - Rationale: [why chosen] + - Alternatives considered: [what else evaluated] + +**Output**: research.md with all NEEDS CLARIFICATION resolved + +## Phase 1: Design & Contracts + +*Prerequisites: research.md complete* + +1. **Extract entities from feature spec** → `data-model.md`: + - Entity name, fields, relationships + - Validation rules from requirements + - State transitions if applicable +2. **Generate API contracts** from functional requirements: + - For each user action → endpoint + - Use standard REST/GraphQL patterns + - Output OpenAPI/GraphQL schema to `/contracts/` +3. **Generate contract tests** from contracts: + - One test file per endpoint + - Assert request/response schemas + - Tests must fail (no implementation yet) +4. **Extract test scenarios** from user stories: + - Each story → integration test scenario + - Quickstart test = story validation steps +5. **Update agent file incrementally** (O(1) operation): + - Run `.specify/scripts/powershell/update-agent-context.ps1 -AgentType copilot` + **IMPORTANT**: Execute it exactly as specified above. Do not add or remove any arguments. + - If exists: Add only NEW tech from current plan + - Preserve manual additions between markers + - Update recent changes (keep last 3) + - Keep under 150 lines for token efficiency + - Output to repository root + +**Output**: data-model.md, /contracts/*, failing tests, quickstart.md, agent-specific file + +## Phase 2: Task Planning Approach + +*This section describes what the /tasks command will do - DO NOT execute during /plan* + +**Task Generation Strategy**: + +- Load `.specify/templates/tasks-template.md` as base +- Generate tasks from Phase 1 design docs (configuration schema, data model, quickstart) +- Each configuration entity → implementation task [P] +- Each API contract → test task [P] +- Each workflow integration → modification task +- Schema validation → dedicated task +- Documentation updates → separate tasks + +**Ordering Strategy**: + +- Foundation first: Configuration schema and validation +- Core implementation: Get-Settings and Test-Configuration actions +- Integration: Workflow modifications and testing +- Documentation: README and example updates +- Mark [P] for parallel execution (independent files/components) + +**Specific Task Categories**: + +1. **Configuration Schema Tasks** [Priority: High] + - Create JSON schema file for configuration validation + - Implement schema versioning and migration + - Add schema documentation and examples + +2. **Action Implementation Tasks** [Priority: High] + - Create Get-Settings composite action + - Create Test-Configuration composite action + - Implement YAML/JSON/PSData parsing + - Add environment variable export logic + +3. **Workflow Integration Tasks** [Priority: Medium] + - Modify main workflow.yml to use configuration + - Update CI workflow for configuration validation + - Add configuration loading to all relevant jobs + - Implement environment detection logic + +4. **Testing Tasks** [Priority: High] + - Create unit tests for configuration parsing + - Create integration tests for workflow execution + - Add schema validation tests + - Implement cross-platform testing + +5. **Documentation Tasks** [Priority: Low] + - Update README with configuration examples + - Create configuration template files + - Add troubleshooting guides + - Update action documentation + +**Estimated Output**: 20-25 numbered, ordered tasks in tasks.md focusing on TDD implementation of configuration management + +**IMPORTANT**: This phase is executed by the /tasks command, NOT by /plan + +**Estimated Output**: 25-30 numbered, ordered tasks in tasks.md + +**IMPORTANT**: This phase is executed by the /tasks command, NOT by /plan + +## Phase 3+: Future Implementation + +*These phases are beyond the scope of the /plan command* + +**Phase 3**: Task execution (/tasks command creates tasks.md) +**Phase 4**: Implementation (execute tasks.md following constitutional principles) +**Phase 5**: Validation (run tests, execute quickstart.md, performance validation) + +## Complexity Tracking + +*Fill ONLY if Constitution Check has violations that must be justified* + +| Violation | Why Needed | Simpler Alternative Rejected Because | +|-----------|------------|-------------------------------------| +| [e.g., 4th project] | [current need] | [why 3 projects insufficient] | +| [e.g., Repository pattern] | [specific problem] | [why direct DB access insufficient] | + +## Progress Tracking + +*This checklist is updated during execution flow* + +**Phase Status**: + +- [x] Phase 0: Research complete (/plan command) +- [x] Phase 1: Design complete (/plan command) +- [ ] Phase 2: Task planning complete (/plan command - describe approach only) +- [ ] Phase 3: Tasks generated (/tasks command) +- [ ] Phase 4: Implementation complete +- [ ] Phase 5: Validation passed + +**Gate Status**: + +- [x] Initial Constitution Check: PASS +- [x] Post-Design Constitution Check: PASS +- [x] All NEEDS CLARIFICATION resolved +- [ ] Complexity deviations documented + +--- +*Based on Constitution - See `.specify/memory/constitution.md`* diff --git a/specs/001-settings-driven-workflow/quickstart.md b/specs/001-settings-driven-workflow/quickstart.md new file mode 100644 index 00000000..d4912487 --- /dev/null +++ b/specs/001-settings-driven-workflow/quickstart.md @@ -0,0 +1,376 @@ +# Quickstart: Settings-Driven Workflow Configuration + +## Overview + +This guide shows how to configure Process-PSModule workflows using the new centralized configuration system. The settings-driven approach eliminates hardcoded values and reduces workflow complexity by at least 50%. + +## Prerequisites + +- Process-PSModule v4.1+ (with configuration support) +- Repository with PowerShell module structure +- GitHub repository with Actions enabled + +## Step 1: Create Configuration File + +Create a configuration file in your repository: + +```bash +# Create the .github directory if it doesn't exist +mkdir -p .github + +# Create the configuration file +touch .github/PSModule.yml +``` + +## Step 2: Basic Configuration + +Add a minimal configuration to `.github/PSModule.yml`: + +```yaml +version: "1.0.0" + +workflows: + build: + enabled: true + triggers: + - push + - pull_request + +environments: + development: + variables: + PSMODULE_ENV: "development" + matrix: + os: [ubuntu-latest] + + production: + variables: + PSMODULE_ENV: "production" + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + +defaults: + timeout: 30 + retries: 2 +``` + +## Step 3: Update Workflow File + +Modify your `.github/workflows/Process-PSModule.yml` to use configuration: + +```yaml +name: Process-PSModule + +on: + push: + branches: [main] + pull_request: + branches: [main] + types: [opened, synchronize, reopened, labeled] + +jobs: + Process-PSModule: + uses: PSModule/Process-PSModule/.github/workflows/workflow.yml@v4 + with: + config-file: .github/PSModule.yml + environment: development + secrets: + APIKEY: ${{ secrets.APIKEY }} +``` + +## Step 4: Advanced Configuration + +### Environment-Specific Settings + +```yaml +environments: + staging: + variables: + PSMODULE_LOG_LEVEL: "Debug" + PSMODULE_TEST_COVERAGE: "80" + secrets: + APIKEY: "STAGING_API_KEY" + matrix: + os: [ubuntu-latest, windows-latest] + + production: + variables: + PSMODULE_LOG_LEVEL: "Error" + PSMODULE_TEST_COVERAGE: "90" + secrets: + APIKEY: "PROD_API_KEY" + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + powershell: ["7.4.0"] +``` + +### Workflow-Specific Overrides + +```yaml +workflows: + ci: + enabled: true + triggers: + - push + - pull_request + matrix: + os: [ubuntu-latest] + powershell: ["7.4.0"] + steps: + - name: "lint" + action: "PSModule/PSScriptAnalyzer@latest" + inputs: + path: "src/" + conditions: + - "github.event_name == 'pull_request'" + + release: + enabled: true + triggers: + - release + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + powershell: ["7.4.0", "7.4.1"] +``` + +### Custom Step Configuration + +```yaml +workflows: + build: + steps: + - name: "build-module" + action: "PSModule/Build-PSModule@main" + inputs: + source-path: "src" + output-path: "output" + timeout: 10 + conditions: + - "always()" + + - name: "test-module" + action: "PSModule/Test-ModuleLocal@main" + inputs: + path: "output" + import-module: true + timeout: 20 +``` + +## Step 5: Migration from Hardcoded Values + +### Before (Hardcoded) + +```yaml +jobs: + test: + strategy: + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + powershell: ['7.4.0', '7.4.1'] + steps: + - uses: actions/checkout@v4 + - name: Test Module + shell: pwsh + run: | + $matrix = @( + @{os = 'ubuntu-latest'; pwsh = '7.4.0'} + @{os = 'windows-latest'; pwsh = '7.4.1'} + @{os = 'macos-latest'; pwsh = '7.4.2'} + ) + # Complex conditional logic here... +``` + +### After (Configuration-Driven) + +```yaml +jobs: + Process-PSModule: + uses: PSModule/Process-PSModule/.github/workflows/workflow.yml@v4 + with: + config-file: .github/PSModule.yml +``` + +With configuration in `.github/PSModule.yml`: + +```yaml +workflows: + test: + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + powershell: ['7.4.0', '7.4.1', '7.4.2'] +``` + +## Step 6: Validation and Testing + +### Validate Configuration + +```bash +# Use the Test-Configuration action +- name: Validate Configuration + uses: PSModule/Process-PSModule/.github/actions/Test-Configuration@main + with: + config-file: .github/PSModule.yml +``` + +### Test Workflow Execution + +1. Create a test branch +2. Push changes to trigger workflow +3. Verify configuration is loaded correctly +4. Check that environment variables are set +5. Validate matrix expansion works as expected + +## Step 7: Troubleshooting + +### Common Issues + +**Configuration file not found:** +``` +Error: Configuration file .github/PSModule.yml not found +``` +**Solution:** Ensure the file exists and is properly formatted YAML + +**Invalid configuration:** +``` +Error: Configuration validation failed: version is required +``` +**Solution:** Check the configuration against the schema and fix validation errors + +**Environment not found:** +``` +Warning: Environment 'production' not found, using defaults +``` +**Solution:** Add the missing environment configuration or use a valid environment name + +### Debug Mode + +Enable debug logging: + +```yaml +jobs: + Process-PSModule: + uses: PSModule/Process-PSModule/.github/workflows/workflow.yml@v4 + with: + config-file: .github/PSModule.yml + debug: true +``` + +## Step 8: Advanced Usage + +### Conditional Workflows + +```yaml +workflows: + nightly: + enabled: true + triggers: + - schedule + conditions: + - "github.event.schedule == '0 2 * * *'" + matrix: + os: [ubuntu-latest] + + pr-only: + enabled: true + triggers: + - pull_request + conditions: + - "github.event_name == 'pull_request'" + matrix: + os: [ubuntu-latest, windows-latest] +``` + +### Dynamic Configuration + +Use repository variables for dynamic values: + +```yaml +environments: + dynamic: + variables: + PSMODULE_VERSION: ${{ vars.MODULE_VERSION }} + PSMODULE_BRANCH: ${{ github.ref_name }} +``` + +### Secrets Management + +```yaml +environments: + production: + secrets: + API_KEY: "PROD_API_KEY" + DB_PASSWORD: "DATABASE_PASSWORD" +``` + +## Examples + +### Complete Configuration Example + +```yaml +version: "1.0.0" + +defaults: + timeout: 30 + retries: 2 + concurrency: 4 + +workflows: + ci: + enabled: true + triggers: + - push + - pull_request + matrix: + os: [ubuntu-latest] + powershell: ["7.4.0"] + + release: + enabled: true + triggers: + - release + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + powershell: ["7.4.0", "7.4.1"] + +environments: + development: + variables: + PSMODULE_ENV: "dev" + PSMODULE_DEBUG: "true" + + staging: + variables: + PSMODULE_ENV: "staging" + PSMODULE_DEBUG: "false" + secrets: + API_KEY: "STAGING_API_KEY" + + production: + variables: + PSMODULE_ENV: "prod" + PSMODULE_DEBUG: "false" + secrets: + API_KEY: "PROD_API_KEY" + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + powershell: ["7.4.0", "7.4.1", "7.4.2"] +``` + +## Next Steps + +1. **Monitor workflow execution** to ensure configuration is applied correctly +2. **Gradually migrate** remaining hardcoded values to configuration +3. **Add environment-specific settings** as needed +4. **Implement automated validation** in CI pipeline +5. **Document configuration options** for team members + +## Support + +- **Documentation:** [Process-PSModule Configuration Guide](https://psmodule.io/configuration) +- **Issues:** [GitHub Issues](https://github.com/PSModule/Process-PSModule/issues) +- **Discussions:** [GitHub Discussions](https://github.com/PSModule/Process-PSModule/discussions) + +--- + +**Version:** 1.0.0 +**Last Updated:** October 14, 2025 \ No newline at end of file diff --git a/specs/001-settings-driven-workflow/research.md b/specs/001-settings-driven-workflow/research.md new file mode 100644 index 00000000..79d4199b --- /dev/null +++ b/specs/001-settings-driven-workflow/research.md @@ -0,0 +1,207 @@ +# Research Findings: Settings-Driven Workflow Configuration + +## Current State Analysis + +### Existing Configuration in Process-PSModule + +**Current Configuration Mechanisms:** +- Repository-specific settings scattered across workflow files +- Hardcoded values in workflow YAML files +- Limited configuration options via workflow inputs +- No centralized configuration validation + +**Pain Points Identified:** +- Conditional logic complexity in workflow files +- Difficulty maintaining consistent settings across repositories +- Lack of configuration validation and error handling +- No support for environment-specific overrides + +### GitHub Actions Configuration Patterns + +**Common Patterns:** +- `workflow_dispatch` inputs for manual triggers +- Environment variables for secrets and runtime values +- Matrix strategies for platform testing +- Job outputs for inter-job communication + +**Limitations:** +- No built-in configuration file support +- Complex conditional expressions using `${{ }}` +- Limited validation of input values +- No hierarchical configuration support + +## Technical Research + +### Configuration Formats + +**YAML (Recommended):** +- Human-readable and writable +- Supports comments and complex data structures +- Widely used in GitHub Actions ecosystem +- Good tool support and validation + +**JSON:** +- Machine-readable, less human-friendly +- Strict syntax requirements +- Better for programmatic generation +- Limited commenting support + +**PowerShell Data (PSD1):** +- Native PowerShell format +- Supports complex objects and types +- Good for PowerShell-centric workflows +- Less familiar to non-PowerShell developers + +**Decision:** Use YAML as primary format with JSON/PSData support for compatibility + +### Configuration Loading Strategies + +**GitHub Actions Approaches:** +1. **Repository Variables/Secrets:** Limited scope, no file-based config +2. **Workflow Inputs:** Per-workflow, not reusable +3. **Environment Files:** `.env` files, security concerns +4. **External Configuration Files:** Repository-based YAML/JSON files + +**Recommended Approach:** +- Central configuration file (`.github/PSModule.yml`) +- Load via composite action with PowerShell script +- Validate syntax and required fields +- Support environment-specific overrides + +### Security Considerations + +**Security Requirements:** +- Configuration files should not contain secrets +- Sensitive values must use GitHub Secrets +- Configuration loading should be secure by default +- No exposure of sensitive data in logs + +**Implementation:** +- Separate secrets from configuration +- Use GitHub Secrets for sensitive values +- Validate configuration doesn't contain secrets +- Secure logging practices + +## Best Practices Research + +### Configuration Management Patterns + +**Twelve-Factor App Principles:** +- Store config in the environment +- Separate config from code +- Config varies between deployments + +**GitOps Principles:** +- Configuration as code +- Version control for configuration +- Automated validation and deployment + +### Validation and Error Handling + +**Validation Strategies:** +- Schema-based validation (JSON Schema for YAML) +- Required field checking +- Type validation +- Cross-reference validation + +**Error Handling:** +- Fail fast on invalid configuration +- Clear, actionable error messages +- Debug mode for troubleshooting +- Graceful degradation where possible + +## Implementation Approaches + +### Option 1: Composite Action with PowerShell +- Create reusable composite action +- PowerShell script for configuration loading +- YAML parsing and validation +- Environment variable export + +**Pros:** Full PowerShell integration, flexible validation +**Cons:** Requires PowerShell runtime, complex setup + +### Option 2: JavaScript Action +- Node.js based configuration loader +- YAML/JSON parsing libraries +- GitHub Actions toolkit integration +- Output to workflow environment + +**Pros:** Native GitHub Actions, fast execution +**Cons:** Additional language dependency, less PowerShell integration + +### Option 3: Workflow-Level Configuration +- Use GitHub Actions built-in features +- Matrix and conditional logic +- Repository variables +- Minimal custom code + +**Pros:** Simple, no custom actions needed +**Cons:** Limited flexibility, complex conditionals + +**Recommended:** Option 1 - Composite Action with PowerShell for full integration with Process-PSModule ecosystem + +## Dependencies and Integration + +### Required Dependencies + +**New Actions to Create:** +- `Get-Settings`: Configuration loading and validation +- `Test-Configuration`: Configuration testing and validation + +**Integration Points:** +- Main workflow: Load configuration early +- CI workflow: Validate configuration +- All jobs: Access configuration via environment variables + +### Backward Compatibility + +**Migration Strategy:** +- Optional configuration file +- Default values for missing settings +- Gradual migration path +- Documentation for migration + +## Performance and Scalability + +### Performance Targets + +**Loading Performance:** +- Configuration loading < 5 seconds +- Validation < 2 seconds +- Memory usage < 100MB + +**Scalability Considerations:** +- Support for large configuration files +- Efficient parsing and validation +- Caching where appropriate + +## Risk Assessment + +### Technical Risks + +- **Configuration File Parsing:** YAML parsing errors, complex validation +- **Environment Variable Limits:** GitHub Actions environment variable limits +- **Cross-Platform Compatibility:** PowerShell script execution on different runners + +### Mitigation Strategies + +- Comprehensive testing across platforms +- Fallback mechanisms for parsing failures +- Clear documentation and examples +- Incremental rollout approach + +## Conclusion + +**Recommended Approach:** +Implement a composite action using PowerShell that loads YAML configuration from `.github/PSModule.yml`, validates it against a schema, and exports values as environment variables. This provides centralized configuration management while maintaining compatibility with the existing Process-PSModule architecture. + +**Key Decisions:** +- YAML as primary configuration format +- Composite action with PowerShell implementation +- Schema-based validation +- Environment variable export +- Backward compatibility support + +**Next Steps:** +Proceed to Phase 1 design with contracts, data models, and implementation specifications. \ No newline at end of file diff --git a/specs/001-settings-driven-workflow/spec.md b/specs/001-settings-driven-workflow/spec.md new file mode 100644 index 00000000..03a6769c --- /dev/null +++ b/specs/001-settings-driven-workflow/spec.md @@ -0,0 +1,169 @@ +# Feature Specification: Settings-Driven Workflow Configuration + +## User Scenarios & Testing *(mandatory)* + +### Primary User Story + +As a workflow maintainer, I want a single, centralized configuration source that defines all workflow settings and parameters, so that the workflow structure is simplified and the complexity of conditional if: statements is significantly reduced. This central configuration should eliminate scattered hardcoded values and make the workflow logic more maintainable and easier to understand. + +### Acceptance Scenarios + +1. **Given** a workflow with multiple conditional branches based on hardcoded values, **When** configuration is centralized in a single source, **Then** the number of if: statements decreases by at least 50% and logic becomes more readable +2. **Given** configuration changes are needed, **When** I update the central configuration file, **Then** all dependent workflow steps automatically use the new values without individual modifications +3. **Given** a new workflow parameter is introduced, **When** I add it to the central configuration, **Then** it becomes immediately available to all workflow steps without code changes +4. **Given** environment-specific settings are required, **When** I define them in the central configuration, **Then** the workflow can access them through simple variable references instead of complex conditionals + +### Edge Cases + +- What happens when the central configuration file is missing or invalid? (Workflow should fail fast with clear error message) +- How does the system handle conflicting configuration values across different sections? (Validation should catch conflicts during configuration loading) +- What happens when a workflow step references a configuration key that doesn't exist? (Should provide default values or fail with descriptive error) +- How does the system handle configuration updates during workflow execution? (Configuration should be immutable once loaded) + +## Requirements *(mandatory)* + +### Functional Requirements + +| ID | Requirement | +|----|-------------| +| **FR-001** | System MUST provide a single configuration file as the authoritative source for all workflow settings | +| **FR-002** | System MUST validate configuration file syntax and required fields on workflow startup | +| **FR-003** | System MUST make configuration values available as environment variables or step inputs throughout the workflow | +| **FR-004** | System MUST support hierarchical configuration with environment-specific overrides | +| **FR-005** | System MUST reduce workflow conditional logic by replacing hardcoded values with configuration references | +| **FR-006** | System MUST provide clear error messages when configuration is invalid or missing required values | +| **FR-007** | System MUST support configuration inheritance and composition for complex scenarios | +| **FR-008** | System MUST allow configuration to be sourced from repository files, secrets, or external sources | +| **FR-009** | System MUST document all available configuration options and their effects on workflow behavior | +| **FR-010** | System MUST maintain backward compatibility with existing workflow structures during migration | + +### Non-Functional Requirements + +| ID | Requirement | +|----|-------------| +| **NFR-001** | Configuration loading MUST complete within 10 seconds to avoid workflow delays | +| **NFR-002** | Configuration validation MUST provide actionable error messages for troubleshooting | +| **NFR-003** | System MUST maintain configuration security by not exposing sensitive values in logs | +| **NFR-004** | Configuration changes MUST not require workflow file modifications for routine updates | +| **NFR-005** | System MUST support configuration versioning and rollback capabilities | + +### Quality Attributes Addressed + +| Attribute | Target Metric | +|-----------|---------------| +| **Maintainability** | Reduce conditional complexity by 60%; single source of truth for all configuration | +| **Reliability** | Configuration validation prevents runtime failures; consistent behavior across environments | +| **Usability** | Clear configuration structure reduces cognitive load for workflow maintainers | +| **Security** | Centralized configuration management with proper secret handling | +| **Efficiency** | Faster workflow updates through configuration changes rather than code modifications | + +### Constraints *(include if applicable)* + +| Constraint | Description | +|------------|-------------| +| **GitHub Actions Environment** | Must work within GitHub Actions workflow syntax and execution model | +| **Configuration Format** | Must support YAML, JSON, or PowerShell data file formats commonly used in workflows | +| **Backward Compatibility** | Must allow gradual migration from existing hardcoded configurations | +| **Repository Structure** | Must integrate with existing PSModule repository conventions | + +### Key Entities *(include if feature involves data)* + +| Entity | Description | +|--------|-------------| +| **Configuration File** | Central YAML/JSON/PSData file containing all workflow settings and parameters | +| **Configuration Schema** | Validation rules defining required and optional configuration properties | +| **Environment Overrides** | Environment-specific configuration values that extend or replace base settings | +| **Configuration Context** | Runtime information (branch, environment, trigger) used to select appropriate configuration | +| **Workflow Parameters** | Derived values from configuration that are passed to workflow steps and jobs | + +--- + +**Feature Branch**: `001-settings-driven-workflow` +**Created**: October 14, 2025 +**Status**: Draft +**Input**: User description: "Lets make the user story clearer, we want to have a single place to handle configuration and have that help simplify the structure of the rest of the workflow. Aiming to reduce complexity of if: statements in the workflow." + +## Execution Flow (main) + +1. Parse user description from Input + → If empty: ERROR "No feature description provided" +2. Extract key concepts from description + → Identify: actors, actions, data, constraints +3. For each unclear aspect: + → Mark with [NEEDS CLARIFICATION: specific question] +4. Fill User Scenarios & Testing section + → If no clear user flow: ERROR "Cannot determine user scenarios" +5. Generate Functional Requirements + → Each requirement must be testable + → Mark ambiguous requirements +6. Identify Key Entities (if data is involved) +7. Run Review Checklist + → If any [NEEDS CLARIFICATION]: WARN "Spec has uncertainties" + → If implementation details found: ERROR "Remove tech details" +8. Return: SUCCESS (spec ready for planning) + +--- + +## ⚡ Quick Guidelines + +- ✅ Focus on WHAT users need and WHY +- ❌ Avoid HOW to implement (no tech stack, APIs, code structure) +- 👥 Written for business stakeholders, not developers + +### Section Requirements + +- **Mandatory sections**: Must be completed for every feature +- **Optional sections**: Include only when relevant to the feature +- When a section doesn't apply, remove it entirely (don't leave as "N/A") + +### For AI Generation + +When creating this spec from a user prompt: + +1. **Mark all ambiguities**: Use [NEEDS CLARIFICATION: specific question] for any assumption you'd need to make +2. **Don't guess**: If the prompt doesn't specify something (e.g., "login system" without auth method), mark it +3. **Think like a tester**: Every vague requirement should fail the "testable and unambiguous" checklist item +4. **Common underspecified areas**: + - User types and permissions + - Data retention/deletion policies + - Performance targets and scale + - Error handling behaviors + - Integration requirements + - Security/compliance needs + +--- + +## Review & Acceptance Checklist + +*GATE: Automated checks run during main() execution* + +### Content Quality + +- [ ] No implementation details (languages, frameworks, APIs) +- [ ] Focused on user value and business needs +- [ ] Written for non-technical stakeholders +- [ ] All mandatory sections completed + +### Requirement Completeness + +- [ ] No [NEEDS CLARIFICATION] markers remain +- [ ] Requirements are testable and unambiguous +- [ ] Success criteria are measurable +- [ ] Scope is clearly bounded +- [ ] Dependencies and assumptions identified + +--- + +## Execution Status + +*Updated by main() during processing* + +- [ ] User description parsed +- [ ] Key concepts extracted +- [ ] Ambiguities marked +- [ ] User scenarios defined +- [ ] Requirements generated +- [ ] Entities identified +- [ ] Review checklist passed + +--- From 2b3ebbcb7b8014012a00884873606ff10a576c13 Mon Sep 17 00:00:00 2001 From: Marius Storhaug Date: Tue, 14 Oct 2025 14:38:35 +0200 Subject: [PATCH 03/34] feat: Complete planning phase for settings-driven workflow configuration - Research configuration patterns and best practices - Define data model for hierarchical configuration system - Create configuration schema contract with validation - Develop quickstart guide for setup and migration - Update agent context with technical details - Mark planning phases complete Implements Phase 0-1 of settings-driven workflow feature. --- .github/prompts/analyze.prompt.md | 150 --------------------- .github/prompts/clarify.prompt.md | 180 ------------------------- .github/prompts/constitution.prompt.md | 114 ---------------- .github/prompts/implement.prompt.md | 175 ------------------------ .github/prompts/plan.prompt.md | 142 ------------------- .github/prompts/pr.prompt.md | 172 ----------------------- .github/prompts/specify.prompt.md | 110 --------------- .github/prompts/tasks.prompt.md | 123 ----------------- 8 files changed, 1166 deletions(-) delete mode 100644 .github/prompts/analyze.prompt.md delete mode 100644 .github/prompts/clarify.prompt.md delete mode 100644 .github/prompts/constitution.prompt.md delete mode 100644 .github/prompts/implement.prompt.md delete mode 100644 .github/prompts/plan.prompt.md delete mode 100644 .github/prompts/pr.prompt.md delete mode 100644 .github/prompts/specify.prompt.md delete mode 100644 .github/prompts/tasks.prompt.md diff --git a/.github/prompts/analyze.prompt.md b/.github/prompts/analyze.prompt.md deleted file mode 100644 index a3525777..00000000 --- a/.github/prompts/analyze.prompt.md +++ /dev/null @@ -1,150 +0,0 @@ ---- -description: Perform a non-destructive cross-artifact consistency and quality analysis across spec.md, plan.md, and tasks.md after task generation. ---- - -# Analyze - -The user input to you can be provided directly by the agent or as a command argument - you **MUST** consider it before proceeding with the prompt (if not empty). - -User input: - -$ARGUMENTS - -Goal: Identify inconsistencies, duplications, ambiguities, and underspecified items across the three core artifacts (`spec.md`, `plan.md`, `tasks.md`) before implementation. This command MUST run only after `/tasks` has successfully produced a complete `tasks.md`. - -STRICTLY READ-ONLY: Do **not** modify any files. Output a structured analysis report. Offer an optional remediation plan (user must explicitly approve before any follow-up editing commands would be invoked manually). - -Constitution Authority: The project constitution (`.specify/memory/constitution.md`) is **non-negotiable** within this analysis scope. Constitution conflicts are automatically CRITICAL and require adjustment of the spec, plan, or tasks—not dilution, reinterpretation, or silent ignoring of the principle. If a principle itself needs to change, that must occur in a separate, explicit constitution update outside `/analyze`. - -Execution steps: - -1. Run `.specify/scripts/powershell/check-prerequisites.ps1 -Json -RequireTasks -IncludeTasks` once from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS. Derive absolute paths: - - SPEC = FEATURE_DIR/spec.md - - PLAN = FEATURE_DIR/plan.md - - TASKS = FEATURE_DIR/tasks.md - Abort with an error message if any required file is missing (instruct the user to run missing prerequisite command). - -2. Load artifacts: - - Parse spec.md sections: Overview/Context, Functional Requirements, Non-Functional Requirements, User Stories, Edge Cases (if present). - - Parse plan.md: Architecture/stack choices, Data Model references, Phases, Technical constraints. - - Parse tasks.md: Task IDs, descriptions, phase grouping, parallel markers [P], referenced file paths. - - Load constitution `.specify/memory/constitution.md` for principle validation. - -3. Build internal semantic models: - - Requirements inventory: Each functional + non-functional requirement with a stable key (derive slug based on imperative phrase; e.g., "User can upload file" -> `user-can-upload-file`). - - User story/action inventory. - - Task coverage mapping: Map each task to one or more requirements or stories (inference by keyword / explicit reference patterns like IDs or key phrases). - - Constitution rule set: Extract principle names and any MUST/SHOULD normative statements. - -4. Detection passes: - A. Duplication detection: - - Identify near-duplicate requirements. Mark lower-quality phrasing for consolidation. - B. Ambiguity detection: - - Flag vague adjectives (fast, scalable, secure, intuitive, robust) lacking measurable criteria. - - Flag unresolved placeholders (TODO, TKTK, ???, , etc.). - C. Underspecification: - - Requirements with verbs but missing object or measurable outcome. - - User stories missing acceptance criteria alignment. - - Tasks referencing files or components not defined in spec/plan. - D. Constitution alignment: - - Any requirement or plan element conflicting with a MUST principle. - - Missing mandated sections or quality gates from constitution. - E. Coverage gaps: - - Requirements with zero associated tasks. - - Tasks with no mapped requirement/story. - - Non-functional requirements not reflected in tasks (e.g., performance, security). - F. Inconsistency: - - Terminology drift (same concept named differently across files). - - Data entities referenced in plan but absent in spec (or vice versa). - - Task ordering contradictions (e.g., integration tasks before foundational setup tasks without dependency note). - - Conflicting requirements (e.g., one requires to use Next.js while other says to use Vue as the framework). - -5. Severity assignment heuristic: - - CRITICAL: Violates constitution MUST, missing core spec artifact, or requirement with zero coverage that blocks baseline functionality. - - HIGH: Duplicate or conflicting requirement, ambiguous security/performance attribute, or untestable acceptance criterion. - - MEDIUM: Terminology drift, missing non-functional task coverage, or underspecified edge case. - - LOW: Style/wording improvements, or minor redundancy not affecting execution order. - -6. Produce a Markdown report (no file writes) with sections: - - ```markdown - ### Specification Analysis Report - - | ID | Category | Severity | Location(s) | Summary | Recommendation | - |----|----------|----------|-------------|---------|----------------| - | A1 | Duplication | HIGH | spec.md:L120-134 | Two similar requirements ... | Merge phrasing; keep clearer version | - (Add one row per finding; generate stable IDs prefixed by category initial.) - ``` - - Additional subsections: - - Coverage Summary Table: - - ```markdown - | Requirement Key | Has Task? | Task IDs | Notes | - |-----------------|-----------|----------|-------| - ``` - - - Constitution Alignment Issues (if any) - - Unmapped Tasks (if any) - - Metrics: - * Total Requirements - * Total Tasks - * Coverage % (requirements with >=1 task) - * Ambiguity Count - * Duplication Count - * Critical Issues Count - -7. **Post issue comment** with analysis results organized by severity: - - Post a comment to the GitHub issue with the analysis findings - - Format the comment with separate tables for each severity level (only include levels that have findings): - ```markdown - ## Analysis Report - - ### Summary - - Total Requirements: X - - Total Tasks: Y - - Coverage: Z% - - Issues Found: N (A critical, B high, C medium, D low) - - ### Critical Issues - | ID | Category | Location(s) | Summary | Recommendation | - |----|----------|-------------|---------|----------------| - | C1 | ... | ... | ... | ... | - - ### High Priority Issues - | ID | Category | Location(s) | Summary | Recommendation | - |----|----------|-------------|---------|----------------| - | H1 | ... | ... | ... | ... | - - ### Medium Priority Issues - | ID | Category | Location(s) | Summary | Recommendation | - |----|----------|-------------|---------|----------------| - | M1 | ... | ... | ... | ... | - - ### Low Priority Issues - | ID | Category | Location(s) | Summary | Recommendation | - |----|----------|-------------|---------|----------------| - | L1 | ... | ... | ... | ... | - - ### Next Actions - - [Recommendation based on findings] - ``` - - Keep tables concise, limit to 20 findings per severity level - - If more than 20 findings in a category, add a note: "_(Additional N issues not shown - see full report)_" - - Include the complete "Next Actions" block with specific recommendations - -8. At end of report, output a concise Next Actions block: - - If CRITICAL issues exist: Recommend resolving them before `/implement`. - - If only LOW/MEDIUM issues: User may proceed, but provide improvement suggestions. - - Provide explicit command suggestions: e.g., "Run /specify with refinement", "Run /plan to adjust architecture", or "Manually edit tasks.md to add coverage for 'performance-metrics'". - -8. Ask the user: "Would you like me to suggest concrete remediation edits for the top N issues?" (Do NOT apply them automatically.) - -Behavior rules: -- NEVER modify files. -- NEVER hallucinate missing sections—if absent, report them. -- KEEP findings deterministic: if rerun without changes, produce consistent IDs and counts. -- LIMIT total findings in the main table to 50; aggregate remainder in a summarized overflow note. -- If zero issues found, emit a success report with coverage statistics and proceed recommendation. - -Context: $ARGUMENTS diff --git a/.github/prompts/clarify.prompt.md b/.github/prompts/clarify.prompt.md deleted file mode 100644 index ed8a26d0..00000000 --- a/.github/prompts/clarify.prompt.md +++ /dev/null @@ -1,180 +0,0 @@ ---- -description: Identify underspecified areas in the current feature spec by asking up to 5 highly targeted clarification questions and encoding answers back into the spec. ---- - -# Clarify - -The user input to you can be provided directly by the agent or as a command argument - you **MUST** consider it before proceeding with the prompt (if not empty). - -User input: - -$ARGUMENTS - -Goal: Detect and reduce ambiguity or missing decision points in the active feature specification and record the clarifications directly in the spec file. - -Note: This clarification workflow is expected to run (and be completed) BEFORE invoking `/plan`. If the user explicitly states they are skipping clarification (e.g., exploratory spike), you may proceed, but must warn that downstream rework risk increases. - -Execution steps: - -1. Run `.specify/scripts/powershell/check-prerequisites.ps1 -Json -PathsOnly` from repo root **once** (combined `--json --paths-only` mode / `-Json -PathsOnly`). Parse minimal JSON payload fields: - - `FEATURE_DIR` - - `FEATURE_SPEC` - - (Optionally capture `IMPL_PLAN`, `TASKS` for future chained flows.) - - If JSON parsing fails, abort and instruct user to re-run `/specify` or verify feature branch environment. - -2. Load the current spec file. Perform a structured ambiguity & coverage scan using this taxonomy. For each category, mark status: Clear / Partial / Missing. Produce an internal coverage map used for prioritization (do not output raw map unless no questions will be asked). - - Functional Scope & Behavior: - - Core user goals & success criteria - - Explicit out-of-scope declarations - - User roles / personas differentiation - - Domain & Data Model: - - Entities, attributes, relationships - - Identity & uniqueness rules - - Lifecycle/state transitions - - Data volume / scale assumptions - - Interaction & UX Flow: - - Critical user journeys / sequences - - Error/empty/loading states - - Accessibility or localization notes - - Non-Functional Quality Attributes: - - Performance (latency, throughput targets) - - Scalability (horizontal/vertical, limits) - - Reliability & availability (uptime, recovery expectations) - - Observability (logging, metrics, tracing signals) - - Security & privacy (authN/Z, data protection, threat assumptions) - - Compliance / regulatory constraints (if any) - - Integration & External Dependencies: - - External services/APIs and failure modes - - Data import/export formats - - Protocol/versioning assumptions - - Edge Cases & Failure Handling: - - Negative scenarios - - Rate limiting / throttling - - Conflict resolution (e.g., concurrent edits) - - Constraints & Tradeoffs: - - Technical constraints (language, storage, hosting) - - Explicit tradeoffs or rejected alternatives - - Terminology & Consistency: - - Canonical glossary terms - - Avoided synonyms / deprecated terms - - Completion Signals: - - Acceptance criteria testability - - Measurable Definition of Done style indicators - - Misc / Placeholders: - - TODO markers / unresolved decisions - - Ambiguous adjectives ("robust", "intuitive") lacking quantification - - For each category with Partial or Missing status, add a candidate question opportunity unless: - - Clarification would not materially change implementation or validation strategy - - Information is better deferred to planning phase (note internally) - -3. Generate (internally) a prioritized queue of candidate clarification questions (maximum 5). Do NOT output them all at once. Apply these constraints: - - Maximum of 5 total questions across the entire session. - - Each question must be answerable with EITHER: - * A short multiple‑choice selection (2–5 distinct, mutually exclusive options), OR - * A one-word / short‑phrase answer (explicitly constrain: "Answer in ≤5 words"). - - Only include questions whose answers materially impact architecture, data modeling, task decomposition, test design, UX behavior, operational readiness, or compliance validation. - - Ensure category coverage balance: attempt to cover the highest impact unresolved categories first; avoid asking two low-impact questions when a single high-impact area (e.g., security posture) is unresolved. - - Exclude questions already answered, trivial stylistic preferences, or plan-level execution details (unless blocking correctness). - - Favor clarifications that reduce downstream rework risk or prevent misaligned acceptance tests. - - If more than 5 categories remain unresolved, select the top 5 by (Impact * Uncertainty) heuristic. - -4. Sequential questioning loop (interactive): - - Present EXACTLY ONE question at a time. - - For multiple‑choice questions render options as a Markdown table: - - ```markdown - | Option | Description | - |--------|-------------| - | A |