Make Step With-schema validation consistent and mandatory via Step Metadata

[ntt-matthias-fleschuetz]

Problem Statement

Step schema validation is currently inconsistent across StepTypes:

• Only some StepTypes provide AllowedWithKeys in their Step metadata.
• When AllowedWithKeys is missing, plan creation cannot detect typos/unknown With.* keys for that StepType (e.g. With.Proivder), and those issues may surface later (or never), reducing reliability and breaking fail-fast expectations.
• The Step metadata catalog loader validates RequiredCapabilities, but does not validate the structure and type of AllowedWithKeys.
• Condition schema validation is performed in multiple places (workflow validation and plan normalization), increasing complexity and risk of divergence.

This undermines IdLE's consistency and fail-fast guardrails for configuration-driven workflows.

Proposed Solution

Goals

1. Make Step schema validation consistent and fail-fast during plan creation.
2. Ensure every StepType defines its supported With.* contract in a data-only metadata structure.
3. Provide a forward-compatible metadata structure that can later validate:
   • required keys,
   • optional keys,
   • nested keys (sub-keys),
   • value types.

1) Make With-schema metadata mandatory (breaking)

Breaking change: Every StepType metadata entry MUST declare its With schema. If it does not, plan creation fails with a clear exception.

This ensures the plan builder can always validate With.* keys at plan-time.

As there are only the steps we deliver in the project at the moment, we do not do any legacy reference remarks, comments or warnings. We just continue and provide what the change needs.

2) Replace AllowedWithKeys with a forward-compatible WithSchema

Because required keys are also allowed keys, rename the concept to avoid redundancy and enable future sub-key/type checks.

Proposed metadata shape (data-only):

@{
  'IdLE.Step.Example' = @{
    RequiredCapabilities = @('Some.Capability')
    WithSchema = @{
      RequiredKeys = @('IdentityKey')      # keys that MUST exist
      OptionalKeys = @('Provider','AuthSession','Keep')  # keys that MAY exist

      # Optional: typed/nested schema (future-ready; implementable now in a limited form)
      Keys = @{
        Keep = @{
          Type = 'Hashtable'               # 'String' | 'Int32' | 'Boolean' | 'String[]' | 'Hashtable' | ...
          Required = $false
          Schema = @{
            Kind = @{ Type = 'String'; Required = $true }
            Id   = @{ Type = 'String'; Required = $true }
          }
        }
      }
    }
  }
}

Implementation approach (V1):

• Mandatory: WithSchema.RequiredKeys (string[]) and WithSchema.OptionalKeys (string[]).
• Plan-time validation must enforce:
  • no unknown keys: With.Keys ⊆ (RequiredKeys ∪ OptionalKeys)
  • all required keys exist
• Optional (nice-to-have within this same issue if small): validate WithSchema.Keys for keys present in With:
  • type match (e.g. Hashtable, String, String[], Boolean, Int32)
  • nested schema validation for Hashtable when Schema is provided

Note: This remains data-only and does not introduce ScriptBlocks.

3) Validate Step metadata in the catalog loader

Extend Resolve-IdleStepMetadataCatalog / related helper(s) so metadata is validated consistently:

• WithSchema must exist for every StepType.
• RequiredKeys and OptionalKeys must be arrays of non-empty strings (case-insensitive uniqueness).
• RequiredKeys must be a subset of (RequiredKeys ∪ OptionalKeys) by construction (no duplicates across sets).
• If Keys exists:
  • it must be a hashtable
  • each entry must have a supported Type
  • nested Schema must be a hashtable with the same constraints

Fail-fast with actionable error messages (StepType name + offending key).

4) Remove redundant Condition schema validation (simplify)

Current state performs condition schema checks in multiple places (workflow validation + plan normalization).

Target state:

• Keep workflow validation as: shape + data-only (no scriptblocks, correct top-level keys).
• Keep all plan-time checks in one place (plan normalization):
  • condition schema validation
  • resolvable path validation
  • condition evaluation

Concretely:

• Reduce Test-IdleStepDefinition to only:
  • step is IDictionary/hashtable
  • required top-level keys exist (Name, Type)
  • With and Condition are hashtables if present
  • Assert-IdleNoScriptBlock (data-only)
• Ensure ConvertTo-IdleWorkflowSteps remains the single source of truth for:
  • WithSchema validation
  • condition schema + path resolvability + evaluation

5) Execution-time considerations

Plan-time validation should be the primary mechanism. Execution should still be defensive, but it should not need to re-validate every With.* key. A minimal execution-time safeguard is acceptable (e.g., assert PlanStep.With is a hashtable / data-only).

Alternatives Considered

1. Keep AllowedWithKeys and add RequiredWithKeys

   • Rejected: conceptually redundant and naming becomes confusing (Required keys are always Allowed).
   • Migrating to OptionalKeys / RequiredKeys under a single WithSchema is clearer and extensible.

2. Validate StepType-specific schema during workflow file validation

   • Rejected: would make workflow validation depend on runtime module/provider state and reduce portability.
   • Plan creation is the correct phase because Step metadata is loaded there.

3. Introduce ScriptBlock validators per StepType

   • Rejected: would violate the data-only configuration guardrail and complicate security review.

Impact

• Breaking change: StepTypes without WithSchema will fail plan creation.
• Step pack modules must be updated to provide WithSchema for every exported StepType.
• Existing workflows with unknown With.* keys may start failing (desired fail-fast behavior).

Additional Context

This change aligns Step validation behavior with existing fail-fast consistency expectations already implemented for StepTypes that have AllowedWithKeys today.

---

Step 0 – Design (no open questions)

Scope boundaries

In-scope for this issue:

• Replace AllowedWithKeys usage in plan building with WithSchema (RequiredKeys/OptionalKeys).
• Enforce that every StepType declares WithSchema.
• Add strict metadata validation in the catalog loader.
• Simplify duplicate condition schema validation as described.

Out-of-scope (explicitly):

• Full-blown JSON Schema / external schema engines.
• Arbitrarily deep type systems. Only the limited typed/nested checks described above (if implemented) and only for Hashtable nesting.

Data model decisions

• WithSchema is part of Step metadata (trusted, module-delivered, data-only).
• Keys are validated case-insensitively (consistent with existing patterns).

Validation rules (plan-time)

For each Step:

1. Determine StepType.
2. Retrieve metadata (must exist).
3. Retrieve WithSchema (must exist).
4. Validate:
   • Required keys exist.
   • No unknown keys.
   • Optional typed/nested checks (if implemented in this issue).

Error handling

• Fail-fast with ArgumentException.
• Error messages include:
  • Step name, Step type,
  • offending key(s),
  • list of supported keys,
  • whether a required key is missing.

Files likely to change (non-exhaustive)

• src/IdLE.Core/Private/ConvertTo-IdleWorkflowSteps.ps1
• src/IdLE.Core/Private/Resolve-IdleStepMetadataCatalog.ps1 (and/or the helper used there)
• src/IdLE.Core/Private/Test-IdleStepDefinition.ps1
• Step packs:
  • src/IdLE.Steps.Common/Public/Get-IdleStepMetadataCatalog.ps1
  • src/IdLE.Steps.DirectorySync/Public/Get-IdleStepMetadataCatalog.ps1
  • src/IdLE.Steps.Mailbox/Public/Get-IdleStepMetadataCatalog.ps1
  • any additional step packs in src/IdLE.Steps.*

Test strategy

• Unit tests (Pester) for:
  • metadata loader rejects missing/invalid WithSchema
  • plan creation rejects unknown With.* keys for every StepType
  • plan creation rejects missing required keys
  • case-insensitive key handling
  • typed/nested validation (if implemented)

---

implementation plan

1. Add WithSchema validation to the metadata catalog loader

   • Implement a dedicated private validator (data-only, no dynamic invocation).
   • Ensure error messages are actionable.

2. Update plan normalization to use WithSchema

   • Replace current AllowedWithKeys validation with:
     • RequiredKeys + OptionalKeys key checking.
   • Keep current behavior for template resolution order (resolve templates before validating values; validate keys at least before execution).

3. Update all existing Step packs to provide WithSchema

   • For each StepType:
     • define RequiredKeys and OptionalKeys explicitly
     • include AuthSession where applicable (keep consistent with Steps behavior)

4. Simplify workflow step definition validation

   • Reduce Test-IdleStepDefinition to shape + data-only checks only.

5. Add/adjust Pester tests

   • Cover at least one representative StepType per step pack.
   • Add negative tests for unknown keys and missing required keys.

6. Docs/examples

   • Update docs that mention AllowedWithKeys to WithSchema.
   • Update any workflow examples if they contain invalid or unsupported With.* keys.

---

Acceptance Criteria

• Plan creation fails if any StepType metadata lacks WithSchema.
• Plan creation fails for unknown With.* keys for all StepTypes.
• Plan creation fails for missing WithSchema.RequiredKeys.
• Metadata catalog loader fails fast on invalid WithSchema types/structures.
• Condition schema validation is not duplicated across workflow validation and plan normalization (single source of truth in plan normalization).
• Pester tests cover the new behavior and at least one negative case per step pack.
• Step schema definition is not inline embedded in code, but defined in a separate commented psd1 file in the src code