Skip to content

ci: validate all config files and report per-file diagnostics#4

Draft
Copilot wants to merge 4 commits into
mainfrom
copilot/update-ci-config-validation
Draft

ci: validate all config files and report per-file diagnostics#4
Copilot wants to merge 4 commits into
mainfrom
copilot/update-ci-config-validation

Conversation

Copy link
Copy Markdown

Copilot AI commented Nov 13, 2025

The validate-configs job exits on first invalid JSON and prints no per-file diagnostics, causing ci-success to fail with "Validate Configs: failure" (job 55250260810).

Changes

  • Replaced separate JSON/YAML validation steps with unified Python script
  • Script validates all files before exiting, prints ✓ path or ✗ path: error per file
  • Exit code 1 only after checking all files

Example output

✓ capsules/lidlift-v1.json
✓ capsules/runbooks.json
✗ .devcontainer/devcontainer.json: Expecting property name enclosed in double quotes: line 2 column 5
✗ prompts.yaml: expected a single document in the stream in "prompts.yaml", line 2, column 1

One or more config files are invalid.

CI logs will now show which files are invalid. Follow-up PR needed to fix the invalid files.

Original prompt

Problem summary:
The CI workflow's validate-configs job currently exits on the first invalid JSON file and prints no per-file diagnostics, causing the ci-success job to fail with an unhelpful "Validate Configs: failure" message (job 55250260810). We need a change that validates all JSON and YAML files, prints per-file success/failure with error messages, and exits non-zero only after checking all files so CI logs show which files are invalid.

Required change:
Update .github/workflows/ci.yml to replace the current JSON loop and YAML "python -c" validation block with a single robust script that:

  • Installs pyyaml
  • Recursively finds all .json, .yml, and .yaml files excluding .git and node_modules
  • Validates each file and prints either a success marker (✓ path) or a failure marker (✗ path: message)
  • Continues checking all files even if some fail
  • Exits with code 1 if any file failed (0 otherwise)

Apply the change as a new branch and open a pull request.

Patch (replace the existing step contents under validate-configs that validate JSON and YAML — lines 184-221 in ref 99f653b — with the following):

      - name: Validate JSON and YAML files
        run: |
          pip install pyyaml
          python - <<'PY'
          import json, yaml, sys
          from pathlib import Path

          repo_root = Path('.')
          failed = False

          def ignore(path: Path) -> bool:
              s = str(path)
              return ('.git' in s) or ('node_modules' in s)

          def validate_json(path: Path):
              global failed
              try:
                  with open(path, 'r', encoding='utf-8') as fh:
                      json.load(fh)
                  print(f'✓ {path}')
              except Exception as e:
                  print(f'✗ {path}: {e}')
                  failed = True

          def validate_yaml(path: Path):
              global failed
              try:
                  with open(path, 'r', encoding='utf-8') as fh:
                      yaml.safe_load(fh)
                  print(f'✓ {path}')
              except Exception as e:
                  print(f'✗ {path}: {e}')
                  failed = True

          for f in repo_root.rglob('*.json'):
              if ignore(f):
                  continue
              validate_json(f)

          for f in list(repo_root.rglob('*.yml')) + list(repo_root.rglob('*.yaml')):
              if ignore(f):
                  continue
              validate_yaml(f)

          if failed:
              print('\nOne or more config files are invalid.')
          sys.exit(1 if failed else 0)
          PY

Notes:

  • This change only improves diagnostics; it will not change or fix repository files.
  • After this PR is merged and CI re-runs, the logs will show exactly which files are invalid. I can then open a follow-up PR to fix those files if you want.

Branch and PR details:

  • Branch name suggestion: ci/validate-configs-better-reporting
  • Commit message suggestion: "ci: improve validate-configs to report invalid JSON/YAML files"

Please confirm and I will create the branch and PR with the above change against the repository's default branch.

This pull request was created as a result of the following prompt from Copilot chat.

Problem summary:
The CI workflow's validate-configs job currently exits on the first invalid JSON file and prints no per-file diagnostics, causing the ci-success job to fail with an unhelpful "Validate Configs: failure" message (job 55250260810). We need a change that validates all JSON and YAML files, prints per-file success/failure with error messages, and exits non-zero only after checking all files so CI logs show which files are invalid.

Required change:
Update .github/workflows/ci.yml to replace the current JSON loop and YAML "python -c" validation block with a single robust script that:

  • Installs pyyaml
  • Recursively finds all .json, .yml, and .yaml files excluding .git and node_modules
  • Validates each file and prints either a success marker (✓ path) or a failure marker (✗ path: message)
  • Continues checking all files even if some fail
  • Exits with code 1 if any file failed (0 otherwise)

Apply the change as a new branch and open a pull request.

Patch (replace the existing step contents under validate-configs that validate JSON and YAML — lines 184-221 in ref 99f653b — with the following):

      - name: Validate JSON and YAML files
        run: |
          pip install pyyaml
          python - <<'PY'
          import json, yaml, sys
          from pathlib import Path

          repo_root = Path('.')
          failed = False

          def ignore(path: Path) -> bool:
              s = str(path)
              return ('.git' in s) or ('node_modules' in s)

          def validate_json(path: Path):
              global failed
              try:
                  with open(path, 'r', encoding='utf-8') as fh:
                      json.load(fh)
                  print(f'✓ {path}')
              except Exception as e:
                  print(f'✗ {path}: {e}')
                  failed = True

          def validate_yaml(path: Path):
              global failed
              try:
                  with open(path, 'r', encoding='utf-8') as fh:
                      yaml.safe_load(fh)
                  print(f'✓ {path}')
              except Exception as e:
                  print(f'✗ {path}: {e}')
                  failed = True

          for f in repo_root.rglob('*.json'):
              if ignore(f):
                  continue
              validate_json(f)

          for f in list(repo_root.rglob('*.yml')) + list(repo_root.rglob('*.yaml')):
              if ignore(f):
                  continue
              validate_yaml(f)

          if failed:
              print('\nOne or more config files are invalid.')
          sys.exit(1 if failed else 0)
          PY

Notes:

  • This change only improves diagnostics; it will not change or fix repository files.
  • After this PR is merged and CI re-runs, the logs will show exactly which files are invalid. I can then open a follow-up PR to fix those files if you want.

Branch and PR details:

  • Branch name suggestion: ci/validate-configs-better-reporting
  • Commit message suggestion: "ci: improve validate-configs to report invalid JSON/YAML files"

Please confirm and I will create the branch and PR with the above change against the repository's default branch.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI changed the title [WIP] Update CI workflow to validate all JSON and YAML files ci: validate all config files and report per-file diagnostics Nov 13, 2025
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/ci.yml
Comment on lines +195 to +196
s = str(path)
return ('.git' in s) or ('node_modules' in s)
Copy link

Copilot AI Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ignore() function uses substring matching which could incorrectly filter out legitimate files. For example, a file named project.github-actions.json would be ignored because it contains .git as a substring. Consider using path component matching instead:

def ignore(path: Path) -> bool:
    parts = path.parts
    return '.git' in parts or 'node_modules' in parts

This ensures only directories named .git or node_modules are excluded, not files containing these strings in their names.

Suggested change
s = str(path)
return ('.git' in s) or ('node_modules' in s)
parts = path.parts
return '.git' in parts or 'node_modules' in parts

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/ci.yml
Comment on lines +223 to +224
for f in list(repo_root.rglob('*.yml')) + list(repo_root.rglob('*.yaml')):
if ignore(f):
Copy link

Copilot AI Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script traverses the directory tree twice to find YAML files - once for *.yml and once for *.yaml. This is inefficient and could be slow in repositories with many files. Consider combining the patterns into a single loop:

for f in repo_root.rglob('*.y*ml'):
    if not ignore(f) and f.suffix in ('.yml', '.yaml'):
        validate_yaml(f)

Or collect all files in one pass:

yaml_files = list(repo_root.rglob('*.yml')) + list(repo_root.rglob('*.yaml'))
for f in yaml_files:
    if ignore(f):
        continue
    validate_yaml(f)

However, note that the second approach still requires two rglob calls, so the first pattern-based approach is more efficient.

Suggested change
for f in list(repo_root.rglob('*.yml')) + list(repo_root.rglob('*.yaml')):
if ignore(f):
for f in repo_root.rglob('*.y*ml'):
if ignore(f) or f.suffix not in ('.yml', '.yaml'):

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants