feat(scripts): add structured JSON log output to Validate-Marketplace.ps1#1430
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
katriendg
left a comment
There was a problem hiding this comment.
Thanks for this contribution, @MauroDruwel! The implementation is looking good.
Before merge, please update the PR description to follow the PR template. You can generate one automatically by checking out your branch locally and running the /pull-request slash command in Copilot Chat, agent mode:
/pull-request issue https://github.com/microsoft/hve-core/issues/991
This will generate the correctly formatted description with the Type of Change checklist, Testing section, Required Automated Checks, and Security Considerations sections.
Then copy the output of the pr.md file into this PR manually, since the PR already exists.
Additional note: also update your other PRs with this approach?
Optional improvements:
- Update the script-level
.EXAMPLEin the comment-based help to show the new-OutputPathparameter usage. - Add
.OUTPUTSand.EXAMPLEto theWrite-MarketplaceValidationReportfunction's comment-based help, consistent with the other helper functions in the file.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thanks @katriendg — updated. I applied the requested changes:
Latest commit: 97491a3. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1430 +/- ##
==========================================
- Coverage 87.64% 87.43% -0.22%
==========================================
Files 65 64 -1
Lines 10152 9930 -222
==========================================
- Hits 8898 8682 -216
+ Misses 1254 1248 -6
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
katriendg
left a comment
There was a problem hiding this comment.
Thank you Mauro for the fast update, LGTM!
….ps1
`Validate-Collections.ps1` was the only collections-facing linting
script that did not write structured results to `logs/`. All other
pipeline scripts persist JSON reports after each run; this gap meant
collection validation results were lost at terminal close, blocking any
tooling or CI step that reads from `logs/`.
Changes made:
- Added `-OutputPath` parameter to `Validate-Collections.ps1` (default:
`logs/collection-validation-results.json`).
- Refactored `$fileErrors` from plain strings to typed hashtables
(`ErrorType` + `Message`), enabling specific error codes in JSON output:
`MissingRequiredField`, `InvalidIdFormat`, `DuplicateCollectionId`,
`InvalidCollectionMaturity`, `RepoSpecificPath`, `PathNotFound`,
`MissingSuffix`, `MissingItemKind`, `InvalidMaturity`,
`IntraCollectionDuplicate`.
- Introduced inner helper `Add-ValidationResult` that accumulates a
typed results list alongside every existing `Write-Host` call (console
output unchanged).
- Added `Export-CollectionValidationReport` function for path
resolution, directory creation, and JSON serialization.
- Moved `Create logs directory` step in `plugin-validation.yml` to
before `Validate collection metadata` to make workflow intent explicit.
- Expanded Pester test coverage: updated `ErrorType` assertion to
specific type, added severity differentiation, directory creation,
multiple distinct `ErrorType` values in one run, and clean-run `Results`
key presence tests.
Output schema:
```json
{
"Timestamp": "2026-04-23T17:00:00.000Z",
"TotalCollections": 14,
"ErrorCount": 0,
"Results": [
{
"Collection": "hve-core",
"Severity": "Warning",
"ErrorType": "MissingCompanionCollectionMd",
"Message": "missing companion 'hve-core.collection.md'"
}
]
}
```
## Related Issue(s)
Closes #992.
## Type of Change
Select all that apply:
**Code & Documentation:**
* [ ] Bug fix (non-breaking change fixing an issue)
* [x] New feature (non-breaking change adding functionality)
* [ ] Breaking change (fix or feature causing existing functionality to
change)
* [ ] Documentation update
**Infrastructure & Configuration:**
* [x] GitHub Actions workflow
* [ ] Linting configuration (markdown, PowerShell, etc.)
* [ ] Security configuration
* [ ] DevContainer configuration
* [ ] Dependency update
**AI Artifacts:**
* [ ] Reviewed contribution with `prompt-builder` agent and addressed
all feedback
* [ ] Copilot instructions (`.github/instructions/*.instructions.md`)
* [ ] Copilot prompt (`.github/prompts/*.prompt.md`)
* [ ] Copilot agent (`.github/agents/*.agent.md`)
* [ ] Copilot skill (`.github/skills/*/SKILL.md`)
**Other:**
* [x] Script/automation (`.ps1`, `.sh`, `.py`)
* [ ] Other (please describe):
## Testing
| Command | Result |
|---|---|
| `npm run lint:collections-metadata` | ✅ pass, writes
`logs/collection-validation-results.json` |
| `npm run lint:ps` | ✅ pass |
| `npm run test:ps -- -TestPath scripts/tests/collections/` | ✅ pass, 6
tests (2 pre-existing + 4 new), no regressions |
Baseline runs confirmed pre-existing test suite intact; post-change runs
confirmed new tests pass and JSON file is created with correct schema.
## Checklist
### Required Checks
* [x] Documentation is updated (if applicable)
* [x] Files follow existing naming conventions
* [x] Changes are backwards compatible (if applicable)
* [x] Tests added for new functionality (if applicable)
### AI Artifact Contributions
* [ ] Used `/prompt-analyze` to review contribution
* [ ] Addressed all feedback from `prompt-builder` review
* [ ] Verified contribution follows common standards and type-specific
requirements
### Required Automated Checks
The following validation commands must pass before merging:
* [ ] Markdown linting: `npm run lint:md`
* [ ] Spell checking: `npm run spell-check`
* [ ] Frontmatter validation: `npm run lint:frontmatter`
* [ ] Skill structure validation: `npm run validate:skills`
* [ ] Link validation: `npm run lint:md-links`
* [ ] PowerShell analysis: `npm run lint:ps`
* [ ] Plugin freshness: `npm run plugin:generate`
* [ ] Docusaurus tests: `npm run docs:test`
## Security Considerations
* [x] This PR does not contain any sensitive or NDA information
* [x] Any new dependencies have been reviewed for security issues
* [x] Security-related scripts follow the principle of least privilege
## Additional Notes
This PR was rebased on top of the main branch which includes #1430
(structured JSON output for `Validate-Marketplace.ps1`). The merge
conflict in `package.json` was resolved by combining both `-OutputPath`
flags: `lint:collections-metadata` and `lint:marketplace` both now pass
their respective output paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pull Request
Description
Add structured JSON log output support to
Validate-Marketplace.ps1so marketplace validation results are persisted tologs/in a machine-readable format, consistent with other linting scripts.This update now also includes review-requested help documentation improvements:
.EXAMPLEshows-OutputPathusageWrite-MarketplaceValidationReportnow includes.OUTPUTSand.EXAMPLERelated Issue(s)
Closes #991
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
prompt-builderagent and addressed all feedback.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md)Other:
.ps1,.sh,.py)Testing
npm run lint:psnpm run test:ps -- -TestPath "scripts/tests/plugins/"Checklist
Required Checks
AI Artifact Contributions
/prompt-analyzeto review contributionprompt-builderreviewRequired Automated Checks
The following validation commands must pass before merging:
npm run lint:mdnpm run spell-checknpm run lint:frontmatternpm run validate:skillsnpm run lint:md-linksnpm run lint:psnpm run plugin:generatenpm run docs:testSecurity Considerations
Additional Notes
JSON output is additive. Existing console output and failure behavior remain unchanged.