Skip to content

Fix security-review skill discovery#1515

Merged
aaronpowell merged 1 commit intogithub:stagedfrom
yeelam-gordon:fix/security-review-skill-search
Apr 29, 2026
Merged

Fix security-review skill discovery#1515
aaronpowell merged 1 commit intogithub:stagedfrom
yeelam-gordon:fix/security-review-skill-search

Conversation

@yeelam-gordon
Copy link
Copy Markdown
Contributor

@yeelam-gordon yeelam-gordon commented Apr 26, 2026

Problem

security-review is published, but exact slug searches like security-review were unreliable because generated skill search metadata only guaranteed the display title and description.

What changed

This keeps the old search inputs and adds the missing ones in generated website data.

Search input Before After
Display title, e.g. Security Review Searched Still searched
Description Searched Still searched
Skill id/folder, e.g. security-review Not guaranteed Added
Frontmatter name, e.g. security-review Not guaranteed Added
Path, e.g. skills/security-review Not searched Added
Category Not searched Added

metadata.name is the raw skill name from SKILL.md (security-review). skill.title is the generated display title (Security Review). The generated searchText now includes both, so existing queries like Security, Review, and Security Review do not regress, while exact slug/path queries now work too.

No skills/security-review/SKILL.md description change is needed anymore; the generator now indexes the skill name/path directly. Without this generator change, a skill could work around the issue by repeating its slug in the description, but that should not be required.

Validation

  • npm run website:data
  • npm run skill:validate
  • npm run website:build
  • git diff --check
  • Confirmed generated skills.json includes Security Review, security-review, and skills/security-review in searchable metadata

Fixes #1513

Copilot AI review requested due to automatic review settings April 26, 2026 13:12
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 26, 2026

🔍 Skill Validator Results

⚠️ Warnings or advisories found

Scope Checked
Skills 1
Agents 1
Total 2
Severity Count
--- ---:
❌ Errors 0
⚠️ Warnings 1
ℹ️ Advisories 0

Summary

Level Finding
ℹ️ Found 1 skill(s)
ℹ️ [security-review] 📊 security-review: 2,051 BPE tokens [chars/4: 2,269] (detailed ✓), 15 sections, 0 code blocks
ℹ️ [security-review] ⚠ No code blocks — agents perform better with concrete snippets and commands.
ℹ️ ✅ All checks passed (1 skill(s))
Full validator output ```text Found 1 skill(s) [security-review] 📊 security-review: 2,051 BPE tokens [chars/4: 2,269] (detailed ✓), 15 sections, 0 code blocks [security-review] ⚠ No code blocks — agents perform better with concrete snippets and commands. ✅ All checks passed (1 skill(s)) ```

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Addresses skill discoverability on the Awesome Copilot skills website by enriching both the skill’s trigger phrasing and the website’s generated search metadata so searches like security-review and /security-review match reliably.

Changes:

  • Add explicit security-review and /security-review trigger terms to the security-review skill description.
  • Expand generated website search metadata for skills to include id/name/path/category (via a precomputed searchText field).
  • Regenerate the docs/README.skills.md entry for the updated skill description.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
skills/security-review/SKILL.md Adds direct trigger phrases to improve matching for security-review-style queries.
eng/generate-website-data.mjs Includes richer skill search text (id/name/path/category/description) and uses it in search-index.json.
docs/README.skills.md Updates the generated README table row to reflect the new description text.

Copy link
Copy Markdown
Contributor

@aaronpowell aaronpowell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewing this PR, I think it's more solving the symptom (this skill doesn't return) rather than the problem (search isn't doing any weighting on results).

I think it'd be better to evaluate the search logic and look to add some weighting into fields (title match > description match) and perform some ordering in the results (sort by match not just alphabetical).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@yeelam-gordon yeelam-gordon force-pushed the fix/security-review-skill-search branch from 6cf7a0a to 128442e Compare April 28, 2026 14:58
@yeelam-gordon
Copy link
Copy Markdown
Contributor Author

Reviewing this PR, I think it's more solving the symptom (this skill doesn't return) rather than the problem (search isn't doing any weighting on results).

I think it'd be better to evaluate the search logic and look to add some weighting into fields (title match > description match) and perform some ordering in the results (sort by match not just alphabetical).

@aaronpowell, I updated this PR description. The fix is not on weighting yet, but adding enough "search fields" for discovery now.
I removed other unnecessary change in this PR.

@aaronpowell
Copy link
Copy Markdown
Contributor

I'll commit this as an interim fix, but I'm going to be revising the website search in general shortly.

@aaronpowell aaronpowell merged commit ae2c51d into github:staged Apr 29, 2026
8 checks passed
fizznix pushed a commit to fizznix/awesome-copilot that referenced this pull request Apr 29, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants