feat(redteam)!: align OWASP LLM Top 10 with the 2026 edition - #10335
feat(redteam)!: align OWASP LLM Top 10 with the 2026 edition#10335stefanoamorelli wants to merge 2 commits into
Conversation
![promptfoo-scanner[bot]](https://avatars.githubusercontent.com/in/2172945?s=60&v=4){kind=small}
There was a problem hiding this comment.
👍 All Clear
I reviewed the PR’s documentation updates and the red-team framework constants remapping to align with OWASP LLM Top 10 (2026). The changes are limited to labels, mappings of OWASP aliases to red-team plugins/strategies, tests, and an SVG. No new LLM calls, prompt handling, or executable agent/tool capabilities were introduced.
Minimum severity threshold: 🟡 Medium | To re-scan after changes, comment @promptfoo-scanner
Learn more
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #10335 +/- ##
==========================================
+ Coverage 81.32% 81.33% +0.01%
==========================================
Files 932 932
Lines 76789 76789
Branches 25196 25196
==========================================
+ Hits 62447 62460 +13
+ Misses 14342 14329 -13
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ce02209dc6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
All checks are green except Redteam (Production API), which fails on the remote generation API returning HTTP 500 ("Error generating AI response") for every call. The same job is currently failing with the identical error across recent unrelated branches (codex, renovate, dependabot, tracing PRs), so it looks like an upstream service issue rather than something in this diff. The review comments are addressed in 50f8515: tool-discovery added to owasp:llm:08, legacy 2025 anchors preserved on the renumbered doc sections, and the two blog examples updated to owasp:llm:03. |
Re-key OWASP_LLM_TOP_10_MAPPING and OWASP_LLM_TOP_10_NAMES to the 2026 numbering, rename System Prompt Leakage to the broader Hidden Context Exposure, and renumber the docs. Each risk keeps its existing plugin and strategy sets under its 2026 number, so the owasp:llm preset union is unchanged. BREAKING CHANGE: owasp:llm:03 through owasp:llm:10 aliases now resolve per the 2026 numbering.
Add tool-discovery to the owasp:llm:08 mapping so hidden-context scans cover tool schema disclosure, preserve the 2025 anchors on renumbered doc sections for existing external links, and update blog examples that labeled owasp:llm:06 as Excessive Agency.
50f8515 to
94e0f0f
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 94e0f0fdf4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| it('should not duplicate plugins when using multiple aliased names', () => { | ||
| const input = { | ||
| plugins: ['owasp:llm:01', 'owasp:llm:02', 'owasp:llm:04'], | ||
| plugins: ['owasp:llm:01', 'owasp:llm:02', 'owasp:llm:05'], |
There was a problem hiding this comment.
This PR reassigns most owasp:llm:NN aliases and adds tool-discovery to the full owasp:llm preset, but the only test change just swaps this dedupe fixture to owasp:llm:05; nothing asserts the new per-risk expansions (for example owasp:llm:03 agency or owasp:llm:06 DoS) or that Hidden Context scans include tool-discovery. A future regression in these public, security-sensitive compliance aliases would still pass this suite, so add explicit parsed-output assertions for the re-keyed aliases and the new full-preset plugin.
Useful? React with 👍 / 👎.
1 participant
We just published the 2026 edition of the Top 10 for LLM Applications on August 4, 2026 (canonical source). It reorders most of the list and replaces System Prompt Leakage with the broader Hidden Context Exposure. This PR re-keys
OWASP_LLM_TOP_10_MAPPINGandOWASP_LLM_TOP_10_NAMESso each risk keeps its existing plugin and strategy sets under its 2026 number, and updatessite/docs/red-team/owasp-llm-top-10.mdplus the cross-references in the supply chain, GDPR, agentic, and API Top 10 pages.Configs that pin individual
owasp:llm:NNaliases will now select the plugins of the 2026 risk with that number, which is why the title carries the breaking marker. The plainowasp:llmpreset gains onlytool-discovery, added to LLM08 during review so hidden-context scans exercise tool schema disclosure; the union is otherwise unchanged.