Project GlassBreak is FailSafe's public good responsible-disclosure initiative for open-source repositories. We report vulnerabilities we discover to maintainers and contribute fixes upstream. Our aim is straightforward: make the software everyone builds on safer, and give back to the community.
Every disclosure below was discovered by FailSafe and reported under this initiative.
| Project | Vulnerability | Status |
|---|---|---|
| NVIDIA NemoClaw | Path traversal via unsanitized --run-id in rollback/status actions, enabling arbitrary file read/write outside the state directory (PR) |
Merged |
Prototype pollution via unsanitized config path in snapshot migration, allowing arbitrary property injection into Object.prototype (PR) |
Merged | |
Incomplete SSRF blocklist missing IANA-reserved IP ranges (0.0.0.0/8, 198.18.0.0/15), allowing bypass to reach internal infrastructure (PR) |
Merged | |
| NEAR AI Ironclaw | Safety layer bypass via output truncation: oversized tool output skipped leak detection, policy enforcement, and injection scanning (PR) | Merged |
| Indirect prompt injection via memory poisoning (PR) | Open | |
| Zip bomb denial of service in document extraction (PR) | Merged | |
| SSRF via extension download and MCP transport redirects (PR) | Open | |
| Hermes Agent | Arbitrary file read through unvalidated MEDIA:<path> tags, exploitable via prompt injection to exfiltrate sensitive files (PR) |
Superseded |
| Missing Twilio webhook signature validation, allowing forged requests to bypass SMS allowlist and impersonate authorized users (PR) | Superseded | |
| Balancer ReClAMM | Mathematical edge case in virtual balance rounding that could cause underflow in extreme market conditions | Merged |
| Euler Finance | Vulnerabilities identified in the Euler Lite codebase | Reported |
| Consensys Web3Auth | Insecure PRNG used for authentication nonce in WalletConnectV2Connector (PR) | Superseded |
| Open redirect via WalletConnect peer metadata (PR) | Merged | |
| Jitsi | Cryptographic weakness: hardcoded salt and low iteration count in AESCrypto.java (PR) | Closed |
| Missing braces logic error leading to UI denial of service (PR) | Open | |
| Business logic flaw: TOCTOU bypass in OTR fingerprint verification (PR) | Open | |
| OKX Wallet Core | Use abi.encodePacked for EIP-712 array hashing (PR) |
Submitted† |
Missing deadline field in CALLS_TYPEHASH for validator execution path (PR) |
Submitted† | |
| Non-standard EIP-712 two-part digest in EIP-1271 validator path (PR) | Submitted† | |
| Vercel | Arbitrary code execution via path traversal in x-matched-path header | Open |
| Supabase MCP | Missing maximum operation limits: unbounded file array and content size in deployEdgeFunction | Open |
| FFmpeg | CVE-level vulnerabilities identified via agentic penetration testing (full report) | Reported |
| OpenBSD | Vulnerabilities identified in OpenBSD's slaacd daemon via agentic penetration testing (full report) | Reported |
| FIM (fim-one) | Authenticated SSRF via unvalidated MCP SSE/HTTP server URLs (no SSRF validation on create/update) (PR) | Merged |
IPv4-mapped IPv6 (::ffff:0:0/96) bypass in is_private_ip() SSRF protection (PR) |
Merged | |
| Aegra | Cross-user assistant config disclosure via missing ownership check in _prepare_run (PR) |
Merged |
| RestAI | IPv4-mapped IPv6 bypass in _is_private_ip SSRF protection (PR) |
Merged |
| SSRF protection for MCP server HTTP/HTTPS/SSE hosts (PR) | Superseded | |
| HeyM | SQL injection in webhook handler via unparameterized node_id lookup (PR) |
Merged |
Default JWT secret accepted at startup; require a non-default secret_key (PR) |
Merged | |
| Helix | Open-redirect phishing via logout redirect_uri; restrict to same-origin (PR) |
Merged |
| Solace Agent Mesh | Fail-open authentication: dev mode left requests authenticated (PR) |
Open |
| XSS in OAuth callback via unescaped error parameters (PR) | Open | |
| Skyvern | SSRF to cloud IMDS via HttpRequestBlock in workflow execution (PR) |
Open |
| ii-agent | Unauthenticated RCE via MCP proxy: require API key auth for /custom-mcp endpoint (PR) |
Open |
eval() code injection via LLM tool args in UserInputField (PR) |
Open | |
| IntentKit | Unauthenticated production routes exposed; gate local routes behind ENV check (PR) | Open |
| Stored prompt injection via unauthenticated autonomous task creation (PR) | Open | |
| LLMStack | or True auth bypass in AuthorizationMiddleware (PR) |
Open |
| Missing SSRF protection and disabled TLS verification in URI source (PR) | Open | |
| Griptape | Path traversal in LocalFileManagerDriver (absolute-path bypass + .. sequences) (PR) |
Open |
| AgentOps | Reflected XSS in /auth/callback via unescaped redirect_to in JS context, enabling OAuth token theft (PR) |
Open |
| AgentScope Runtime | Missing admin auth guard on process control endpoints (PR) | Open |
| OpenRAG | Missing OAuth credentials silently enable anonymous authenticated access (PR) | Open |
| IBM MCP Context Forge | Weak secrets accepted on exposed gateway binds (PR) | Open |
| AnythingLLM | Thread lookup not scoped to validated workspace in validWorkspaceAndThreadSlug (PR) |
Open |
| Airweave | Auth-disabled bootstrap without explicit local-dev opt-in (PR) | Open |
| Webhook management not gated behind privileged org role (PR) | Open | |
| python-a2a | Missing API key auth on mutating routes (before_request gate) (PR) |
Open |
| FastAPI LangGraph Template | Fail-open on missing or placeholder environment secrets (PR) | Open |
| AFFiNE | RevenueCat webhook handling hardening (PR) | Open |
| ConnectOnion | Missing word boundary in Bash(cmd *) permission wildcard (PR) |
Open |
Status legend: Merged — accepted and merged upstream. Open — submitted, pending maintainer review. Superseded — maintainer fixed the reported vulnerability independently and closed the PR (contribution acknowledged). Closed — closed without adoption. Reported — disclosed via security report; no public PR. † Submitted — okx/wallet-core disclosures predate the repository disabling public pull requests; the original PR links no longer resolve.
Also see how we compare against Claude Mythos here using Gemini 3 Flash.
Every finding above comes from FailSafe's Agentic Offensive Security Platform — an autonomous system that maps out threat models, system architectures, invariants, and trust boundaries through multiple specialised frontier models, then uses harnessed tooling and artifacts to guide autonomous red-team agents toward exploit validation in an isolated environment. The same approach applies to any codebase with security-critical logic: smart contracts, AI agent frameworks, web and mobile applications.
The rest of this README documents how the platform works and how it performs.
To make our results reproducible, we evaluated FailSafe against EVMBench, an open-source benchmark of 120 confirmed HIGH-severity vulnerabilities across 40 audit contests. Anyone can run the same evaluation against the same codebases.
| Approach | Detected | Recall |
|---|---|---|
| FailSafe | 83 / 120 | 69.2% |
| Claude Opus 4.6 (single agent) | ~55 / 120 | 45.6% |
| GPT-5.2 (single agent) | ~26 / 120 | ~22% |
- 22 / 40 contests with perfect detection
- All 40 contests completed within the 3-hour time limit
The benchmark tests only HIGH-severity findings, but the original audit contests also produced MEDIUM-severity findings (typically 10-26 per contest). Because the platform produces full threat models rather than isolated bug reports, its confirmed findings cover this territory too.
To illustrate, we cross-referenced the platform's output against the complete set of confirmed findings from the original Curves Code4rena contest.
Curves: 9 of 14 confirmed contest vulnerabilities detected. The contest produced 4 HIGHs and 10 MEDIUMs. FailSafe detected 3 of 4 HIGHs and independently identified 6 of 10 MEDIUMs, hitting 64% total recall across all severities.
| ID | Contest Finding | FailSafe Finding |
|---|---|---|
| H | (3 of 4 HIGHs detected) | |
| M-01 | Protocol fee permanently locked on sells | Protocol Fee Permanently Locked on Sells |
| M-03 | Lack of slippage protection in buy/sell | Missing Slippage Protection in buy() and sell() |
| M-05 | Anyone can set referral fee for any address | Referral Fee Manipulation via setReferralFeeDestination |
| M-07 | Wrapping all tokens causes permanent DoS | DoS on All Trading by Wrapping All Tokens to ERC20 |
| M-09 | Excess ETH from buy overpayment locked | Excess ETH from Buy Overpayment Permanently Locked |
| M-10 | onBalanceChange exploitable for fee theft | Weaponized onBalanceChange Wipes Victim's Unclaimed Fees |
The platform's core insight is that structured threat modeling provides better coverage than free-form code review. The pipeline builds a layered threat model through four phases, then uses those artifacts to guide autonomous deep-dive agents.
flowchart TD
A["Phase A - Foundation<br/>5 specialist analyses"]
B["Phase B - Threat Generation<br/>6 specialists × 2 LLM passes"]
C["Phase C - Deduplication<br/>Semantic consolidation"]
D["Phase D - Validation<br/>CONFIRMED / REFUTED"]
E["Phase E - Agentic Deep Dive<br/>Claude Opus + Codex 5.3"]
A -->|"invariants &<br/>architecture"| B
B -->|"50-80<br/>hypotheses"| C
C -->|"unique threats"| D
D -->|"full threat model +<br/>confirmed findings"| E
style A fill:#4a9eff,color:#fff,stroke:#2563eb
style B fill:#4a9eff,color:#fff,stroke:#2563eb
style C fill:#7c3aed,color:#fff,stroke:#5b21b6
style D fill:#059669,color:#fff,stroke:#047857
style E fill:#dc2626,color:#fff,stroke:#b91c1c
Five specialist LLMs analyze the codebase in parallel, each from a different perspective:
| Specialist | Focus |
|---|---|
| Architecture & Entry Points | Asset inventory, system structure, public interfaces |
| Security & Trust Boundaries | Trust zones, state transitions, vulnerability surface |
| Data Flow & Logic | Data propagation paths, business logic edge cases |
| State Machine Invariants | Lifecycle rules, monotonicity, access control invariants |
| Economic Invariants | Conservation laws, solvency rules, yield consistency |
Phase A establishes structural understanding: invariants, trust boundaries, and entry points. No attack hypotheses are generated here. This phase produces the context that downstream phases build on.
Six specialists generate concrete attack hypotheses informed by Phase A's analysis. Each specialist runs two passes with different LLMs to maximize coverage through model diversity:
| Specialist | Pass 1 | Pass 2 |
|---|---|---|
| Technical Threats | LLM-A | LLM-B |
| Economic Threats | LLM-A | LLM-C |
| Operational Threats | LLM-A | LLM-B |
Every hypothesis must be code-anchored: exact file, line numbers, and the specific pattern that triggered it. Typical output: 50-80 hypotheses per codebase.
Multiple specialists often flag the same vulnerability from different angles. A "reentrancy" finding from the technical specialist and a "flash loan manipulation" finding from the economic specialist may target the same state change. Phase C consolidates semantic duplicates while preserving distinct findings. Typical reduction: ~45%.
Each deduplicated hypothesis is validated independently through deep code analysis:
- Verify the proof-of-signal exists in the actual code
- Trace the complete execution path from entry point to vulnerability
- Confirm all preconditions are achievable
- If config-dependent, validate against deployment scripts
Each hypothesis receives a verdict: CONFIRMED, REFUTED, or CONTESTED (when validators disagree). No hypothesis is confirmed without citing the specific code that proves the defect.
Phases A-D produce the majority of detections. Phase E supplements them with autonomous agents (Claude Opus 4.6 and Codex 5.3) that run independent deep dives into the codebase. These agents receive the platform's full threat model as context: the architecture, invariants, trust boundaries, confirmed findings, and refuted hypotheses from Phases A-D. This lets them build on what the pipeline has already established and focus on areas with known gaps: integration boundaries, mathematical edge cases, and multi-step attack chains.
Phase E contributed 8 additional detections across the 40 benchmark contests.
The platform uses multiple LLM providers (Claude, GPT, Gemini) across all phases. Different models surface different classes of vulnerabilities; the heterogeneous ensemble provides broader coverage than any single model.
The primary miss pattern involves vulnerabilities at the boundary between audited code and external protocols, e.g. Pendle's skim() behavior, Balancer's getActualSupply vs totalSupply, or Morpho Blue decimal normalization. These require knowledge of external protocol interfaces that isn't present in the audited codebase.
In controlled experiments, providing integration documentation for external protocols increased detection from 10/20 to 15/20 on the noya contest (+50%). We did not include integration documentation in our benchmark submission to maintain parity with other approaches that operate on code alone. In production deployments, users supply third-party protocol documentation, which improves detection of integration boundary bugs.
The GPT-5 LLM judge exhibits +-2-3% variance across grading runs on borderline cases. All results reported here are from a single consistent grading session.
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| TOTAL: 83 / 120 (69.2%) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This repository includes full artifacts for all 40 contests. Each directory has its own README with detailed documentation.
| Directory | Contents | Start Here |
|---|---|---|
results/ |
Judge inputs and outputs (40 contests) | audit-graded-all-combined.json - the grading verdict for each contest |
swarm-outputs/ |
Full threat models (Phases A-D, ~4,750 files) | phase-d/confirmed/ - validated findings with root cause and code paths |
scripts/ |
Phase E runners, grading, and aggregation scripts | phase-e-agent.js - the Claude Phase E autonomous agent |
prompts/ |
Phase E prompt template | phase-e-prompt.txt |
To examine the platform's full analysis of a specific contest (e.g., Curves):
- Grading results -
results/per-contest/2024-01-curves/audit-graded-all-combined.json - Confirmed findings -
swarm-outputs/2024-01-curves/phase-d/confirmed/*.json - Threat model context -
swarm-outputs/2024-01-curves/phase-a-*.json - Raw submission -
results/per-contest/2024-01-curves/audit.json
- Phase E: Requires a Claude API key (
phase-e-agent.js) and/or an OpenAI API key (phase-e-codex.mjs). Run against any contest codebase with the platform's artifacts as input. - Grading: Requires an OpenAI API key (GPT-5 judge). Run
grade-detect.jsagainst ground truth. - Pipeline (Phases A-D): The pipeline scripts and prompts are not included. Outputs for all 40 contests are provided in
swarm-outputs/.
Built by the FailSafe team.