Skip to content

SecurePatch Bench v0.1.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 04 Sep 07:27
· 3 commits to main since this release
v0.1.0
e26daa9
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

SecurePatch Bench v0.1.0

SecurePatch Bench v0.1.0 is the first public release of Pcnaid Inc.'s synthetic defensive benchmark for complete SaaS security remediation.

Included

  • Nine TypeScript scenarios across authorization, tenant isolation, and webhook security.
  • Paired functional and security oracles.
  • Vulnerable baselines, deliberately incomplete repairs, and complete reference repairs.
  • Reproducible CLI, JSON results, Markdown report, schemas, benchmark card, threat model, responsible-use guidance, and technical report.

Oracle-validation result

  • Rejected 18 of 18 negative controls.
  • Accepted 9 of 9 complete reference repairs.
  • All candidate families retained their designated functional behavior.

Important boundary

This release validates the benchmark and its test oracles. It does not report language-model or coding-agent performance and is not a general model leaderboard.

Publication

Research status: Technical Report / Preprint - not peer reviewed.

Assets 3
Loading