DeltaBench

DeltaBench

DeltaBench is the controlled benchmark used to evaluate selective recertification.

Components

• operating-system and API-oriented scenarios
• deterministic security checks
• controlled configuration mutations
• multiple operational domains
• adaptive attacks
• four locally hosted LLM profiles
• repeated execution

Core evaluation question

After a security-relevant agent change, can change-aware selection execute fewer tests while detecting more induced regressions than an equal-budget random selector?

Mutation-level outputs support analysis of regression prevalence, recall, random recall, test reduction, risk, and model-specific failure behavior.