Skip to content

Latest commit

 

History

11 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

The SCRIPT Framework for AI's Impact Evaluation

The SCRIPT framework evaluates how AI automation affects different tasks and domains. Formalized from Andrej Karpathy's heuristic, it reveals why AI provides attackers a structural advantage over defenders in information security.

The SCRIPT framework

What is SCRIPT?

Short: Brief in time
Closed: Requires limited context
Repetitive: Repetition of one rote task
Independent: Standalone
Permissive: Forgiving of mistakes
Tech-ready: Digitally automatable

Tasks scoring high on SCRIPT (15-18/18) are ideal candidates for AI automation. Tasks scoring low (0-9/18) require human expertise and judgment.

Scoring Guide

Each SCRIPT dimension is scored 0-3:

  • 0 = Not at all
  • 1 = Somewhat
  • 2 = Mostly
  • 3 = Completely

Total Score Range: 0-18 points

Example Scorecards

Information Security Use Cases

Domain Attacker Score Defender Score Gap
Lateral Movement 18/18 ⭐⭐⭐⭐⭐ 8/18 ⭐⭐ +10
Phishing 18/18 ⭐⭐⭐⭐⭐ 12/18 ⭐⭐⭐ +6
Vulnerability Management 18/18 ⭐⭐⭐⭐⭐ 13/18 ⭐⭐⭐⭐ +5

Key Insight: Attackers consistently score perfect 18/18 because their tasks are repetitive, error-tolerant, brief, context-light, and independent. Defenders struggle with context requirements and error intolerance.

Create Your Own Scorecards

Use this prompt with Claude Sonnet 4.5 to evaluate any security domain:

Please read https://raw.githubusercontent.com/talbeerysec/SCRIPT-AI-Framework/main/SCRIPT-context.txt for context,
then using the format of this scorecard for lateral movement for attackers in https://raw.githubusercontent.com/talbeerysec/SCRIPT-AI-Framework/refs/heads/main/Scorecards/lateral_movement_attacker_scorecard.html 
and using the format of this scorecard for lateral movement for defenders in https://raw.githubusercontent.com/talbeerysec/SCRIPT-AI-Framework/refs/heads/main/Scorecards/lateral_movement_defender_scorecard.html 
create similar scorecards for attackers and defenders for <your security domain, e.g., Incident Response, Threat Hunting, etc.>

Replace <your security domain> with your specific use case.

Repository Contents

  • /Scorecards/ - Interactive HTML scorecards for various security domains
  • SCRIPT-context.txt - Full framework definition and scoring guidelines
  • /image/ - Framework diagrams and visualizations

Background

Read the full analysis:
📝 Follow the SCRIPT: Why Attackers are Winning the AI Arms Race

The Two Dimensions:

  1. Automation Readiness (R, P, T): Is this task suitable for automation at all?
  2. Context Window Fit (S, C, I): Can this fit in an AI's limited memory?

Acknowledgements

Claude Sonnet 4.5 for scorecard generation and analysis.

License

[Your License Here]

About

Framework for Assessing AI Automation Potential in Any Domain

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages