ASOPB Benchmark

ASOPB Benchmark

The Automated Security and Operational Posture Benchmark (ASOPB) is ADEPT's integrated security scanning framework. It runs a 7-pass pipeline to produce the Automated Security Posture Index (ASPI) score.

---

Overview

ASOPB scans the release tree (or any arbitrary directory) for security issues across 8 domains:

Domain	ID	Weight	What It Checks
Information Disclosure	D5	0.4	Secrets, credentials, PII, internal paths
Supply Chain	D8	0.4	CVEs, dependency vulnerabilities, version drift
Compliance	D7	0.2	Toxic content, inappropriate material

---

ASPI Scoring Model

Formula

domain_score = max(0, 100 - SUM(weight * ln(1 + count)))

Finding Weights

Category	Weight	Impact
D5 verified secrets	40	Gate-failing (1 finding = score 63)
D5 infrastructure findings	15	Significant
D5 real PII	8	Moderate
D5 unverified/FP	1.5	Minimal
D8 verified CVEs (Critical)	40	Gate-failing
D8 unverified/advisory	2.5	Minimal
D7 toxicity	8	Moderate

Composite Score

ASPI = D5 * 0.4 + D8 * 0.4 + D7 * 0.2

Stage Classification

Stage	ASPI Range	Meaning
S4 Hardened	>= 90	Release gate PASS
S3 Moderate	>= 70	Some issues, review required
S2 Weak	>= 50	Significant issues
S1 Critical	< 50	Not publishable

---

7-Pass Pipeline

Pass	Tool	Category	Purpose
P0	Regex scanner	SAST	Pattern matching for secrets, paths, credentials
P1	detect-secrets	SAST	Entropy-based secret detection
P1.5	TruffleHog	SAST	Git history + verified secret scanning
P2	NER/PII (spaCy)	DLP	Named entity recognition for PII
P3	Toxicity (detoxify)	DLP	Content safety classification
P4	Grype	SCA	CVE scanning against vulnerability databases
P5	Promptfoo	DAST	LLM adversarial red-teaming (OWASP LLM Top 10)

---

Running a Scan

Via Claude Code Skill

> /scan-release

This stages the release tree and runs the full 7-pass scan (30-75 minutes).

Via Makefile

# Stage the release tree
make stage

# Run the scan
make docker-scan-staged

# Check results
SCAN_DIR=$(ls -td logs/scan-staged-* | head -1)
python3 scripts/release/gate_check.py "$SCAN_DIR" --verbose

Via Docker Directly

docker run --rm \
  -v /tmp/adept-release-stage:/scan:ro \
  -v $(pwd)/logs/scan-output:/output \
  ghcr.io/pnnl/adept-release-scanner:latest \
  /scan --passes all --format json --output-dir /output

---

Scanner Image

• Image: ghcr.io/pnnl/adept-release-scanner:latest
• Size: ~8 GB (includes ML models, TruffleHog, Node.js for Promptfoo)
• Air-gapped: Runs without network access (all models bundled)
• Source: examples/adept_asopb/

---

CI/CD Integration

The ASOPB scan runs as a GitHub Actions workflow (asopb-pre-release-scan.yml):

Phase 1: Lightweight SAST (Blocking)

• Pure Python regex scan (no Docker image needed)
• Runs in ~30 seconds
• Blocks PR if ASPI falls below threshold

Phase 2: Full Pipeline (Advisory)

• Requires scanner Docker image from GHCR
• Parallelized across multiple runners
• ML chunks (NER/PII/Toxicity) conditional on label or manual dispatch
• Results posted as PR comment with ASPI score

---

False Positive Management

Allowlist

False positives are managed via .asopb-allowlist.yaml:

rules:
  - pattern: "example-api-key-12345"
    reason: "Documentation placeholder"
    category: "secrets"
    file_glob: "docs/**"

Compensating Controls

For CVEs that cannot be upgraded:

# .grype.yaml
ignore:
  # CVE-2024-XXXXX | HIGH | package 1.2.3 | Title
  #
  # Compensating Control:
  #   Vulnerable code path is not reachable in ADEPT because...
  #
  # Evidence:
  #   grep -rn "VulnerableClass" src/ → no results
  #
  - vulnerability: CVE-2024-XXXXX
    package:
      name: affected-package

---

Standalone Usage

ASOPB can scan any directory, not just ADEPT:

# Scan an arbitrary path
> /scan-path /path/to/project

# Or via Docker
docker run --rm \
  -v /path/to/project:/scan:ro \
  -v /tmp/output:/output \
  ghcr.io/pnnl/adept-release-scanner:latest \
  /scan --passes all --format json --output-dir /output

---

Source Code

The ASOPB framework is available as a standalone example:

• Source: examples/adept_asopb/
• Tests: 367 tests (217 pipeline + 150 ASOPB framework)
• Documentation: examples/adept_asopb/docs/

---

Further Reading

• CI/CD GitHub Actions
• Versioning & Releases
• Claude Code Skills (Phase 10: Release)