Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 180 additions & 0 deletions examples/stpa-ai/artifacts/ml-safety.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# STPA-for-AI artifacts: Pedestrian detection ML controller
#
# Demonstrates the full ML safety lifecycle:
# - Base STPA artifacts (loss, hazard, controller, UCA)
# - ML controller with training data provenance
# - Data hazard from insufficient coverage
# - ML-specific UCA refinement
# - Post-deployment monitoring and retraining

artifacts:
# ── Step 1: Losses and hazards ──────────────────────────────────────────

- id: L-001
type: loss
title: Loss of pedestrian safety
status: draft
description: >
Pedestrian is struck by autonomous vehicle due to perception
or decision failure, resulting in injury or death.
fields:
stakeholders: [pedestrians, vehicle-occupants, operator]

- id: H-001
type: hazard
title: Vehicle fails to stop for pedestrian in crosswalk
status: draft
description: >
Vehicle does not decelerate when a pedestrian is present in
the planned trajectory, leading to L-001.
fields:
severity: catastrophic
links:
- type: leads-to-loss
target: L-001

# ── Step 2: Control structure ───────────────────────────────────────────

- id: CTRL-001
type: controller
title: Perception and braking controller
status: draft
description: >
Automated controller responsible for detecting obstacles and
issuing brake commands.
fields:
controller-type: automated

- id: ML-CTRL-001
type: ml-controller
title: Pedestrian detection CNN
status: draft
description: >
Convolutional neural network that detects pedestrians in camera
frames and outputs bounding boxes with confidence scores.
fields:
model-type: cnn
training-framework: PyTorch
inference-latency-ms: '35'
links:
- type: refines
target: CTRL-001

# ── Training data provenance ────────────────────────────────────────────

- id: TDS-001
type: training-data-source
title: Pedestrian detection training dataset
status: draft
description: >
Combined dataset of urban driving scenes with annotated
pedestrian bounding boxes.
fields:
data-sources: >
NuScenes (40k frames), internal fleet recordings (120k frames),
synthetic scenes from CARLA simulator (80k frames).
collection-method: >
Fleet vehicles equipped with front-facing cameras. Frames
sampled at 2 Hz during urban driving. Synthetic data generated
with randomized pedestrian models and lighting.
labeling-method: Semi-automated with human QA review
size: 240k annotated frames
bias-assessment: >
Under-representation of wheelchair users and children under 5.
Night-time scenes are 15% of dataset vs 30% of operating hours.
Mitigation: targeted collection campaign planned for Q3.
distribution-characteristics: >
70% daytime, 15% dusk/dawn, 15% night. Urban environments only.
links:
- type: trains
target: ML-CTRL-001

# ── Data hazard ─────────────────────────────────────────────────────────

- id: DH-001
type: data-hazard
title: Insufficient night-time pedestrian coverage
status: draft
description: >
Training data under-represents night-time conditions, risking
degraded detection performance in low-light scenarios.
fields:
hazard-category: insufficient-coverage
affected-population: Pedestrians in low-light conditions
links:
- type: leads-to-hazard
target: H-001

# ── Step 3: UCAs ────────────────────────────────────────────────────────

- id: UCA-001
type: uca
title: Controller does not issue brake when pedestrian detected
status: draft
description: >
Not providing a brake command when a pedestrian is detected
in the vehicle path leads to H-001.
fields:
uca-type: not-providing
context: >
Pedestrian is in crosswalk, vehicle approaching at city speed.
links:
- type: issued-by
target: CTRL-001
- type: leads-to-hazard
target: H-001

- id: ML-UCA-001
type: ml-uca
title: CNN misclassifies pedestrian as background at night
status: draft
description: >
The pedestrian detection model fails to detect a pedestrian
in low-light conditions due to distribution gap in training data.
fields:
ml-failure-mode: misclassification
operational-design-domain: >
Urban roads, speed below 50 km/h, ambient light above 1 lux.
links:
- type: refines
target: UCA-001

# ── Post-deployment monitoring ──────────────────────────────────────────

- id: MON-001
type: monitoring-trigger
title: Detection accuracy drop monitor
status: draft
description: >
Monitors real-time pedestrian detection accuracy against
ground-truth from shadow-mode lidar cross-check.
fields:
metric-name: pedestrian-recall
threshold: '<0.95 over rolling 7-day window'
detection-method: >
Lidar-based shadow detector provides ground-truth labels;
camera detections compared daily.
links:
- type: monitors
target: ML-CTRL-001

- id: RTR-001
type: retraining-requirement
title: Retrain on low-light failures
status: draft
description: >
When night-time recall drops below threshold, retrain with
augmented low-light dataset.
fields:
trigger-condition: >
Pedestrian recall at night (ambient light < 10 lux) falls
below 0.93 for 3 consecutive days.
validation-criteria: >
Retrained model must achieve >= 0.96 recall on night-time
holdout set and not regress on daytime recall (>= 0.98).
data-requirements: >
Minimum 20k additional night-time frames with pedestrian
annotations from diverse urban environments.
links:
- type: satisfies
target: MON-001
20 changes: 20 additions & 0 deletions examples/stpa-ai/rivet.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# STPA-for-AI Example — Autonomous Vehicle Pedestrian Detection
#
# Demonstrates STPA analysis extended with ML-specific artifact types
# for an ML-based pedestrian detection system. Covers training data
# provenance, data hazards, ML UCAs, and post-deployment monitoring.
#
# Run: rivet --schemas ../../schemas validate
# Dashboard: rivet --schemas ../../schemas serve

project:
name: av-pedestrian-detection
version: "0.1.0"
schemas:
- common
- stpa
- stpa-ai

sources:
- path: artifacts
format: generic-yaml
Loading
Loading