Add Testing Documentation for Standalone Detailed Inspection

tests/detailed_inspection_guide.ipynb

@@ -0,0 +1,130 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "c60b39da",
+   "metadata": {},
+   "source": [
+    "### ATC-138 Python Numerical Validation Testing Against MATLAB\n",
+    "\n",
+    "#### 1. Overview\n",
+    "\n",
+    "This documentation provides instructions on how to validate ATC-138 Python implementation against the original MATLAB framework for numerical consistency and correct reproduction of calculations.\n",
+    "\n",
+    "The focus is on the standalone use of the comparison engine (`compare_runs.py`) for users who need to validate their own Python and MATLAB models.\n",
+    "\n",
+    "The goal is to:\n",
+    "- Explain the testing metrics used\n",
+    "- Outline the testing workflow\n",
+    "\n",
+    "This workflow is independent of the integration tests and can be applied to any model. Two example models are included here for demonstration purposes.\n",
+    "\n",
+    "#### 2. Example Models Used \n",
+    "\n",
+    "| Model | Description |\n",
+    "|------|-------------|\n",
+    "| ICSB | 6-story special moment frame |\n",
+    "| haseltonRCMF_4story | 4-story reinforced concrete moment frame building |\n",
+    "\n",
+    "\n",
+    "#### 3. Validation Workflow\n",
+    "\n",
+    "1. Run the MATLAB ATC-138 implementation using the provided example inputs.\n",
+    "2. Run the Python ATC-138 implementation using the same demand/data. \n",
+    "3. Extract both output files `recovery_outputs`. \n",
+    "4. Compare Python and MATLAB outputs using quantitative metrics using `compare_runs.py`.\n",
+    "5. Evaluate whether results fall within acceptable tolerance thresholds.\n",
+    "\n",
+    "#### 4. Comparison Metrics\n",
+    "\n",
+    "Four quantitative metrics are used to evaluate agreement between MATLAB and Python results:\n",
+    "\n",
+    "</br>\n",
+    "\n",
+    "**Metric 1 — Absolute difference of mean functional recovery time**\n",
+    "\n",
+    "```| mean_FR_python − mean_FR_matlab |```\n",
+    "\n",
+    "Units: ```days```\n",
+    "\n",
+    "Threshold: ```≤ 2 days```\n",
+    "\n",
+    "</br>\n",
+    "\n",
+    "**Metric 2 — Relative difference of mean functional recovery time**\n",
+    "\n",
+    "```| mean_FR_python − mean_FR_matlab | / mean_FR_matlab```\n",
+    "\n",
+    "Units: ```percent```\n",
+    "\n",
+    "Threshold: ```≤ 4 %```\n",
+    "\n",
+    "</br>\n",
+    "\n",
+    "**Metric 3 — Worst system mean absolute error (MAE)**\n",
+    "\n",
+    "System-level wise, the maximum MAE of the fraction of unresolved damage over time, across all functional-recovery and reoccupancy systems. \n",
+    "\n",
+    "Threshold: ```≤ 0.02``` (maximum mean difference of 2% unresolved fraction)\n",
+    "\n",
+    "</br>\n",
+    "\n",
+    "**Metric 4 — Worst system 95th percentile absolute error**\n",
+    "\n",
+    "System-level wise, the maximum 95th percentile aboslute error of the fraction of unresolved damage over time, across all functional-recovery and reoccupancy systems. \n",
+    "\n",
+    "Threshold: ```≤ 0.04``` (maximum mean difference of 4% unresolved fraction)\n",
+    "\n",
+    "</br>\n",
+    "\n",
+    "#### 5. Recovery Trajectory and System-Level Recovery Comparison\n",
+    "\n",
+    "For each model, plots of recovery trajectories and system-level recovery curves are saved: \n",
+    "\n",
+    "- `recovery_trajectory.jpg`\n",
+    "- `system_breakdowns_functional.jpg`\n",
+    "- `system_breakdowns_reoccupancy.jpg`\n",
+    "\n",
+    "\n",
+    "#### 6. Validation Results\n",
+    "\n",
+    "The comparison results for the two example models are summarized below. Twenty batch runs were carried out for each model, and the mean values are compared against the defined metrics.\n",
+    "\n",
+    "| Model | Absolute difference (days) | Relative difference (%) | Worst system MAE | Worst system P95 |\n",
+    "|------|------|------|------|------|\n",
+    "| ICSB | 1.38 | 0.241 | 0.015 | 0.030 |\n",
+    "| haseltonRCMF_4story | 1.74 | 1.61 | 0.007 | 0.027 |\n",
+    "\n",
+    "\n",
+    "All comparison metrics here fall within the defined tolerance thresholds."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0d7cebb0",
+   "metadata": {},
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}