Skip to content

Add golden regression playback and tests#21

Merged
DevOpsMadDog merged 1 commit into
mainfrom
codex/extend-evaluate-method-for-decisions
Oct 4, 2025
Merged

Add golden regression playback and tests#21
DevOpsMadDog merged 1 commit into
mainfrom
codex/extend-evaluate-method-for-decisions

Conversation

@DevOpsMadDog
Copy link
Copy Markdown
Owner

Summary

  • add a golden regression store that can replay stored cases against a decision engine and report decision deltas
  • refresh the regression dataset with block/allow/defer scenarios and metadata needed to reconstruct context
  • provide a CLI playbook and regression tests that highlight mismatches with mock decision outcomes

Testing

  • pytest tests/test_golden_regression_store.py

https://chatgpt.com/codex/tasks/task_e_68df34fee50883298b83ad06d3ff97f6

@DevOpsMadDog DevOpsMadDog merged commit 2662ffa into main Oct 4, 2025
DevOpsMadDog added a commit that referenced this pull request May 1, 2026
…e 3 cluster S22 Post-Incident Knowledge

Folds 3 standalone post-incident analytics pages into a single tabbed hero
at /remediate/incidents/knowledge per docs/UX_CONSOLIDATION_PLAN_2026-04-26.md
§2.22 (S22 Incident Response — Post-Incident Knowledge sub-cluster).

  tab        | source page                  | endpoint(s)
  -----------|------------------------------|----------------------------------
  metrics    | IncidentMetricsDashboard     | /api/v1/incident-metrics/{stats,incidents}
  knowledge  | IncidentKBDashboard          | /api/v1/incident-kb/{articles,stats}
  lessons    | IncidentLessonsDashboard     | /api/v1/incident-lessons/{lessons,stats}

Changes:
- new: suite-ui/aldeci-ui-new/src/pages/IncidentKnowledgeHub.tsx (lazy-imports
  the 3 source pages, syncs ?tab= via useSearchParams, framer-motion entry,
  PageHeader with active-tab badge, PageSkeleton suspense fallback)
- App.tsx: lazy import + canonical /remediate/incidents/knowledge route +
  /incident-metrics, /incident-kb, /incident-lessons → Navigate redirects
- FOLDED markers added to top of all 3 source pages (preserve git blame —
  files still imported by hub)
- docs/UX_CONSOLIDATION_PLAN_2026-04-26.md §2.22: added Post-Incident
  Knowledge sub-cluster DONE entry

Persona target: #7 Incident Responder, #6 SOC T2, #14 Engineering Manager,
                #21 QA Engineer (IR Lessons).

Verified via headless Chromium (Playwright):
- canonical URL loads → defaults to ?tab=metrics, 5 /api/v1 calls, 0 mocks
- /incident-metrics → /remediate/incidents/knowledge?tab=metrics (5 APIs)
- /incident-kb → /remediate/incidents/knowledge?tab=knowledge (5 APIs)
- /incident-lessons → /remediate/incidents/knowledge?tab=lessons (5 APIs)
Screenshot: docs/ui-snapshots/ux-consolidation-incident-knowledge-2026-05-02.png

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 1, 2026
… S29

Wired 3 standalone training/culture dashboards into single tabbed hero
(routes already present from prior partial; this commit completes the source-page FOLDED markers and Playwright verification):

- /security-training        → /admin/training-culture?tab=training
- /training-effectiveness   → /admin/training-culture?tab=effectiveness
- /security-culture         → /admin/training-culture?tab=culture

Changes:
- App.tsx: lazy import + canonical /admin/training-culture route + 3 Navigate redirects all already in place from prior partial work
- SecurityTrainingDashboard.tsx, TrainingEffectivenessDashboard.tsx, SecurityCultureDashboard.tsx: prepended FOLDED comment

Verified via Playwright (domcontentloaded):
  http://localhost:5173/admin/training-culture
  - 9 real /api/v1/security-training/* calls fired (NO MOCKS)
  - Tabs: Training | Effectiveness | Culture
  - Screenshot: docs/ui-snapshots/ux-consolidation-training-culture-hub-2026-05-02.png

Persona target: Security Awareness Lead (#21), CISO (#1), GRC Analyst (#12)
Plan: docs/UX_CONSOLIDATION_PLAN_2026-04-26.md (S29 Training & Culture sub-cluster)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 2, 2026
…endpoints

Closed in this batch (canonical envelope, mirroring batch-6 pattern):
- /api/v1/posture-reports/reports #7: canonical envelope shipped
- /api/v1/cloud-ir/incidents #17: canonical envelope shipped
- /api/v1/network-forensics/captures #21: canonical envelope shipped
- /api/v1/network-segmentation/segments #22: canonical envelope shipped
- /api/v1/microsegmentation/segments #23: canonical envelope shipped
- /api/v1/awareness-gamification/challenges #29: canonical envelope shipped
- /api/v1/gdpr/activities #30: canonical envelope shipped

Pattern (class-c): all seven list endpoints upgraded from minimal
{<legacy_key>, total, hint} to the canonical batch-6/batch-7 envelope:
    {
        "items": [...],
        "<legacy_key>": [...],   # back-compat (reports/incidents/captures/etc.)
        "total": int,
        "org_id": str,
        "limit": int,            # ge=1, le=500 — defaults to 50
        "offset": int,           # ge=0 — defaults to 0
        "filters_applied": {...} # echoes every filter param (None if unset)
        "hint": str              # only present when total == 0
    }

Each endpoint now (1) accepts limit + offset query params with FastAPI
ge/le validation, (2) echoes every filter back into filters_applied even
when None (no missing keys), (3) always returns the full envelope shape
even on hit (legacy clients keep their original key, new clients use
items + pagination context), (4) preserves the actionable empty-state
hint with a "this is correct for fresh tenants" framing.

Triage status update: 26/30 fully closed. 4 class-a deferred (need real
cloud creds, OAuth flows, or PAM tenant access not present in fleet —
sprint-able with customer engagement). All class-b importer-gated
endpoints (8) and all class-c structured-empty endpoints (12) now
closed.

Verified: pytest tests/test_empty_endpoints_batch7.py 11/11 PASS.
Beast Mode regression on phase4/phase7/trustgraph/pipeline_api: 170/170 PASS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 5, 2026
Beast Mode 13-file: 753/753 passed in 8.74s
-m perf: 194 passed, 2 skipped, 44782 deselected in 26.55s
-m owasp: 47 passed, 2 skipped, 44929 deselected in 17.95s

7 commits since sweep #20 (a8a0862) validated:
cad33d9 dedupe, 3519e40 owasp marker, 465317a ci-doc,
64c84ec snapshot v3, 6381af4 CLAUDE.md, 426fa14 marker smoke,
cfd36eb dead marker cleanup. Zero regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 5, 2026
…ion close

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 5, 2026
…9f406a3

Adds "Late-Session Additions (2026-05-05 evening)" sub-section under
[Unreleased] covering sweeps #7#21 (range 43d43d9..98d04d2):
- Hardening: async-emit fix, asyncio races x2, dead lazy imports purge, docker/k8s, shell scripts
- CI: UI build verification job, regression-gates audit
- Tests: sweeps #7#21, lockdown tests, marker smoke, conftest DRY, broken-collector triage, 5 UI test files
- Docs: HANDOFF v5–v13, dep audits, shell/bundle audit, state snapshots
- Features: /api/v1/admin/connectors/inventory, owasp pytest marker

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 5, 2026
…at HEAD d9b7051

3 doc-only commits since sweep #21. Abbreviated check: 111 phase tests green,
-m perf 194 stable, -m owasp 47 stable. Zero regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant