Skip to content

Implement regression similarity lookup and tests#17

Closed
DevOpsMadDog wants to merge 1 commit into
mainfrom
codex/implement-regression-validation-logic-and-tests
Closed

Implement regression similarity lookup and tests#17
DevOpsMadDog wants to merge 1 commit into
mainfrom
codex/implement-regression-validation-logic-and-tests

Conversation

@DevOpsMadDog
Copy link
Copy Markdown
Owner

@DevOpsMadDog DevOpsMadDog commented Oct 2, 2025

Summary

  • replace the golden regression stub with a database-backed lookup that scores historical policy decisions for similarity and aggregates confidence
  • include matched attribute details and overall validation metadata in the regression response consumed by the consensus logic
  • add an asynchronous pytest that seeds sample policy decisions, drives the decision engine in production mode, and verifies the regression statistics are used

Testing

  • pytest tests/test_golden_regression_validation.py

https://chatgpt.com/codex/tasks/task_e_68dee0517c988329bdc8f1378ff261ea

DevOpsMadDog added a commit that referenced this pull request Apr 26, 2026
@DevOpsMadDog @claude
beast-mode(scrum): sprint 2 demo backlog + tasks-state refresh with t…
17728eb 
…oday's closures

- docs/SPRINT_2_DEMO_BACKLOG_2026-04-22.md: DEMO-001..DEMO-005 P0 items
  with owners, acceptance criteria, 44h total effort, sprint goal,
  definition of done, risks + mitigations.
- .omc/TASKS_STATE_2026-04-22.md: closed threads #1 (graphify visual,
  commit 7386db5) and #7 (TrueCourse side-by-side, commit 0639bb3);
  added new threads #9 (promote NEW-G070/G071 to gap-matrix + PRDs)
  and #10 (backend-code graphify ingest); flagged threads #17/#18 as
  in-flight (enterprise-architect reconcile + ux-architect UI dispatch).
- Historical task rows 1-12 preserved immutable.
- READ FIRST honest-state section preserved.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 1, 2026
@DevOpsMadDog @claude
beast-mode(ux): finish merge for AppLayerSecurityHub — Phase 3 cluste…
4520c85 
…r S10

Wired 3 standalone application-layer pages into single tabbed hero:
- /app-security        → /discover/app-security?tab=web
- /mobile-app-security → /discover/app-security?tab=mobile
- /browser-security    → /discover/app-security?tab=browser

Changes:
- App.tsx: added AppLayerSecurityHub lazy import + canonical /discover/app-security route
- App.tsx: converted 3 standalone Routes → Navigate redirects (preserves deep-links)
- AppSecurity.tsx, MobileAppSecurityDashboard.tsx, BrowserSecurityDashboard.tsx: prepended FOLDED comment
- Source pages preserved as-is — lazy-mounted inside hub TabsContent (no behavior loss)

Verified via Playwright (domcontentloaded):
  http://localhost:5173/discover/app-security
  - 5 real /api/v1/appsec/* calls fired (NO MOCKS)
  - Tabs: Web (SAST/DAST) | Mobile | Browser Policy
  - Screenshot: docs/ui-snapshots/ux-consolidation-app-layer-security-hub-2026-05-02.png

Persona target: AppSec Engineer (#10), Sec Architect (#11), Mobile Eng (#17), Platform Eng (#20)
Plan: docs/UX_CONSOLIDATION_PLAN_2026-04-26.md §2.10

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 2, 2026
@DevOpsMadDog @claude
beast-mode(empty-endpoints): wire/501 batch-7 of 29 — 7 more class-c …
a8a3518 
…endpoints

Closed in this batch (canonical envelope, mirroring batch-6 pattern):
- /api/v1/posture-reports/reports #7: canonical envelope shipped
- /api/v1/cloud-ir/incidents #17: canonical envelope shipped
- /api/v1/network-forensics/captures #21: canonical envelope shipped
- /api/v1/network-segmentation/segments #22: canonical envelope shipped
- /api/v1/microsegmentation/segments #23: canonical envelope shipped
- /api/v1/awareness-gamification/challenges #29: canonical envelope shipped
- /api/v1/gdpr/activities #30: canonical envelope shipped

Pattern (class-c): all seven list endpoints upgraded from minimal
{<legacy_key>, total, hint} to the canonical batch-6/batch-7 envelope:
    {
        "items": [...],
        "<legacy_key>": [...],   # back-compat (reports/incidents/captures/etc.)
        "total": int,
        "org_id": str,
        "limit": int,            # ge=1, le=500 — defaults to 50
        "offset": int,           # ge=0 — defaults to 0
        "filters_applied": {...} # echoes every filter param (None if unset)
        "hint": str              # only present when total == 0
    }

Each endpoint now (1) accepts limit + offset query params with FastAPI
ge/le validation, (2) echoes every filter back into filters_applied even
when None (no missing keys), (3) always returns the full envelope shape
even on hit (legacy clients keep their original key, new clients use
items + pagination context), (4) preserves the actionable empty-state
hint with a "this is correct for fresh tenants" framing.

Triage status update: 26/30 fully closed. 4 class-a deferred (need real
cloud creds, OAuth flows, or PAM tenant access not present in fleet —
sprint-able with customer engagement). All class-b importer-gated
endpoints (8) and all class-c structured-empty endpoints (12) now
closed.

Verified: pytest tests/test_empty_endpoints_batch7.py 11/11 PASS.
Beast Mode regression on phase4/phase7/trustgraph/pipeline_api: 170/170 PASS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 2, 2026
@DevOpsMadDog @claude
beast-mode(suite-core): fix 18 silenced imports — symbol renames per …
a4c3aa2 
…triage 248911b

18 FIX-IMPORT cases from suite-core/core triage (`docs/suite_core_silenced_imports_2026-05-03.md`).
Each broken import was silently swallowed by a try/except wrapper, masking
either a renamed/removed symbol or a never-implemented helper. Disposition
was determined per-callsite — when both the import name AND the consumer-side
method names had drifted, the safest correction is REMOVE (replace the
unreachable try-arm with the same fallback the broad-except already produced)
rather than RENAME (which would surface latent crashes through the new
broad-except). Pattern matches Wave-A in 60a8ea9 and the top-9 commit 55adab9.

Per-callsite dispositions (18 total):

  # 1 compliance_engine.py:958      get_latest_summary           REMOVED — no canonical helper
  # 2 task_queue.py:269              MicroPentestEngine           REMOVED — only Config/Result/Status
  # 3 task_queue.py:439              MicroPentestEngine           REMOVED — same
  # 4 pipeline_orchestrator.py:655   compute_exploit_probability  REMOVED — canonical compute_forecast
                                                                  has incompatible signature
  # 5 feed_correlator.py:293         abuseipdb get_by_cve         REMOVED — never implemented
  # 6 feed_correlator.py:306         otx get_by_cve               REMOVED — never implemented
  # 7 autofix_engine.py:1283         get_velocity_tracker/        REMOVED — only MaterialChangeDetector
                                     get_detector                 class exists; use class directly
  # 8 report_generator.py:337        ComplianceEngine             REMOVED — renamed to
                                                                  ComplianceAutomationEngine, no
                                                                  .get_controls method
  # 9 unified_dashboard.py:163       ComplianceEngine             REMOVED — same; no .get_summary
  #10 report_scheduler.py:526        ComplianceEngine             REMOVED — same; no .get_compliance_status
  #11 unified_dashboard.py:262       AttackSurfaceAnalyzer        REMOVED — renamed to
                                                                  AttackSurfaceMapper, returns
                                                                  Pydantic model not Dict
  #12 air_gap_bundle_engine.py:76    EmitEvent                    PRE-DONE in 55adab9 (verified)
  #13 brain_pipeline.py:881          blast_radius                 PRE-DONE in 55adab9 (verified)
  #14 graphql_schema.py:565          get_incident_manager         REMOVED — only IncidentResponseManager
  #15 graphql_schema.py:593          get_compliance_automation    RENAMED — use ComplianceAutomation class
  #16 graphql_schema.py:909          get_compliance_automation    RENAMED — same
  #17 graphql_schema.py:874          get_incident_manager,        REMOVED — neither symbol exists
                                     IncidentCreate
  #18 report_scheduler.py:499        CVEEnrichmentEngine          REMOVED — renamed to
                                                                  CVEEnrichmentService, no
                                                                  .get_recent_cves method
  #19 aws_security_hub.py:422        SecurityHubNormalizer        REMOVED — no AWS SH normalizer
                                                                  in scanner_parsers (33 vendor
                                                                  classes, none for SH)

Files touched (per-file diff stats):
  - suite-core/core/autofix_engine.py        (+10/-16)
  - suite-core/core/aws_security_hub.py      (+ 8/-28)
  - suite-core/core/compliance_engine.py     (+10/- 9)
  - suite-core/core/feed_correlator.py       (+19/-24)
  - suite-core/core/graphql_schema.py        (+38/-40)
  - suite-core/core/pipeline_orchestrator.py (+ 8/-15)
  - suite-core/core/report_generator.py      (+ 9/- 7)
  - suite-core/core/report_scheduler.py      (+15/-23)
  - suite-core/core/task_queue.py            (+22/-20)
  - suite-core/core/unified_dashboard.py     (+34/-32)

Net delta: -21 LOC.

Verified:
  - All 12 touched modules import clean (`importlib.import_module`).
  - py_compile clean on all 10 files.
  - 351/351 regression PASS on the 7 brief-specified suites
    (test_phase4/5/6/7/9 + test_pipeline_api + test_trustgraph) —
    identical to pre-edit baseline.

Cumulative suite-core silenced-import cleanup: 27/47 (top-9 in 55adab9 +
this batch of 18). Remaining: 20 INSTALL/RETIRE-DEP decisions (per-feature
judgment — quantum_crypto, llm_guard, chromadb, celery, pomegranate/mchmm/
river, sentry_sdk, GCP/PKCS11/peft/llama_cpp).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 5, 2026
@DevOpsMadDog @claude
beast-mode(qa): regression sweep #17 — all green at HEAD d65b60d
e755e17 
Suite 1 Beast Mode 13 files: 753/753 passed. Suite 2 perf -m perf broad: 194 passed, 2 skipped, 0 failed. Suite 3 OWASP/asyncio lockdown: 1/1. 0 broken collectors. Commits 667f62b (tally v2) + d751b66 (HANDOFF v9) are docs-only — zero regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog added a commit that referenced this pull request May 5, 2026
@DevOpsMadDog @claude
beast-mode(qa): regression sweep #18 — production build restored, all…
3b49bcb 
… green at HEAD 3eb988a

Beast Mode 13 files: 753/753. Perf -m perf: 194 passed, 2 skipped, 0 failed. OWASP lockdown: 47/47. 0 broken collectors. 0 regressions vs sweep #17.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

codex

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@DevOpsMadDog