Add ingestion pipeline API backed by maintained parsers#13
Merged
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting
Comment on lines
+32
to
+38
| app.state.normalizer = normalizer | ||
| app.state.orchestrator = orchestrator | ||
| app.state.artifacts: Dict[str, Any] = {} | ||
|
|
||
| def _store(stage: str, payload: Any) -> None: | ||
| logger.debug("Storing stage %s", stage) | ||
| app.state.artifacts[stage] = payload |
There was a problem hiding this comment.
The API keeps every uploaded artefact in app.state.artifacts, a single process‑wide dictionary that is updated by each request and never cleared when /pipeline/run completes. This means concurrent users or successive runs from different clients will read and overwrite the same shared state, leaking one client’s design/SBOM/CVE/SARIF data into another’s pipeline execution. Consider scoping artefacts to a session identifier or clearing the store after a run so that independent uploads cannot interfere with or access each other’s data.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Summary
- Added an
X-Session-IDheader requirement and per-session artefact store so concurrent uploads cannot leak into other runs. backend/app.pyL34-L166 - Ensured artefacts are purged once a pipeline run completes to prevent residual session data from persisting. backend/app.pyL157-L166
Testing
⚠️ pytest -q(fails: missing optional dependenciesssvc,lib4sbom,fastapi, and existing syntax errors infixops-blended-enterprisetest fixtures)
devin-ai-integration Bot
pushed a commit
that referenced
this pull request
Oct 28, 2025
…ility - Read required_frameworks from decision_tree.required_frameworks first - Fall back to overlay root for backward compatibility - Store in self.required_frameworks in __init__ to avoid repeated lookups - Update analyze() to use self.required_frameworks instead of reading overlay - Add documentation note explaining both locations are supported This fixes violations #3, #12, #13 from code review: The code was reading required_frameworks from overlay root but docs/config placed it under decision_tree. Now supports both locations with decision_tree preferred.
DevOpsMadDog
added a commit
that referenced
this pull request
Apr 28, 2026
…redential fallback
- OktaConnector: Okta Users API (paginated, Link-header), System Log API
- Normalizes users to identity shape; converts security log events
(user.account.lock, group.user_membership.add, etc.) to ALDECI findings
- Graceful needs_credentials no-op when OKTA_API_KEY/OKTA_DOMAIN absent
- 1-hour result cache; idempotent via correlation_key=okta_log|{uuid}
- Wires /api/v1/pag/accounts (triage item #13)
- 4 tests: missing-creds, mock-parse+findings, live (skipif), Link-header pagination
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 1, 2026
…se type-a empty endpoint Triage item #13 (NEEDS-CONNECTOR). Adds list_privileged_accounts_with_okta_fallback() to PrivilegedAccessGovernanceEngine: when an org has zero registered PAG accounts AND OKTA_API_KEY/OKTA_DOMAIN env vars are set, the existing OktaConnector is invoked and privileged Okta users (admin/devops/sre/owner titles OR LOCKED_OUT/SUSPENDED/RECOVERY/PASSWORD_EXPIRED status) are projected as derived PAG account rows tagged source="okta". Org-registered rows always take precedence; needs_credentials returns structured empty + hint, never mock data. Router /api/v1/pag/accounts now uses the fallback method. 6 new tests cover: needs_credentials, ok+derived, org_registered precedence, account_type filter on derived rows, sync exception handling, no-privileged-users path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 1, 2026
…+Jamf) DONE in triage doc Backlog 10 closes 2 type-a endpoints. Class (a) tally: 10 → 8 deferred. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 1, 2026
… cluster S25
Phase 3 UX consolidation, cluster S25 SBOM/Provenance sub-cluster (per
docs/UX_CONSOLIDATION_PLAN_2026-04-26.md §2.25). Folds 6 standalone
provenance/attestation pages into one tabbed hero at /comply/provenance:
tab | source page | endpoint
-------------|------------------------------|----------------------------------
export | SBOMExportDashboard | /api/v1/sbom-export/{projects,components,history}
pipeline-bom | PipelineBomDashboard | /api/v1/pbom/stats + /run/{id}/export
pbom-prop | PBOMViewer | /api/v1/pbom/artifact/{digest}/propagation
slsa | SlsaProvenanceDashboard | /api/v1/slsa/{stats,attestations,attest}
attestation | PipelineAttestationGraph | /api/v1/provenance/{artifact}/attestation
sign | SLSAAttestationSigner | /api/v1/provenance/sign
Why this cluster:
- All 6 sources verified using REAL APIs (no mocks, no fixtures)
- Tight persona blast radius — GRC Analyst (#12) + Compliance Manager (#13)
- Identical evidence/provenance shape, clean Tab pattern
- Inventory subset of S25 (SBOMInventory/Management/Dashboard/ContinuousMonitoring)
intentionally NOT included — those already route through /assets?tab=sbom (S9
Inventory hero) for the asset-centric persona path
Pattern matches UpgradePathsHub (974787c) + FinanceHub (852c780):
- New page: src/pages/SBOMProvenanceHub.tsx (lazy-imports children, ?tab= deep link)
- Source routes converted to <Navigate replace> with FOLDED comments
- App.tsx: 1 lazy import + 1 canonical route + 6 redirect updates
Verified via Playwright (chromium headless):
- Hub renders heading + 6 tabs at /comply/provenance
- 0 mock signatures in DOM
- 6/6 redirects work (/sbom-export, /pipeline-bom, /slsa-provenance,
/pbom/propagation, /provenance/attestation, /provenance/sign)
- Real /api/v1/sbom-export/*, /pbom/*, /slsa/* calls fire on tab switch
- Screenshot: docs/ui-snapshots/ux-consolidation-sbom-provenance-2026-05-02.png
Plan doc updated with DONE-2026-05-02 marker.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 1, 2026
…er S23 Maturity
Folds 3 standalone maturity dashboards into a single tabbed hero at
/comply/maturity per docs/UX_CONSOLIDATION_PLAN_2026-04-26.md §2.23
(S23 Compliance — Maturity sub-cluster).
Pages folded:
- SecurityMaturityDashboard → MaturityHub#security (/api/v1/security-maturity/{stats,assessments})
- SecurityPostureMaturityDashboard → MaturityHub#posture (/api/v1/posture-maturity/overview)
- ProgramMaturityDashboard → MaturityHub#program (/api/v1/program-maturity/domains)
Routes:
- /comply/maturity → MaturityHub (canonical)
- /security-maturity → Navigate ?tab=security
- /posture-maturity → Navigate ?tab=posture (was un-routed before)
- /program-maturity → Navigate ?tab=program
Verification (NO MOCKS gate):
- All 6 URLs (canonical + 3 ?tab= deep-links + 3 legacy redirects) load clean
- Real API calls fire on every tab: /api/v1/{security-maturity,posture-maturity,program-maturity}/*
- 0 mock signatures in DOM (Acme Corp / John Doe / lorem ipsum / MOCK_)
- 0 page errors, 0 console errors
- Screenshot: docs/ui-snapshots/ux-consolidation-maturity-2026-05-02.png
Source pages preserved with FOLDED-into-MaturityHub git-history comments;
hub re-imports them lazily so behavior, loading/error states, and form
interactions stay intact.
Persona target: GRC Analyst (#12), Compliance Mgr (#13), CISO (#1)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 1, 2026
…e 3 cluster S23 (Privacy/Controls)
Folds PrivacyGDPRDashboard, PrivacyImpactDashboard, and ControlTestingDashboard
into a single tabbed hero at /comply/privacy?tab={gdpr,impact,controls}.
Old routes (/privacy-gdpr, /privacy-impact, /control-testing) now Navigate-redirect
to the hub with the matching tab key. Source pages kept as redirect-only stubs with
"FOLDED into PrivacyComplianceHub" header comments.
NO MOCKS verified via Playwright (domcontentloaded, not networkidle):
- 11 real /api/v1/... calls fire on mount
- 0 mock signatures in DOM (MOCK_, lorem ipsum, Acme Corp, etc.)
- Screenshot: docs/ui-snapshots/ux-consolidation-privacy-compliance-2026-05-02.png
Plan section §2.23 (S23 Compliance Dashboard) updated with the
"Privacy/Controls sub-cluster: DONE-2026-05-02" backfill.
Persona impact: GRC Analyst (#12), Compliance Mgr (#13), CISO (#1).
Page count: 370 → 367 (3 routes folded).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 1, 2026
…e 3 cluster S3 Behavior
Folds 3 standalone behavior-analytics dashboards into a single tabbed hero
at /mission-control/behavior per UX_CONSOLIDATION_PLAN_2026-04-26.md §2.3:
tab | source page | endpoint
-----------|------------------------------|----------------------------------------
uba | UBADashboard | /api/v1/uba/{stats,users,events,alerts}
behavioral | BehavioralAnalyticsDashboard | /api/v1/behavioral-analytics/{anomalies,stats}
insider | InsiderThreatMonitor | /api/v1/insider-threat/{alerts,stats}
Old routes (/uba, /behavioral-analytics, /insider-threats) now redirect to the
hub with ?tab= preserving deep-links and persona muscle memory. Source pages
kept (lazy-imported) so git history + 9 real API calls verified live.
Verified via Playwright (chromium headless):
- all 4 URLs land on /mission-control/behavior?tab=<key>
- 9 unique /api/v1/* endpoints fire across the 3 tabs
- 0 mock signatures
- screenshot: docs/ui-snapshots/ux-consolidation-behavior-2026-05-02.png
Cluster #13 of Phase 3 — running total 13 hubs / 39 source pages folded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 1, 2026
…se 3 cluster S22 (Incident Extensions) Phase 3 UX consolidation §2.22 — Incident Response extensions sub-cluster folded into a single tabbed hero per docs/UX_CONSOLIDATION_PLAN_2026-04-26.md. Folded: cloud → /remediate/incidents/extensions?tab=cloud (CloudIRDashboard) breach → /remediate/incidents/extensions?tab=breach (BreachResponse) comms → /remediate/incidents/extensions?tab=comms (IncidentCommsDashboard) - New hero: suite-ui/aldeci-ui-new/src/pages/IncidentExtensionsHub.tsx Lazy-imports all 3 source pages so behavior + API calls are preserved. ?tab= deep-linking + back-sync via useSearchParams. - Legacy routes /cloud-ir, /breach-response, /incident-comms now Navigate redirect into the unified hub (preserves bookmarks). - FOLDED markers added to source files (kept on disk for lazy-import). - Persona target: IR Lead (#7), SOC T2 (#6), Crisis Comms (#13). Verified via Playwright (domcontentloaded): hero "Incident Extensions" renders, 3 real /api/v1/ calls fire screenshot: docs/ui-snapshots/ux-consolidation-incident-extensions-2026-05-02.png Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 1, 2026
…3 cluster S27 Policy Lifecycle (combined)
Folds 3 standalone policy-lifecycle pages into one tabbed hero, sibling
to the existing PolicyAuthoringHub at /comply/policies/authoring.
Authoring covers create/edit/hooks; lifecycle covers browse/inherit/stage-edit.
tab | source page | endpoint
-------------|--------------------------|------------------------------------------
library | PolicyLibraryBrowser | GET /api/v1/policies + /api/v1/policies/stats
inheritance | PolicyInheritanceView | GET /api/v1/organizations + /api/v1/policies
stage-edit | PolicyStageEditor | GET/PATCH /api/v1/policies/{id}
Route: /comply/policies/lifecycle (new)
Old routes preserved as Navigate redirects:
/policies/library -> /comply/policies/lifecycle?tab=library
/policies/inheritance -> /comply/policies/lifecycle?tab=inheritance
/policies/stage-editor-> /comply/policies/lifecycle?tab=stage-edit
Source pages stay on disk (FOLDED comment + lazy import) so they remain
in git history and the React import graph; their real-API behavior is
preserved verbatim — every fetch, error/empty state and form action.
Persona target: Policy Author (#15), Compliance Lead (#13),
Security Architect (#3)
Plan: docs/UX_CONSOLIDATION_PLAN_2026-04-26.md §2.27
NO MOCKS — all three folded pages call real /api/v1/policies and
/api/v1/organizations endpoints; the tabbed wrapper adds no client-side
data shimming.
Phase 3: 48 -> 49 hubs (combined Policy cluster split into authoring +
lifecycle for cohesion; both share the /comply/policies/* prefix).
Multica issue: deferred — multica DB has no `issues` table at HEAD
(only daemon_token + personal_access_token); backfill in next docs
pass once Multica migrations are re-run, mirroring the
ec079de / d344507 / 669b3ab pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 2, 2026
…triage 248911b 18 FIX-IMPORT cases from suite-core/core triage (`docs/suite_core_silenced_imports_2026-05-03.md`). Each broken import was silently swallowed by a try/except wrapper, masking either a renamed/removed symbol or a never-implemented helper. Disposition was determined per-callsite — when both the import name AND the consumer-side method names had drifted, the safest correction is REMOVE (replace the unreachable try-arm with the same fallback the broad-except already produced) rather than RENAME (which would surface latent crashes through the new broad-except). Pattern matches Wave-A in 60a8ea9 and the top-9 commit 55adab9. Per-callsite dispositions (18 total): # 1 compliance_engine.py:958 get_latest_summary REMOVED — no canonical helper # 2 task_queue.py:269 MicroPentestEngine REMOVED — only Config/Result/Status # 3 task_queue.py:439 MicroPentestEngine REMOVED — same # 4 pipeline_orchestrator.py:655 compute_exploit_probability REMOVED — canonical compute_forecast has incompatible signature # 5 feed_correlator.py:293 abuseipdb get_by_cve REMOVED — never implemented # 6 feed_correlator.py:306 otx get_by_cve REMOVED — never implemented # 7 autofix_engine.py:1283 get_velocity_tracker/ REMOVED — only MaterialChangeDetector get_detector class exists; use class directly # 8 report_generator.py:337 ComplianceEngine REMOVED — renamed to ComplianceAutomationEngine, no .get_controls method # 9 unified_dashboard.py:163 ComplianceEngine REMOVED — same; no .get_summary #10 report_scheduler.py:526 ComplianceEngine REMOVED — same; no .get_compliance_status #11 unified_dashboard.py:262 AttackSurfaceAnalyzer REMOVED — renamed to AttackSurfaceMapper, returns Pydantic model not Dict #12 air_gap_bundle_engine.py:76 EmitEvent PRE-DONE in 55adab9 (verified) #13 brain_pipeline.py:881 blast_radius PRE-DONE in 55adab9 (verified) #14 graphql_schema.py:565 get_incident_manager REMOVED — only IncidentResponseManager #15 graphql_schema.py:593 get_compliance_automation RENAMED — use ComplianceAutomation class #16 graphql_schema.py:909 get_compliance_automation RENAMED — same #17 graphql_schema.py:874 get_incident_manager, REMOVED — neither symbol exists IncidentCreate #18 report_scheduler.py:499 CVEEnrichmentEngine REMOVED — renamed to CVEEnrichmentService, no .get_recent_cves method #19 aws_security_hub.py:422 SecurityHubNormalizer REMOVED — no AWS SH normalizer in scanner_parsers (33 vendor classes, none for SH) Files touched (per-file diff stats): - suite-core/core/autofix_engine.py (+10/-16) - suite-core/core/aws_security_hub.py (+ 8/-28) - suite-core/core/compliance_engine.py (+10/- 9) - suite-core/core/feed_correlator.py (+19/-24) - suite-core/core/graphql_schema.py (+38/-40) - suite-core/core/pipeline_orchestrator.py (+ 8/-15) - suite-core/core/report_generator.py (+ 9/- 7) - suite-core/core/report_scheduler.py (+15/-23) - suite-core/core/task_queue.py (+22/-20) - suite-core/core/unified_dashboard.py (+34/-32) Net delta: -21 LOC. Verified: - All 12 touched modules import clean (`importlib.import_module`). - py_compile clean on all 10 files. - 351/351 regression PASS on the 7 brief-specified suites (test_phase4/5/6/7/9 + test_pipeline_api + test_trustgraph) — identical to pre-edit baseline. Cumulative suite-core silenced-import cleanup: 27/47 (top-9 in 55adab9 + this batch of 18). Remaining: 20 INSTALL/RETIRE-DEP decisions (per-feature judgment — quantum_crypto, llm_guard, chromadb, celery, pomegranate/mchmm/ river, sentry_sdk, GCP/PKCS11/peft/llama_cpp). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 5, 2026
…, all green at HEAD 8b9738e Suite 1 Beast Mode 13 files: 753/753 Suite 2 Perf -m perf: 182 passed, 2 skipped, 0 failed Suite 3 OWASP lockdown: 47/47 Spot-check test_brain_pipeline_perf::test_full_pipeline_100_findings_under_500ms: GREEN (3.54s) Total: 982 passed, 0 failed, 2 skipped — 0 regressions Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DevOpsMadDog
added a commit
that referenced
this pull request
May 5, 2026
#2) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Testing
https://chatgpt.com/codex/tasks/task_e_68de840d68c0832981f0764f747310fd