Skip to content

v0.2.0 — Epic 14 UX Avançado, Scans page, compliance redesign

Latest

Choose a tag to compare

@lichti lichti released this 25 Aug 20:06
· 45 commits to main since this release
b502478

Added

  • New /scans page (Configuration): registers and lists every CSPM scan job — job id, provider, frameworks, start/end/duration, checks completed, and a findings/assets summary (new/updated/removed, total assets, assets removed) — with status pill filters (queued/running/completed/failed, all selected by default), cursor pagination, and a "Trigger Scan" modal picking any connected provider. Finished jobs (completed/failed) show a "View Logs" button opening the raw execution log. Backend: ScanJob gained findings_new/findings_updated/findings_removed/assets_total/assets_removed/duration_seconds, computed in run_cspm_scan (workers/tasks/cspm_scan.py) — _soft_delete_stale now returns the deleted resource_ids per collection, a new _cascade_delete_findings hard-deletes findings whose resource was just soft-deleted this run (sweeping their HAS_FINDING edges), and _count_new_and_updated_findings reads the pre-existing first_seen_scan_id/scan_count fields. scan_service.py (rewritten) adds keyset-paginated list_scan_jobs (status/provider filters) and get_scan_job_logs, both scoped to CSPM jobs only (task_name starting with cspm_scan/) since scan_jobs is shared with Side-Scanning's differently-shaped documents. New GET /api/v1/scans (paginated) and GET /api/v1/scans/{job_id}/logs.
  • Semantic id/data-testid attributes and a shared page-header pattern applied across the entire frontend, adopted as the standing convention (documented in CLAUDE.md): structural elements (page root, major sections, cards/panels, tables, forms) carry an id when unique per page or a data-testid when repeated via .map(); a page's title and one-line description now live in the top bar (Header.tsx's ROUTE_LABELS/ROUTE_SUBTITLES) instead of a local <h1>/<p> duplicated in every page body.
  • Epic 14 Sprint 4, Compliance Avançado batch (UX Avançado II — score duality, section heatmap, score trend, nested requirement drill-down): a new compliance_catalog.py loads the full AWS requirement catalog for every compliance framework at runtime via the prowler package's own Compliance.get_bulk("aws") (not vendored JSON), computing the same {Framework}-{Version} slug Prowler already embeds in Finding.framework_mapping — verified byte-identical against resolve_family()'s existing slug table. GET /api/v1/compliance/frameworks/{id} (new) returns score_by_control (fraction of controls with zero failing assets, Unscored controls excluded from the denominator) alongside the pre-existing finding-count-based score, now exposed as score_by_asset, plus unscored_count (catalog controls with no PASS/FAIL finding yet for the tenant) and a section → sub-section → requirement tree (each requirement carries PASS/FAIL/UNSCORED status and a finding_key link) that backs both the new SectionHeatmap (color-coded ≥80/50–79/<50, reusing ScoreGauge's existing thresholds) and RequirementAccordion (plain React state, no shadcn — see Fixed below) components. A new compliance_score_snapshots collection, written by snapshot_compliance_scores at the end of every run_cspm_scan (any provider, upserted by tenant+framework+day — no backfill), backs GET /api/v1/compliance/frameworks/{id}/trend?period=7d|14d|1m and the new ScoreTrendChart (first real recharts usage in the project; categorical color pair validated with the dataviz skill's palette validator against the dark chart surface). The catalog — and therefore score_by_control/Unscored — is AWS-only (decision #14 scope); non-AWS frameworks still get score_by_asset and a trend, surfaced via catalog_available: false rather than a fabricated Unscored count. Follow-up in the same batch: MUTED/ACCEPTED findings, previously dropped silently by every compliance query (FILTER f.status IN ["FAIL", "PASS"]), are now tracked explicitly. GET /api/v1/compliance/frameworks/{id} and its section tree carry two parallel breakdowns — Control view (control_pass_count/control_fail_count/control_unscored_count/control_total, where an ACCEPTED-only control folds into Pass and a MUTED-only control folds into Unscored — a control's status is FAIL if any asset fails it, else PASS if any asset passes or has accepted risk, else Unscored) and Findings view (finding_pass_count/finding_fail_count/finding_accepted_count/finding_muted_count, the real per-finding-status counts, both ACCEPTED and MUTED excluded from the score_by_asset ratio) — surfaced via a "By Control / By Findings" toggle above the summary row and each SectionHeatmap row, switching client-side against data already fetched in one request rather than refetching per toggle.
  • Epic 14 Sprint 4, Attack Paths batch (UX Avançado II — hierarchical path list, View Results By toggle, path narrative): GET /api/v1/attack-paths and /stats now enrich every row with target_asset_category (server-side DOCUMENT() join + a new resource_categories.py, a direct Python port of inventoryCategories.ts kept in lockstep) and target_crown_jewel_reason (derived from real signals only — is_public/exposed_internet → internet_facing, contains_credentials/contains_secrets → stores_sensitive_data, has_admin_policy → high_privilege_identity, else manually_flagged; null when the target isn't a crown jewel), both computed at read time rather than persisted since crown-jewel status can change out-of-band between path-detection runs; both are also new list filters and stats dimensions. AttackPathList is now a 2-level expandable tree (severity → category, both expanded by default with per-level counts), and a "View Results By: Paths / Alerts / Assets" toggle on /attack-paths reorders the same underlying path list by a different second-level key (target_asset_category for Paths/Assets, detection rule for Alerts) — no new endpoint, per the decision to keep one search engine behind all three views. AttackPathCanvas gained a synthetic riskInsight node type (amber, dotted edge, ⊕/🔒 status badge) attached per-resource from the path's associated findings via a new attachRiskInsightNodes helper in graph-layout.ts. build_attack_path_docs now persists exposure (internet_facing/public_facing/trusted_access/none, via a new classify_path_exposure in graph/exposure.py), is_cross_account, is_cross_cloud_provider, and account_ids on every detected path; a new GET /api/v1/attack-paths/{path_id}/narrative returns a deterministic 4-step paginated narrative (attack_path_narrative_service.py, same template-only generated_by contract as US-14.02's ResourceNarrativeSummary — Epic 05/Ogum.AI-RAG still isn't implemented). The path detail bottom bar gained a "Path Info" section (status, created, length, copyable ID, ExposureBadge, cross-account/cross-cloud, accounts — "Status Changed" is an explicit "not available yet", there's no status-transition history on attack_paths yet) and a Take Action menu (Export path and View each resource are functional; Generate remediation PR is disabled, Epic 05 dependency). Removed the Epic 02 Sprint 2 table-based Attack Paths components (AttackPathsTable.tsx, AttackPathFilters.tsx, AttackPathDetailPanel.tsx) — dead code superseded by the React Flow canvas UI, unused by any page since the canvas redesign.
  • Epic 14 Sprint 3 (UX Avançado II — Software Inventory, Resource Compliance, Findings SLA/Exposure/Timeline): the Inventory detail panel's Software Inventory and Compliance tabs now render real data. GET /api/v1/inventory/{resource_key}/software derives Installed Packages (name, version, linked CVE ids, filesystem path when Trivy exposes it) and Licenses (SPDX id, category via a new license_catalog.py permissive/copyleft/weak-copyleft table, deprecated-id flag) from the CycloneDX SBOM already produced by Side-Scanning (Epic 03); the Applications/Running Services sub-tabs show an explicit "not available yet" state since no pipeline in the platform produces that data yet (documented gap, not fabricated). GET /api/v1/inventory/{resource_key}/compliance adds a per-resource framework selector + control table, reusing compliance_service.py's family/section resolution logic scoped to one resource's findings. On Findings: a configurable per-severity remediation SLA (GET/PUT /api/v1/settings/sla, defaults CRITICAL 7d/HIGH 30d/MEDIUM 90d/LOW 180d, new /settings/sla page) backs a GET /api/v1/findings/sla-summary aggregate panel and SLABadge wired into the findings table and detail panel (the SLA classification logic — overdue past deadline, at-risk within 20% of the remaining window — is a direct Python port of the existing frontend classifySLA helper, kept in lockstep in sla_service.py); GET /api/v1/findings/{key}/exposure-path adds a mini blast-radius graph to the finding detail panel, reusing inventory_detail_service.get_blast_radius rather than duplicating the AQL traversal; and a Timeline section shows first-detection date and a new scan_count re-scan counter. first_seen_scan_id/last_seen_scan_id/scan_count are new Finding fields, populated by a new upsert_finding() helper (workers/tasks/cloud_utils.py) that replaces the ad-hoc upsert previously duplicated in cspm_scan.py and side_scanning.py — a small, deliberately scoped pull-forward of Epic 12's US-12.06 (only the scan-history fields, not the grouped view/is_new badge/bulk-mute that story also covers).
  • Epic 14 Sprint 2 (UX Avançado II — Inventory detail panel): the Inventory detail panel is rewritten as a two-tier tab system (Info | Risk | Network | IAM | Configurations | Software Inventory | Compliance, each with sub-tabs), ~58% viewport width instead of a 420px drawer, with breadcrumb, Scan Now/Open in Console/kebab (Copy ARN/Resource ID) actions, keyboard-navigable tabs (roving tabindex, arrow keys), and deep-linkable state via ?tab=&subtab= querystring. Info > Overview now renders an AI-shaped but fully deterministic narrative summary (GET /api/v1/inventory/{resource_key}/summary — no LLM is wired up yet, Epic 05 isn't built, so this is the fallback path, not a fallback from something) with clickable deep-link chips, relationships regrouped into per-type mini-tables (replacing the old flat edge list), and an embedded blast-radius mini-graph (GET /api/v1/inventory/{resource_key}/blast-radius, outbound graph traversal capped at 3 hops) that's also the full view under Risk > Blast Radius. AttackPathCanvas gained a mode="mini" variant (reduced toolbar, no MiniMap, CenterNode for the focal resource) reused by both — no forked canvas component, per the established convention. Tabs without a data source yet (Alerts, Vulnerabilities, IAM, Network, Software Inventory, Compliance) show an explicit "not available yet" state rather than fabricated content; they land in later Epic 14 sprints per plan.
  • Epic 14 Sprint 1 (UX Avançado II — foundation): new components/ui/ primitives — RiskScoreBadge (numeric 0.0–10.0), AlertVulnQuintet, ExposureBadge, CrownJewelBadge, SLABadge (with classifySLA helper) — plus PillFilter (search + operator + count popover, apply/clear semantics) and SavedViewSelector (pinned quick-access + all-views dropdown + save-as-new). Backed by a new GET/POST/PATCH/DELETE /api/v1/views endpoint (app/api/v1/views.py, app/services/view_service.py) with three curated system views seeded per tenant and per-user scoping via the same dev-mode header pattern as X-Tenant-ID (X-User-Id — Epic 06/RBAC replaces this later). System views are read-only from the API; only the owning user can rename, edit, pin, or delete their own views.
  • docs/qa/awsgoat-detection-coverage.md: maps every vulnerability in the vendored AWSGoat modules (infra/terraform/awsgoat/) to the Ogum module/mechanism that should detect it, marking explicit gaps (application-layer SAST/DAST logic bugs, S3 object-content secret scanning, a missing PASSES_ROLE graph edge for the iam:PassRole + ec2:RunInstances escalation pattern) where nothing today does. Use it to plan a live end-to-end validation session and to prioritize backlog items.

Security

  • h2 4.3.0 (PYSEC-2026-3628, fix exists in 4.4.1) flagged by the PR-time pip-audit job: h2 arrives transitively via prowler, which hard-pins it at exactly 4.3.0 across every 5.x release — the same pin pattern as prowler's cryptography pin documented below. An explicit h2 = ">=4.4.1" floor in pyproject.toml was attempted first and made poetry resolution unsolvable (every candidate prowler version requires h2 4.3.0), so it was reverted; the advisory is now on the pip-audit ignore list with that rationale in the comment, tracked for chore/security-upgrades until prowler unpins.
  • pip 26.0.1 (PYSEC-2026-3721) flagged for the first time by the audit job: affects pip itself inside the poetry venv on the CI runner, same category as the three pre-existing pip ignores — added to the ignore list with an updated comment grouping all four.
  • langchain-core known vulnerability (PYSEC-2026-2193) with no fix in the 0.3.x line: upgraded langchain to ^1.0.0 and langchain-community to ^0.4.0 (backend/pyproject.toml), resolving to langchain-core 1.4.9 and langchain-text-splitters 1.1.2. This also removes the two prior pip-audit ignore entries (GHSA-gr75-jv2w-4656, PYSEC-2026-77) that were waiting on a langchain 1.x release to exist on PyPI — it now does. No application code depends on langchain yet (RAG/GitOps module not yet implemented), so this upgrade carries no migration risk.
  • Open Dependabot alerts against langchain (GHSA/PYSEC advisories fixed in 1.3.9+): tightened the langchain floor from ^1.0.0 to ^1.3.9 (backend/pyproject.toml) so the declared manifest range itself no longer permits a vulnerable version — the actual resolution was already patched (see above), but Dependabot evaluates the manifest range directly since this repo doesn't commit poetry.lock. Confirmed cryptography (pinned to exactly 46.0.7 by prowler across all 5.3x releases — not a range, a hard pin) and ecdsa (no fix published) cannot be bumped without breaking Prowler compatibility; both remain in the pip-audit CI ignore list, already documented with rationale. Added .github/dependabot.yml scoping version-update PRs to backend/ (pip) and frontend/ (npm) only — the vendored, deliberately-vulnerable infra/terraform/awsgoat/ fixture is excluded, and its pre-existing alert backlog was bulk-dismissed as accepted risk (test fixture, not deployed via CI).

Changed

  • Frontend dependency hygiene: removed the dead reactflow v11 entry from package.json — only @xyflow/react v12 was ever imported (the v12 migration happened without dropping the old dependency), so nothing changes at runtime and the lockfile loses a whole unused subtree. Added @playwright/test as an actual devDependency plus a test:e2e script: the three E2E specs (e2e/auth|inventory|findings.spec.ts) and playwright.config.ts existed since Sprint 4 but imported a package that was never installed, making them unrunnable as committed; tsc --noEmit now also type-checks them against the real API.
  • Dependency batch (August 2026 Dependabot triage): widened five backend ranges to their dependabot-proposed ceilings (ruff <0.16, uvicorn <0.52, pytest-cov <8, redis <7.0, mypy <3.0), refreshed agent/go.mod (cilium/ebpf 0.22.0, confluent-kafka-go 2.15.0 — also pulling cobra 1.10.2 / x/sys transitively and creating the previously-missing go.sum the CI cache path already referenced), and refreshed the frontend lockfile within existing semver ranges via targeted npm update (next 15.5.24, postcss 8.5.26, nanoid 3.3.18, @eslint/eslintrc 3.3.6). Deliberately not taken: TypeScript 7 (the Go-based compiler rewrite — needs a dedicated migration), Recharts 3 (breaking API across every chart component), tailwind-merge 3 (targets Tailwind v4; project is on v3). A blanket npm update was blocked by an unpublished transitive version (@xyflow/system@0.0.81) referenced during resolution hence the targeted form.
  • Compliance reverted to a single By Control score — the By Asset and By Findings views, and the compliance-wide default-metric setting, are removed. Three parallel scores per framework/section/control (score_by_control/score_by_asset/score_by_finding, plus a "Default Metric" selector in Compliance Settings) added confusion without adding a decision that changes a customer's behavior — one number, By Control, is the framework's stated compliance posture. compliance_service.py and ComplianceFrameworkDetail/ComplianceSectionNode (backend and frontend) dropped every _by_asset/_by_finding/finding_*_count field; ComplianceView, SectionHeatmap's toggle, and the settings default-metric field are deleted rather than deprecated. Unscored still counts toward the compliant side of score_by_control ((Pass + Unscored) / Total), unchanged from the prior formula.
  • FrameworkDetail merges the Sections and Requirements accordion into one expandable component (Sections.tsx, replacing SectionHeatmap.tsx + RequirementAccordion.tsx), sections stay open by default instead of collapsed, and the score trend chart moved above the sections list and shrank (h-48→h-32, h-24→h-16) — the trend is a supporting signal, not the primary read. The "+ Set goal" affordance on a control with no configured target is now always visible (light gray), not hover-only.
  • Clicking a scored control (Pass/Fail) in FrameworkDetail opens a two-tab drilldown panel (ControlDrilldownPanel.tsx): Findings (paginated, All/Pass/Fail filter, resource shown per finding — new resource_id/check_id exact-match filters on GET /api/v1/findings) and Assets (GET /api/v1/compliance/frameworks/{id}/control-assets, pass/fail counts per resource; expanding a row loads that resource's own findings with its own All/Pass/Fail filter).
  • Compliance Settings table: added an Enabled column (was: Listed) as the first column, ahead of Framework/Goal (was: "By Control target") — Enabled, Goal, Framework left-to-right.
  • dev adopted as the local integration trunk (ADR-014): work branches now branch off dev and merge into it locally (--no-ff, no push, no per-branch CI); a single batched dev → main PR triggers the full CI matrix once instead of once per small branch. .github/workflows/ci.yml gained a push: branches: [main, dev] trigger and if: github.event_name == 'pull_request' || github.ref == 'refs/heads/main' on the four heavy jobs (docker-build, backend-integration, backend-openapi-drift, backend-security), so pushes to dev only run the lightweight jobs (backend-lint, backend-test unit, frontend-check, agent-check, ~2–3min, no Docker/ArangoDB/Redis). main remains protected — no direct commits, full CI still gates every merge into it.
  • infra/terraform/awsgoat/module-1: fixed 5 terraform validate deprecation warnings on aws_api_gateway_deployment.apideploy_ba — surfaced while validating the AMI pin change above. Setting stage_name/variables directly on an aws_api_gateway_deployment is deprecated by the AWS provider; split into a dedicated aws_api_gateway_stage.apideploy_ba resource (mirroring the pattern already used elsewhere in the same file for aws_api_gateway_stage.api) and repointed the 4 .invoke_url references in null_resource.file_replacement_api_gw/_cleanup from the deployment to the new stage resource. terraform validate now returns zero diagnostics for this module.
  • infra/terraform/test-fixtures: pinned the create_ec2_instances-gated instances (public_exposed, private_clean) to a fixed Debian 11 AMI (ami-053413bdacb39d8dc, debian-11-amd64-20241202-1949) via new ec2_instances_ami_id variable, instead of a data "aws_ami" lookup that resolved to whatever the latest Amazon Linux 2023 build happened to be on a given terraform apply — reproducible side-scanning test runs need a fixed OS/package baseline, not a moving target. Region-specific; re-pin if aws_region changes. The DVWA instance in vulnerable_apps.tf still uses the dynamic Amazon Linux lookup (unrelated flag, out of scope for this pin).
  • Documented the two-tier tab detail panel, embedded exposure/blast-radius mini-graphs, and pill-filter/saved-Views pattern as the standard frontend convention for resource, finding, and path detail views (CLAUDE.md); added corresponding roadmap items to Phase 2 (README.md).

Fixed

  • OpenAPI drift gate failing on the first PR that exercises it (docs/api/openapi.json): the schema was last regenerated before Epic 14 Sprints 3–4, the compliance redesign, and the Scans page batch landed — every endpoint added since (GET /scans, /scans/{id}/logs, /views, /settings/sla, /findings/sla-summary, /findings/{key}/exposure-path, /compliance/frameworks/{id} + /trend + /control-assets, /attack-paths/{id}/narrative, /inventory/{key}/software + /compliance + /summary + /blast-radius) was missing from the committed spec. Regenerated via scripts/export_openapi.py; the drift job is green again.

  • CI red on dev after the Scans page batch: two failures in the lightweight jobs. (1) mypy failed on scan_service.py:52 (Returning Any from function declared to return "list[str] | None") — doc.get("logs", []) on a dict[str, Any] returns Any, which the new warn_return_any config correctly flags; fixed by binding through an annotated local (logs: list[str] = doc.get("logs") or [], the same pattern already used in ciem_service.py). (2) The new ScanTable.test.tsx row-render assertion used the loose regex /10/, which matched both the findings-total span and the started-at timestamp cell (10:00:00 via toLocaleString) once real fixture data landed; replaced with exact-match assertions (/^10$/) for both counters.

  • Invalid mypy option in pyproject.toml silently skipped since introduction: disallow_redundant_casts = true is not a valid mypy setting (the flag has always been --warn-redundant-casts; there is no "disallow" variant) — every run printed an "Unrecognized option" warning that was easy to miss in CI noise while the intended check never ran. Renamed to warn_redundant_casts = true; full-codebase mypy confirmed clean under the now-active option.

  • Disconnecting a provider silently left its findings behind (_purge_provider_resources, provider_service.py): the findings-deletion filter used doc.provider_id == @provider_id, but Finding has no provider_id field — only provider + account_id/cluster_name — so the filter never matched anything and every disconnected provider's findings survived the "delete" untouched. Found while cleaning the dev database for a fresh scan; the test covering cascade-delete had seeded a Finding with an artificial provider_id that real ingestion never writes, masking the bug. Fixed by scoping findings the same way vertices already are (provider + account_id/cluster_name); also added STS_ASSUMEROLE_ALLOW, ATTACHED_POLICY, and HAS_FINDING to the orphaned-edge sweep, which previously missed them entirely.

  • GET /api/v1/scans 500ing against real data: scan_jobs is shared between CSPM jobs (task_name: cspm_scan/*, has frameworks) and Side-Scanning jobs (task_name: side_scan/*, no frameworks field at all) — ScanJob(**doc) raised a Pydantic ValidationError on the first side-scan document it hit. Fixed by filtering to STARTS_WITH(j.task_name, "cspm_scan/") in scan_service.py's AQL and in get_scan_job/get_scan_job_logs's Python-side check — Side-Scanning already has its own dedicated page, so CSPM-only is the correct scope here, not a workaround.

  • resource_type: "unknown" on findings/inventory: traced to prowler_service.py::_normalize, which falls back to the literal string "unknown" (an Ogum-invented default, not a Prowler value) whenever result.metadata is falsy. compliance_service.py's asset-grouping query now prefers a real resource_type over "unknown" when both are present for the same resource (SORT g.f.resource_type == "unknown" ASC). Also manually removed 17 orphaned check_id == "unknown" findings left over from two stale scan runs superseded by correct re-scans.

  • EBS snapshot SBOM generation returning empty on real AWS accounts (Epic 03): _generate_sbom (app/workers/tasks/side_scanning.py) ran trivy vm ebs:{snapshot_id} --format cyclonedx immediately after run_trivy_ebs had already read the same snapshot's blocks through the same trivy-server sidecar for the vuln/secret scan — a second, back-to-back ebs:GetSnapshotBlock/ListSnapshotBlocks pass over the same snapshot is prone to EBS Direct API throttling, which Trivy surfaces as exit 1 with empty stdout rather than a distinguishable rate-limit error. This was flagged as a known, unfixed residual gap when scan_ec2_instance_v2 was first verified end-to-end against a real account. Confirmed by manually re-running the same trivy vm ebs:... --format cyclonedx command in isolation (no prior read of the same snapshot) against a real snapshot, which succeeded every time. Fixed by retrying _generate_sbom with linear backoff (3 attempts, 5s/10s waits) instead of treating the first failure as final; still best-effort and never fails the parent scan.

  • Terminated EC2 instances resurrected as status: "active", permanently breaking "Scan Now" (app/services/prowler_inventory.py): found live in dev-tenant — two public-exposed/private-clean test-fixtures EC2 instances, replaced by a terraform apply days earlier, kept showing as active inventory with a "Scan Now" button that always failed 422 ("Resource could not be resolved to a scannable target"). Root cause: AWS's describe_instances (and therefore Prowler's own EC2 collector) keeps reporting a terminated instance for a limited window after termination — extract_inventory_from_findings unconditionally wrote status: "active" for every resource Prowler reported, with no check against the EC2 instance's own state field. This silently undid a correct prior soft-delete (the stale doc still carried a deleted_at from days earlier) the next time a scan happened to catch the instance inside that post-termination window, and left it permanently stranded as "active" once AWS's window elapsed and Prowler stopped reporting it too — at that point _soft_delete_stale (app/workers/tasks/cspm_scan.py) never gets a chance to mark it deleted, since that only fires for resources absent from the current scan, not ones that were merely absent from a past one. Fixed by skipping ec2_instance findings with raw_metadata.state == "terminated" during extraction entirely, so _soft_delete_stale's existing absence-based logic catches them on the very next scan instead of a second code path fighting it. Manually corrected the two stale documents found live during this investigation.

  • AWSGoat terraform apply failed on macOS (infra/terraform/awsgoat): terraform validate/plan don't execute provisioners, so this only surfaced on a real local terraform apply. Upstream's local-exec provisioners assume a GNU userland (their own CI runs on Ubuntu): (1) sed -i 'script' file with no attached suffix is GNU-only — BSD sed (macOS's default /usr/bin/sed) requires an in-place suffix argument and fails with sed: -I or -i may not be used with stdin otherwise. Fixed across all 9 sed -i call sites (module-1 and module-2) with the portable sed -i.bak 'script' file && rm -f file.bak idiom, accepted identically by BSD and GNU sed. (2) module-2's RDS_URL=$${RDS_URL::-5} uses bash ≥ 4.2 negative-length substring syntax; macOS ships bash 3.2 (frozen there for GPL licensing reasons) and fails with substring expression < 0. Replaced with $${RDS_URL%:*} (strip from the last : onward), which is bash-3.2-safe and also correct regardless of port digit count, unlike upstream's hardcoded "last 5 characters" assumption. Also documented a pre-existing, unrelated blocker in infra/terraform/awsgoat/README.md: module-1's populate_table provisioner needs boto3 on python3 — not vendored or auto-installed, now called out as a prerequisite.

  • EC2 side-scanning failed against every real AWS account (Epic 03): end-to-end browser verification of the "Scan Now" trigger against a live dev/test AWS account (not moto) surfaced a chain of issues that no prior test caught, because nothing had ever exercised the trigger against a real account before. (1) _get_aws_session (cloud_utils.py) always built the STS client used for AssumeRole from ambient credentials, ignoring any static aws_access_key_id/aws_secret_access_key also present on the tenant's provider config — worked in production (EC2 instance profile provides ambient identity) but failed with NoCredentialsError in any environment without one, including local dev. Now signs the STS client with the static keys when both are provided, falling back to ambient credentials otherwise. (2) enqueue_side_scan() resolved external_id for its own resolve_ec2_scan_metadata session but never passed it to the scan_ec2_instance_v2.delay()/scan_lambda_function.delay() calls, and those task signatures didn't even accept the parameter — so the worker's own AssumeRole call always omitted ExternalId, causing a persistent AccessDenied against any role whose trust policy requires one (confused-deputy protection), even after IAM permissions were otherwise correct. This looked like AWS IAM eventual-consistency lag at first; it was a reproducible code bug. Fixed by threading external_id through all three task signatures. (3) The trivy binary was never installed in docker/backend.Dockerfile at all — the whole Sprint 2 sidecar architecture assumed it was present. Added the official install script alongside the existing dependency install step. (4) Once trivy was present, every call site (run_trivy_ebs, run_trivy_rootfs, run_trivy_image, SBOM/SARIF generation, plus the dead-code scan_with_ebs_direct) used the pre-0.72 syntax trivy client --server <url> vm/rootfs/image/sbom ... — current Trivy releases only support client for image scanning (trivy client [flags] IMAGE_NAME); vm, rootfs, image, and sbom are top-level subcommands that each take --server directly, with no client wrapper. The old form FATALed immediately with unknown flag: --server. Fixed across all 8 call sites. With all four fixes applied, scan_ec2_instance_v2 completed successfully against a real EC2 instance and EBS snapshot for the first time this project has been able to verify. Known residual gap, not fixed here: SBOM generation for EBS snapshots (_generate_sbom) still returns nothing on a real account — the underlying vuln/secret scan against the same snapshot in the same task run succeeds, so this looks like an EBS Direct API rate limit or a quirk of Trivy's [EXPERIMENTAL] vm mode rather than a syntax issue; SBOM generation is best-effort by design and does not fail the scan.

  • Side-scanning jobs untrackable after the manual/auto trigger (Epic 03): browser-testing the newly-shipped "Scan Now" button surfaced two bugs no prior test caught because nothing had ever populated real side-scanning job data before. (1) scan_ec2_instance_v2/scan_lambda_function ignored the job_id the caller passed in and minted their own via start_discovery_job() — unlike scan_k8s_container/scan_container_image, which correctly reuse it. enqueue_side_scan()'s pre-created scan_jobs doc (the one returned to the UI) was silently orphaned at status: "queued" forever, while a second, disconnected doc with a different key received the real status. Fixed by reusing the passed job_id (scan_job_id = job_id; update_job_to_running(db, scan_job_id)), matching the K8s/ECR convention. (2) /side-scanning's page.tsx read data.items/data.total directly off the useQuery result, but sideScanApi.listJobs() returns a raw Axios response — the payload is one level deeper at data.data.items/data.data.total. The ?? [] fallback made "wrong shape" indistinguishable from "genuinely zero jobs," so the jobs table, KPI cards, and job count have shown nothing for every job type since Sprint 4 (PR #156), not just EC2/Lambda — the existing test's mock skipped the Axios wrapper too, so it couldn't have caught this either. Both fixes include regression tests exercising non-empty job data, which no test in this area did before.

  • EC2 side-scanning silently stopped detecting malware/webshells (Epic 03): Sprint 2's move to EBS Direct API (scan_ec2_instance_v2) never mounts a filesystem, but YARA needs actual file access — it can't scan raw disk blocks. The Sprint 1 task that did mount and run YARA (scan_ec2_instance) was left in the codebase but never wired to any trigger after Sprint 2 shipped, so malware detection silently went inactive with no error, no log, no test failure. Found while auditing Epic 03 documentation against the shipped code. Fixed by adding an optional scoped-mount step to scan_ec2_instance_v2: when OGUM_SCANNER_INSTANCE_ID and an availability zone are available, it creates a volume from the same snapshot, attaches+mounts it read-only to the scanner's own instance, and runs YARA in parallel with the EBS Direct API vuln+secret scan — always unmounting/detaching/deleting the volume in finally, and skipping (with a log line, never silently) when the scanner isn't running on EC2. Trivy (vuln+secret) remains the primary analysis engine — its failures still propagate and fail the task; YARA is best-effort and never blocks the primary scan. The now-fully-redundant scan_ec2_instance (Sprint 1 task) was removed, along with its only-used-there _normalise_cve/_normalise_secret helpers; cleanup_orphan_snapshots (still active) keeps its own test coverage in a trimmed test_side_scanning_task.py. Also fixed a related bug never previously exercised: the volume detach_volume call was missing InstanceId, which only surfaced once a real attach+detach cycle was covered by a test.

  • Every Prowler CSPM finding classified as MEDIUM severity, no INFORMATIONAL findings ever surfaced: ProwlerService._normalize computed severity_str = str(getattr(check_meta, "Severity", "medium")).lower(). prowler.lib.check.models.Severity — the real, required type of CheckMetadata.Severity — subclasses (str, Enum) without overriding __str__, the same shape as the Status bug fixed earlier (see below): str(Severity.high) returns "Severity.high", not "high". That string never matched _SEVERITY_MAP's keys, so every finding silently fell through to the SeverityLevel.MEDIUM default regardless of its real severity — found after a full-catalog scan against the test AWS account showed all 407 findings as MEDIUM with zero at any other level. Fixed by reading severity_obj.value instead of str(severity_obj), mirroring the existing status_obj.value fix. Same root-cause class as the Status bug, different enum, undetected until now because the existing test suite's _make_result helper passed a plain Python string for Severity rather than the real enum member every check actually carries at runtime.

  • Attack path detection returning zero results (Epic 02): find_privilege_escalation_paths and _detect_tc03_overpermissioned_to_db in attack_path_service.py were traversing ASSUMES_ROLE edges (resources→identities) for identity→identity paths. Corrected to STS_ASSUMEROLE_ALLOW, ASSUMES; TC-03 traversal now uses OUTBOUND i STS_ASSUMEROLE_ALLOW, ASSUMES, STORES_SENSITIVE_DATA. End-to-end result: 7 attack paths detected in the Terraform test environment (was 0).

  • STORES_SENSITIVE_DATA key mismatch with Prowler-created identities (Epic 02): _create_data_access_edges built edge _from references using arango_key() (structured key). When Prowler CSPM creates an identity document first (hash key), the discovery _upsert updates it in-place, keeping the hash key. The in-memory structured key produced dangling edge _from references — TC-03 AQL traversal found no outbound edges and returned zero results. Fixed: _create_data_access_edges now queries identities from ArangoDB with FILTER has_admin_policy == true, resolving actual _key values regardless of insertion path (discovery vs. Prowler).

  • Attack path STORES_SENSITIVE_DATA edge coverage (Epic 02): _create_data_access_edges now queries the full data_assets collection instead of only S3 buckets from the current discovery run. DynamoDB tables, SecretsManager secrets, and any other data assets discovered by Prowler CSPM now correctly receive edges from admin IAM identities, enabling TC-03 detection.

  • EC2 → IAM role ASSUMES_ROLE edge (profile name mismatch): _create_resource_edges now uses _build_profile_role_map (one paginated list_instance_profiles call) to resolve instance profile names to role arango keys. Previously the lookup silently failed when the profile name differed from the role name (e.g., foo-profile vs foo-role).

  • S3 is_public detection: _list_s3_buckets now calls get_public_access_block per bucket and sets is_public=true when any block setting is disabled. Buckets with no public access block configuration (the default prior to 2023) are also marked is_public=true.

  • Array query params silently ignored by the backend (apiClient): axios's default paramsSerializer encodes arrays as key[]=a&key[]=b, but FastAPI's Query(list[str]) only recognizes repeated bare keys (key=a&key=b) — the bracketed form is a different, unrecognized param name, so the filter silently defaulted to "no filter" instead of erroring. Found while browser-testing the new inventory multi-select filters (request succeeded with HTTP 200 but returned unfiltered results). Fixed with a custom paramsSerializer on the shared apiClient (frontend/src/lib/api.ts) that emits repeated bare keys for array values. This also fixes the same latent bug in POST /inventory/discover?regions=..., which had no test coverage catching the wire format.

  • Compliance page listing thousands of duplicate "frameworks" instead of ~45 (Epic 01): compliance_service.get_compliance_summary grouped findings by the raw framework_mapping string, which Prowler stores as {framework_slug}/{control_id} (e.g. "CIS-7.0/1.1"). Grouping on the full string turned every individual control into its own top-level "framework" — 2,506 sidebar entries in the dev dataset instead of the 45 real frameworks. Fixed by splitting prefix and control before aggregating (backend/app/services/compliance_frameworks.py + rewritten compliance_service.py).

  • Findings framework filter dropdown never matched any real data: FindingFilters.tsx hardcoded framework values ("CIS-AWS-2.0", "PCI_DSS", "NIST_800_53", ...) that don't correspond to any string Prowler actually writes to framework_mapping — selecting any of them silently returned zero results. Replaced with real slugs (CIS-7.0, PCI-4.0, NIST-800-53-Revision-5, ...).

  • Findings framework filter only matched exact strings: a link like /findings?framework=CIS-7.0 (family/version id, no control suffix) never matched anything, since framework_mapping stores the full "CIS-7.0/1.1" string. list_findings's framework filter now also matches by prefix (fw == sel OR STARTS_WITH(fw, sel + "/")).

  • Compliance "Top Failing Controls" ignored the selected framework: the list was always computed globally regardless of which framework card was selected in the sidebar. GET /api/v1/compliance/summary now accepts an optional ?framework= to scope it.

  • identity_type/asset_type missing on CSPM-enriched identities and data assets (Epic 00): prowler_inventory.py::extract_inventory_from_findings wrote a generic resource_type field regardless of which ArangoDB collection a document was routed to, but Identity requires identity_type and DataAsset requires asset_type — neither is resource_type. In the dev dataset, 83% of identities and 100% of data assets created this way were missing their required field. This silently broke attack path TC-02 detection, which filters data_assets by resource_type and therefore only ever matched CSPM-enriched assets, never natively-discovered ones. Fixed by classifying identity_type (role/user/group/service-account) and setting asset_type = type_name per collection.

  • Every Prowler CSPM finding classified as FAIL, even PASS results (Epic 01): ProwlerService._normalize computed status_str = str(getattr(result, "status", "FAIL")).upper(). prowler.lib.outputs.common.Status subclasses (str, Enum) but does not override __str__, so str(Status.PASS) returns "Status.PASS" — not "PASS". That string never matched _STATUS_MAP's keys, so every result (PASS, FAIL, MANUAL alike) silently fell through to the FindingStatus.FAIL default. Root-caused after noticing 100% of findings in the dev dataset were FAIL, including checks against resources the Terraform test fixtures document as compliant baselines (e.g. s3_bucket_public_access reporting FAIL for a bucket with all public access blocked). Fixed by reading status_obj.value instead of str(status_obj). A re-scan of the CIS-AWS-2.0 framework against the test account went from 111/111 FAIL to 60 FAIL / 51 PASS. This bug predates and is unrelated to the Compliance/Findings PRs above — only Prowler CSPM findings were affected; Checkov/IaC findings pass FindingStatus enum members directly and were never affected.

Added

  • Admin Jobs: job name, start time, duration, and captured logs (Epic 06): /admin/jobs only showed job ID, tenant, status, provider, created-at, and worker — no way to tell what a job actually did or how long it took, and clicking a row did nothing. Rows are now clickable, opening a detail panel (JobDetailPanel, mirroring the existing Inventory DetailPanel slide-over pattern) with a human-readable job name, created/started/completed timestamps, duration, checks/findings counts, error message (if failed), and a log viewer. The list table gained Job Name, Started At, and Duration columns. Backend: run_cspm_scan now attaches a JobLogHandler (app/workers/tasks/job_logging.py) to the root logger for the scan's duration, capturing its own logging plus anything that propagates through it (Prowler, boto3 at WARNING+) into the job doc's logs field (bounded to the last 500 lines) — the JobDetail.logs field and its SSE streaming endpoint already existed but nothing had ever populated them. Side-scanning (ec2/lambda) and discovery jobs don't have log capture wired yet — their detail panel shows "No logs available for this job" rather than an error. Along the way, fixed a job-naming bug found while building this: ScanJob had no task_name field, so every CSPM-scan-shaped job doc — including IaC scans (provider="iac") and, after a fallback fix, side-scan (ec2/lambda) docs that had no provider field at all — displayed as cspm_scan/* regardless of what it actually was. ScanJob and the side-scan job-doc writer (app/services/side_scanning/trigger.py) now set task_name explicitly (cspm_scan/{provider}, iac_scan/iac, side_scan/ec2, side_scan/lambda); admin_service._infer_task_name provides the same correction for already-persisted docs from before this fix.
  • AWSGoat vendored and unified into test-fixtures as a single-apply, end-to-end vulnerable-app test target: infra/terraform/awsgoat/module-1 (serverless blog: Lambda + API Gateway + DynamoDB + S3, XSS/SQLi/IDOR/SSRF chained into IAM privilege escalation via IMDS credential theft) and module-2 (ECS/Fargate HR payroll app, SQLi/file-upload chained into container breakout + IAM privilege escalation), vendored from ine-labs/AWSGoat (MIT, commit b24869a) along with its attack manuals and minimum deploy IAM policy. Complements test-fixtures/ (synthetic, no running app) by exercising the full chain Ogum needs to detect end-to-end — Ogum.Dynamic catching the app-layer vulnerability, Ogum.Pulse/CDR the resulting cloud API activity, Ogum.Graph the reconstructed attack path — rather than only CSPM misconfigurations. Both modules are wired into test-fixtures/awsgoat.tf as toggleable child modules (create_awsgoat_module1/create_awsgoat_module2, both off by default) so terraform apply from test-fixtures/ deploys everything from one place instead of three separate directories/states. Making them child modules required three documented local patches on top of upstream (see infra/terraform/awsgoat/README.md#local-patches-applied-on-top-of-upstream): dropping each module's own provider "aws" block (inherits test-fixtures' ~> 5.0 provider instead — module-2 was previously pinned to ~> 3.27, validated clean under 5.x with only deprecation warnings, no breaking changes), prefixing module-1's local file references with ${path.module} (they resolved against the root's working directory, not the module's, once this stopped being a standalone root), and renaming an S3 bucket both modules declared with the identical literal name. Deliberately excludes upstream's GitHub Actions auto-deploy workflows (manual terraform apply only, consistent with this repo's no-CI-cloud-deploy posture). See infra/terraform/awsgoat/README.md for safety notes — these are real, deliberately internet-reachable exploitable applications, not simulated findings, and still require a dedicated disposable sandbox account.
  • Known-vulnerable test fixtures: Metasploitable + DVWA (infra/terraform/test-fixtures): two opt-in EC2 instances (create_metasploitable_ec2, create_dvwa_ec2, both false by default) for validating Ogum.Dynamic side-scanning against real CVEs and exposed secrets, complementing the synthetic CSPM misconfigurations already in main.tf. DVWA runs the official vulnerables/web-dvwa Docker image via user_data — no custom AMI needed. Metasploitable requires a self-built AMI (Rapid7 doesn't distribute a public one; license restricts it to isolated local VM use), looked up by tag or supplied directly via metasploitable_ami_id. Both instances deploy into the private subnet behind a new security group with no default ingress — vulnerable_apps_allowed_cidrs (empty by default, 0.0.0.0/0 blocked by a Terraform validation rule) is the only way to open network access; otherwise reachable only via SSM Session Manager (IAM role/instance profile attached automatically).
  • EC2/Lambda side-scanning trigger — manual and automatic (Epic 03): /side-scanning was empty in every environment, including production — nothing ever called scan_ec2_instance_v2/scan_lambda_function. K8s and ECR scans only fire from external webhooks (a real DaemonSet, a real ECR push); EC2 and Lambda had no equivalent, no UI action, and no automatic trigger after discovery. POST /jobs/{id}/retry for ec2/lambda jobs was also a silent no-op (else: pass). New POST /api/v1/side-scans/trigger ({resource_key}) — surfaced as a "Scan Now" button in the Inventory detail panel for active ec2_instance/lambda_function resources — and a first-seen auto-trigger hooked into run_cspm_scan (AWS only): each EC2/Lambda resource gets exactly one side-scan the first time it's discovered, deduplicated against existing scan_jobs entries, never re-triggered periodically (keeps EBS snapshot cost and Celery/trivy-server load bounded — later re-scans are manual or via the already-free daily rescan_sboms). Retry for ec2/lambda jobs now actually re-enqueues. All three call sites (manual trigger, retry, auto-hook) share one implementation in app/services/side_scanning/trigger.py. Along the way, confirmed directly against the installed prowler-core package (not just its docs) that its EC2 Instance model has no availability_zone or volume/block-device fields at all — volume_id is required just to create the scan snapshot, so the trigger makes one batched ec2:DescribeInstances call (resolve_ec2_scan_metadata) at scan time. This doesn't reintroduce a second discovery pipeline (the "Prowler as single source of truth" rule applies to inventory sync, not to the act of scanning itself).
  • Prowler as the single source of truth for Inventory — Part 1/3 (Epic 00): first step of retiring the native AWS discovery pipeline (discovery.py) in favor of building the resource graph entirely from Prowler scan output, avoiding two parallel, schema-incompatible inventory paths. ProwlerService.run_*_scan now accept frameworks: list[str] | None = None — omitting frameworks (the new default for the eventual scheduled scan) runs prowler's full check catalog for the provider instead of a curated 3-framework subset, since Scan() with no compliances= filter already runs every check and each result still tags itself with every framework it belongs to. extract_inventory_from_findings now extracts relationship data already present in Prowler's resource_metadata (EC2 security_groups/subnet_id/iam_instance_profile, IAM trust_policy/policies) into the fields graph/iam_edges.py already reads with zero AWS API calls — so IAM deep-model edges (STS_ASSUMEROLE_ALLOW, ASSUMES, ATTACHED_POLICY) work correctly off Prowler-sourced identities without any change to that module. One confirmed gap: prowler's Lambda Function model has no execution-role field at all — closed with a small targeted lambda:list_functions call (reusing Prowler's already-resolved credentials) rather than a broader rediscovery pass. VpcSubnet.public (route-table-derived) is now also recognized as an internet-exposure signal, a more accurate source than the map_public_ip_on_launch flag native discovery used to rely on. Parts 2/3 (decoupled resource-edge builder, replacing _create_resource_edges's in-memory coupling to live discovery) and 3/3 (repoint all call sites, retire discovery.py and its Celery Beat schedule) land separately.
  • Prowler as the single source of truth for Inventory — Part 2/3 (Epic 00): new graph/resource_edges.py::build_resource_edges rebuilds BELONGS_TO (EC2→VPC via subnet), ATTACHED_TO (SG→EC2), and ASSUMES_ROLE (EC2 and Lambda→IAM role) purely by querying resources/identities already persisted in ArangoDB — no in-memory coupling to a live discovery run, unlike the _create_resource_edges it replaces. MEMBER_OF (IAM User→Group) is also rebuilt this way; confirmed against prowler-core that iam_administrator_access_with_mfa/iam_group_administrator_access_policy audit AwsIamGroup as their primary resource, so Group.users is actually populated by real scans, not just a theoretical field. _create_data_access_edges (STORES_SENSITIVE_DATA) moved unchanged to graph/data_access_edges.py. run_cspm_scan now runs build_iam_edges + build_resource_edges + build_data_access_edges + compute_exposed_internet after every inventory upsert, and soft-deletes resources/identities/data_assets absent from the scan (scoped to tenant+provider — a single Prowler run covers every resource type in one pass, unlike the old per-type/per-region soft-delete). Two accepted gaps carried over from Part 1: ROUTES_TRAFFIC (Internet Gateway→VPC) has no equivalent in prowler-core (no InternetGateway model) and is not rebuilt; EC2 instance-profile→role resolution is name-based only (the old live iam:ListInstanceProfiles lookup for mismatched names is not reintroduced). New gap found while wiring this up: prowler's IAM Role.inline_policies is list[str] of names only (no document content), so the inline-policy wildcard-action detection discovery.py used for dangerous_permissions has no equivalent — has_admin_policy is still derived correctly (managed-policy ARN suffix match, e.g. AdministratorAccess/PowerUserAccess), only the inline-policy path is gapped. Also fixed a pre-existing, previously-undetected bug found while testing this: prowler_inventory.py's AWS collection-routing regex matched "Group"/"Role" as bare substrings, misrouting AwsEc2SecurityGroup, AwsAthenaWorkGroup, AwsAutoScalingAutoScalingGroup, AwsLogsLogGroup, and AwsWafRuleGroup/AwsWafRegionalRuleGroup into the identities collection instead of resources — silently broke ATTACHED_TO edges for every EC2 instance with a security group. AWS routing now matches by service prefix (^Aws(Iam|RolesAnywhere)) instead of bare-word matching.
  • Prowler as the single source of truth for Inventory — Part 3/3 (Epic 00): repointed every AWS discovery call site to run_cspm_scan — provider registration and re-trigger (providers.py::_dispatch_discovery), the ad-hoc /inventory/discover endpoint, the admin panel's TaskType.DISCOVERY trigger, and trigger_job's retry path. POST /api/v1/scans and the scheduled Celery Beat scan (trigger_all_cspm_scans) now default to frameworks=None (full check catalog) instead of a curated subset (CIS-AWS-2.0, or ["CIS-AWS-2.0", "PCI_DSS_v4", "SOC2"] depending on the entry point) — the whole point of running the full catalog is that every check still tags its result with every framework it belongs to, so a curated subset was strictly less complete for no benefit. trigger_all_cspm_scans also stops skipping k8s/kubernetes providers (Prowler does support Kubernetes CSPM scans; the old skip was a leftover from the curated-framework-per-provider table). Celery Beat's dev-trigger-aws-every-6h entry (and the 1h offset on cspm-scan-all-tenants-every-6h that existed to let it finish first) are both removed — a CSPM scan is now the entire discovery pass, so there's nothing left to wait on. ConnectWizard.tsx's "Connecting..." copy now sets the expectation that this runs a full scan (minutes, not moments) instead of implying a quick API listing call.
  • Manual QA test scenarios (docs/qa/, Epic 13 US-13.08): structured pre-conditions → steps → expected result → what-to-check-if-it-fails scenarios for the flows automated tests don't fully cover — provider onboarding, a full CSPM scan staying consistent across Findings/Inventory/Compliance/Attack Paths at once (including direct regression checks for the Status/Severity enum-stringification bugs fixed above), the attack-path graph UI, side-scanning (documents that EC2/Lambda scans have no HTTP trigger endpoint yet, only K8s and ECR webhooks do), and the admin panel. docs/scanning.md updated to reflect frameworks=None (full catalog) as the default instead of the old curated 3-framework example.
  • API reference via Swagger UI without the full stack (Epic 13 US-13.07): docs/api/openapi.json — the FastAPI-generated OpenAPI schema — is now committed to the repository, generated by backend/scripts/export_openapi.py (pure route introspection, no ArangoDB/Redis/network access needed since every Settings field has a default). New swagger-ui service in docker-compose.yml (swaggerapi/swagger-ui, port 8085) serves it standalone — docker compose up -d swagger-ui with nothing else running. New CI job Backend — OpenAPI Schema Drift runs the export script in --check mode and fails the build if the committed schema doesn't match what the current code generates, so this can't silently go stale the way docs/scanning.md's framework example did.

Removed

  • backend/app/workers/tasks/discovery.py: AWS discovery is retired — discover_aws, discover_aws_basic, all _list_* AWS API functions, _create_resource_edges, _create_data_access_edges (moved to graph/data_access_edges.py), _build_profile_role_map, _classify_iam_role, and retry_with_backoff (only used by the now-removed _list_* functions) are gone. Genuinely cross-cutting helpers used by other providers' discovery tasks and by side-scanning — _get_aws_session, _get_tenant_db, _upsert, _mark_stale_deleted, _set_provider_status — moved unchanged to the new backend/app/workers/tasks/cloud_utils.py. Azure, GCP, and Kubernetes discovery tasks are untouched by this change.

  • Compliance page redesign: unified framework families with drill-down sections (Epic 01): replaces the flat, unusable framework list with family/version grouping — near-duplicate framework versions (CIS AWS Foundations Benchmark ships 8 slugs, NIST 800-53 has 2 revisions, ISO 27001 has 2 editions, PCI DSS has 2 versions, KISA ISMS-P has English/Korean slugs, FedRAMP has 3 profiles) now collapse into one sidebar entry with a version switcher, cutting the dev dataset's sidebar from 2,506 entries down to 31. Controls within a version are grouped into sections — NIST 800-53 uses its standard 2-letter control family codes (AC, CM, IA, ...), CIS/ISO27001/PCI use their dot-numbered section prefixes (backend/app/services/compliance_frameworks.py::derive_section). New FrameworkSidebar (searchable), FrameworkDetail (version tabs + section breakdown + "View findings" deep link), and TopFailingControls (now scoped to the selected framework) components replace the single-file page.

  • Findings: multi-select filters and real stats-backed summary (Epic 01): /findings page's severity/status/provider/framework/source dropdowns are now checkbox multi-selects (reusing components/ui/MultiSelectFilter), and the summary cards now show real counts from GET /api/v1/findings/stats (which gained by_provider) instead of counting only the current page of results — clicking a card toggles that filter. Backend: GET /api/v1/findings and /findings/export accept repeated query params for severity/status/provider/source; framework matches if any selected framework is in f.framework_mapping (AQL INTERSECTION).

  • Inventory: category and multi-select filters (Epic 00): /inventory page now supports selecting multiple providers, categories, regions, and accounts at once (checkbox dropdowns with "All"/individual selection), replacing the old single-select dropdowns. Resource types are grouped into 7 categories (Compute, Containers, Database, Storage, Networking, Security & Identity, Other) via a client-side taxonomy (frontend/src/lib/inventoryCategories.ts); unmapped types fall back to "Other". New InventorySummary cards show asset counts by provider and by category — clicking a card toggles that filter. Backend: GET /api/v1/inventory accepts repeated query params (?provider=aws&provider=azure) filtered via AQL IN instead of equality; InventoryStats gained by_region and by_account_id breakdowns to populate the region/account filter options. ProviderTabs component removed (superseded by the summary cards).

  • Signal Score roadmap (Ogum.Pulse, Epic 07 Sprint 7 — planned): documented the high-confidence detection design — an event is only prioritized as high-confidence when it both deviates from the per-principal IAM behavioral baseline (actions, IPs, regions, user-agent) and matches a known TTP, instead of treating either condition alone as sufficient. See README.md Features/Roadmap.

  • Side-scanning EBS Direct API (Epic 03 Sprint 2): scan_ec2_instance_v2 Celery task replaces the volume/mount pipeline with trivy client vm ebs:{snapshot_id} — no volume creation, no OS mount required. Snapshots are deleted in finally via delete_snapshot_safe (now non-raising). New run_trivy_ebs() in trivy_analyzer.py returns (vulnerabilities, secrets) using Trivy's Severity field as the primary severity source (CVSS only as fallback for UNKNOWN).

  • Trivy sidecar server: trivy-server service in docker-compose.yml (port :4954, persistent trivy-cache volume, aquasec/trivy:latest). Celery worker gains TRIVY_SERVER_URL env var and depends_on: trivy-server.

  • Secret scanning migrated from TruffleHog3 to Trivy: secret_analyzer.py removed; Trivy --scanners vuln,secret covers both CVEs and secrets in EBS Direct scans. Match field (secret value) is never stored — Trivy redacts it as ****. Sprint 1 scan_ec2_instance no longer performs secret scanning.

  • SBOM generation and persistence: _generate_sbom() generates CycloneDX via trivy client vm ebs:{id} --format cyclonedx per scan. _persist_sbom() upserts to the new sboms ArangoDB collection with a HAS_SBOM edge from the EC2 resource. Schema additions: sboms vertex collection, HAS_SBOM edge collection, indexes on (tenant_id) and (tenant_id, resource_id).

  • rescan_sboms Celery Beat task: daily at 03:00 UTC — queries stored SBOMs, runs trivy client sbom against each, upserts new CVE findings with detection_method: sbom_rescan. Added to celery_app.py beat schedule.

  • scan_lambda_function Celery task: downloads Lambda deployment package via pre-signed URL to /dev/shm/ogum-{job_id}/ (RAM disk), extracts ZIP, runs trivy fs, normalises findings with detection_method: lambda_zip, and always wipes the RAM disk in finally. Image-based Lambda deferred to Sprint 3.

  • scan_with_ebs_direct() in snapshot_manager.py: low-level helper that builds and runs the trivy client command for EBS Direct API, returns parsed JSON.

  • _normalise_cve_v2(), _normalise_trivy_secret(): Sprint 2 normalisation helpers using Trivy Severity as primary and annotating detection_method in raw_output.

  • _trivy_severity() helper: maps Trivy severity strings to SeverityLevel; falls back to cvss_to_severity() only for UNKNOWN.

  • 16 unit tests for Sprint 2: TestRunTrivyEbs (7 tests — parse vulns, parse secrets, severity precedence, empty output, nonzero exit, ignorefile flag, target field); TestDeleteSnapshotSafe (2 tests — success and non-raising error); TestScanWithEbsDirect (4 tests — command structure, ignorefile, nonzero exit, empty stdout).

  • 9 integration tests for Sprint 2 in test_side_scanning_v2.py: TestScanEc2InstanceV2 (snapshot lifecycle, 2+1 findings persisted, SBOM+edge stored, cleanup on trivy error, severity from Trivy field not CVSS); TestScanLambdaFunction (ZIP download+scan+finding, cleanup on error); TestRescanSboms (SBOM found+rescanned, empty DB, severity from Trivy field).

  • K8s container scan via DaemonSet (Epic 03 Sprint 3): scan_k8s_container Celery task scans running containers via /proc/<PID>/root without restarting pods. run_trivy_rootfs() added to trivy_analyzer.py — calls trivy client rootfs {path} against the sidecar server with --skip-dirs /proc,/sys,/dev,/run to avoid virtual filesystem loops. SBOM generated per container scan via _generate_sbom_rootfs(). Findings use detection_method: k8s_proc_root (CVEs) and trivy_secret_k8s (secrets), linked to the Pod vertex via HAS_FINDING.

  • DaemonSet manifest: infra/k8s/ogum-scanner-daemonset.yaml — hostPID: true, privileged: true, hostPath: /proc mounted at /host/proc, minimal ClusterRole (get/list on pods/nodes only — no access to Secrets or ConfigMaps), hostNetwork: false.

  • K8s scan webhook: POST /api/v1/side-scans/webhooks/k8s-scan — DaemonSet posts scan trigger; backend validates x-ogum-token against tenant_config.scanner_token, creates scan_jobs record, enqueues scan_k8s_container task, returns 202.

  • 5 unit tests for run_trivy_rootfs: parse vulns, parse secrets (no Match field), nonzero exit raises RuntimeError, empty output returns empty lists, --skip-dirs included in command.

  • 5 integration tests (test_side_scanning_k8s.py): findings persisted in ArangoDB, SBOM+edge stored, FileNotFoundError on missing proc path, tenant isolation (security), severity from Trivy field.

  • 3 integration tests (test_k8s_webhook.py): valid token → 202 + task enqueued + scan_jobs created, invalid token → 401, missing header → 422.

  • infra/k8s/ directory created for Kubernetes manifests.

  • Registry image scan (Epic 03 Sprint 4): scan_container_image Celery task scans ECR and any OCI-registry image via trivy client image {uri}@{digest} — no docker pull, runs vuln + secret + misconfig scanners. run_trivy_image() added to trivy_analyzer.py returning (vulns, secrets, misconfigs). _normalise_misconfig() normalises Trivy misconfigurations into Finding with resource_type=container_image. SARIF report generated per scan via _generate_sarif() and upserted to the new sarif_reports ArangoDB collection (_persist_sarif()). Schema additions: sarif_reports vertex collection with persistent index on (tenant_id, image_digest).

  • ECR push webhook: POST /api/v1/side-scans/webhooks/ecr — receives ECR push events, validates x-ogum-token against tenant_config.scanner_token, creates scan_jobs record (type: ecr), enqueues scan_container_image, returns 202.

  • Jobs API: GET /api/v1/side-scans/jobs (list with status/job_type filters, pagination), GET /api/v1/side-scans/jobs/{job_id} (detail with tenant isolation — wrong tenant → 404), POST /api/v1/side-scans/jobs/{job_id}/retry (re-enqueues failed jobs only — non-failed → 422). All endpoints enforce X-Tenant-ID isolation via AQL FILTER j.tenant_id == @tid.

  • Image security badge: GET /api/v1/side-scans/images/{digest}/security — CI/CD badge endpoint returning overall_status: pass|fail and CVE counts by severity (CRITICAL/HIGH/MEDIUM/LOW). Returns fail when CRITICAL or HIGH findings exist.

  • /side-scanning UI page: Next.js page with KPI cards (EC2 / Lambda / K8s / Registry job counts), status/type filter dropdowns, jobs table (type, resource, status badge with pulse animation for running, findings count, duration, started-at), and one-click Re-scan button for failed jobs (triggers retry mutation). Sidebar nav updated: "Side Scanning" now routes to /side-scanning (previously disabled badge="Soon").

  • sideScanApi added to frontend/src/lib/api.ts: listJobs, getJob, retryJob, imageSecurityStatus.

  • Types: SideScanJob, SideScanJobStatus, SideScanJobType, PagedSideScanJobs, ImageSecurityStatus added to frontend/src/lib/types.ts.

  • 5 unit tests (TestRunTrivyImage): parse vulnerabilities, parse secrets (no Match field), parse misconfigurations, nonzero exit raises RuntimeError, empty output returns empty tuples.

  • Integration tests (test_side_scanning_registry.py): ECR webhook enqueues task + creates scan_jobs record (5 tests covering: valid push, invalid token 401, list jobs tenant isolation, status filter, job detail, wrong-tenant 404, not-found 404, retry creates new job + dispatches task, retry non-failed 422, badge pass/fail).

  • Frontend test (SideScanningPage.test.tsx): 5 unit tests — renders title, KPI cards, status filter, type filter, loading state.

Changed

  • delete_snapshot_safe() now logs but does not raise on failure (previously re-raised). Sprint 1 task cleanup blocks that wrapped it in try/except are unchanged and still safe.

  • scan_ec2_instance (Sprint 1) no longer runs TruffleHog3 for secret scanning — secret_count is always 0. Secrets are now detected in scan_ec2_instance_v2 via Trivy.

  • Sprint 1 integration tests updated to remove run_trufflehog mocks; test_scan_persists_secret_finding replaced with test_scan_v1_has_no_secret_scanning.

  • IAM Deep Model + Graph UX (Epic 02 Sprint 6): New app/services/graph/ sub-package with three modules. iam_edges.py materialises STS_ASSUMEROLE_ALLOW (trust-policy-derived role assumption edges), ASSUMES (active assumption from stored assumed_role_arn), and ATTACHED_POLICY (identity-to-policy links) from identity metadata already in ArangoDB — no new cloud API calls. privilege_escalation.py implements 10 IAM privilege escalation patterns (PRIVESC-01 through PRIVESC-10) covering direct/chain role assumption, dangerous permission combinations (CreatePolicyVersion, SetDefaultPolicyVersion, AttachRolePolicy, AttachUserPolicy, PutUserPolicy, CreateAccessKey, UpdateFunctionCode), and PassRole+RunInstances to admin instance profile. exposure.py derives exposed_internet: bool on resources/data_assets/network_endpoints from existing is_public/is_internet_facing fields.

  • Graph API (GET/PATCH/POST/DELETE /api/v1/graph/...): Crown Jewels endpoints (PATCH /resources/{id}/crown-jewel, GET /crown-jewels) to mark/list crown-jewel assets. AQL Console (POST /aql) executes tenant-scoped read-only queries — blocks INSERT/UPDATE/REMOVE/REPLACE/UPSERT, injects @tenant_id bind var when referenced, caps results at 200 rows. Saved Queries (GET/POST/DELETE /queries) persists favourite AQL queries per tenant (max 50). Shortest Path (GET /paths/{from}/{to}) uses AQL SHORTEST_PATH across all IAM and graph edges. Exposure Summary (GET /exposure) returns internet-exposed resource counts per collection.

  • Celery tasks for graph: build_iam_graph_edges(tenant_id, provider_id) calls the IAM edge builder and exposure computation; refresh_active_sessions_from_cloudtrail(tenant_id) is a documented stub deferred to Epic 05 (Pulse).

  • Schema additions: saved_queries vertex collection; STS_ASSUMEROLE_ALLOW, ASSUMES, ATTACHED_POLICY, HAS_ACTIVE_SESSION edge collections; is_crown_jewel persistent indexes on resources and data_assets; saved_queries tenant index.

  • Frontend Graph UX: QueryConsole component — AQL editor with template query picker, saved queries sidebar (create/delete), read-only result table with truncation warning and execution time. PathfindingPanel component — input pair for ArangoDB node IDs with shortest-path visualisation showing entry→edge→target chain. Both modals wired into AttackPathsPage via "AQL Console" and "Pathfinding" header buttons. graphApi client in api.ts exposes all new graph endpoints.

  • Frontend types: SavedQuery, AqlResult, ShortestPathResult, ExposureSummary interfaces in types.ts.

  • conftest.py: ARANGO_URL now falls back to http://{ARANGO_HOST}:{ARANGO_PORT} instead of localhost:8529 only, enabling integration tests to run inside Docker Compose without manual env override.

  • 13 unit tests: 8 tests for _extract_principal_arns() covering single/multi principal, wildcard action, Deny exclusion, non-AssumeRole action, service principal, empty inputs, list of actions; 5 tests for escalation pattern metadata (count, uniqueness, required fields, coverage, lowercase keys).

  • 12 integration tests for Graph API: Crown Jewels CRUD (not-found, empty list, mark+list cycle); AQL console (valid read query, blocked write statements, invalid AQL 422, @tenant_id injection); Saved Queries (empty list, create+list+delete, write query rejected, nonexistent delete 404); Exposure Summary structure validation.

  • MITRE ATT&CK Intelligence Integration (Epic 02 Sprint 5): New app/services/mitre_service.py with MITRE_CHAINS mapping the four built-in Toxic Combination rules (TC-01 through TC-04) to MITRE technique sequences. build_mitre_chain_for_tc() returns independent copies of the chain (mutation-safe). get_techniques_for_path() queries ogum_admin for full technique/tactic/APT-group details, returning gracefully when MITRE data hasn't been imported yet. enrich_finding_with_mitre() creates MAPPED_TO edges between findings and techniques in the tenant DB.

  • MITRE collections in ArangoDB schema: mitre_techniques, mitre_tactics, and mitre_groups vertex collections added to ogum_admin global database; APT_USES edge collection for group→technique relationships; MAPPED_TO and IMPLEMENTS edge collections in tenant DBs.

  • GET /api/v1/attack-paths/{path_id}/mitre endpoint: returns techniques (with tactic IDs and platform lists), tactic names, APT groups, and mitre_chain breadcrumb. Returns 404 on wrong-tenant access. Falls back gracefully to empty technique lists when MITRE bundle hasn't been imported.

  • actively_exploited filter on GET /api/v1/attack-paths: boolean query parameter filters paths by active runtime exploitation status (set by Ogum.Pulse CDR events).

  • mitre_ttp filter on GET /api/v1/findings: filter findings by MITRE technique ID (checks mitre_ttps array).

  • mitre_chain, mitre_ttps, actively_exploited, last_runtime_event_at fields on attack path documents: populated by build_attack_path_docs() for Toxic Combination rules.

  • scripts/import_mitre_bundle.py: idempotent STIX 2.0 import script — downloads enterprise-attack.json from MITRE CTI GitHub, parses techniques/tactics/intrusion-sets/relationships, and upserts into ogum_admin. Supports --local-file for offline use. Graceful error if download fails.

  • make mitre-import target: convenience wrapper for the STIX import script.

  • MITRE ATT&CK section in AttackPathDetailPanel: shows technique chips (linked to attack.mitre.org), tactic breadcrumb chain for Toxic Combinations, APT group badges with country, and graceful fallback to raw T-code chips when admin MITRE data isn't imported yet.

  • Frontend types: MitreTechnique, MitreTactic, MitreGroup, MitreIntelligence interfaces added to types.ts; AttackPath extended with mitre_ttps, mitre_chain, actively_exploited, last_runtime_event_at.

  • attackPathsApi.getMitre(pathKey) method in api.ts.

  • 17 unit tests for mitre_service: cover all four chain constants, build_mitre_chain_for_tc() for known/unknown rules, mutation safety, get_techniques_for_path() with None admin DB, missing collection, chain preservation, and empty-field handling.

  • 6 integration tests for MITRE API: 404 on missing/wrong-tenant path, response structure, empty technique list graceful degradation, actively_exploited filter true/false behavior.

  • 2 integration tests in test_attack_path_service.py: toxic combination docs include MITRE chain/ttps; non-toxic paths have empty MITRE chain.

  • EC2 Side-Scanning Pipeline (Epic 03 Sprint 1): app/services/side_scanning/ sub-package implementing agentless EC2 analysis via ephemeral EBS snapshots. snapshot_manager.py handles the full AWS lifecycle: create_scan_snapshot() tags snapshots with ogum:scan=true, ogum:tenant_id, and ogum:expires_at (+2h); wait_for_snapshot() polls with 10s intervals (max 15 min); create_volume_from_snapshot() creates a gp3 volume; mount_volume_ro() mounts read-only; cleanup (delete_volume_safe, delete_snapshot_safe) always runs in finally. Three analyser wrappers: trivy_analyzer.run_trivy_fs() parses JSON output extracting CVE ID, package, CVSS score; yara_analyzer.run_yara() returns rule+path matches (gracefully skipped if rules dir absent); secret_analyzer.run_trufflehog() returns metadata-only dicts (never logs actual secret values). cvss_to_severity() maps CVSS v3 scores to Ogum severity tiers (CRITICAL ≥9.0, HIGH ≥7.0, MEDIUM ≥4.0, LOW >0.0, INFO =0.0).

  • scan_ec2_instance Celery task (Epic 03 Sprint 1): app/workers/tasks/side_scanning.py. Runs Trivy, YARA, and TruffleHog in parallel via ThreadPoolExecutor. Normalises CVE findings (from Trivy), exposed secrets (CRITICAL severity, path metadata only), and malware detections (CRITICAL). Persists findings to ArangoDB with HAS_FINDING edges. Creates and completes a scan_jobs record per run. Cleanup always runs in finally regardless of analyser failures. OGUM_SCANNER_INSTANCE_ID env var controls whether volume attachment is attempted (skipped in dev/test).

  • cleanup_orphan_snapshots Celery Beat task (Epic 03 Sprint 1): lists all snapshots tagged ogum:scan=true and deletes any with ogum:expires_at in the past. Provides recovery from worker crashes that leave orphaned snapshots in customer accounts.

  • FindingSource.SIDE_SCANNING enum value: added side_scanning to app/models/finding.py FindingSource enum for findings originating from agentless disk analysis.

  • 15 unit tests for side-scanning: 10 tests for cvss_to_severity() covering all boundary values (CRITICAL/HIGH/MEDIUM/LOW/INFORMATIONAL thresholds, negative scores); 5 tests for _extract_cvss() covering NVD V3 extraction, vendor scores, missing CVSS block, missing V3 field, and malformed data.

  • 8 integration tests for scan_ec2_instance and cleanup_orphan_snapshots: verify snapshot created and deleted (moto + real ArangoDB), CVE/secret/malware findings persisted with correct severity, cleanup runs after analyser exception, scan_jobs record created and completed, and orphan cleanup correctly deletes expired / skips future snapshots.

  • CIEM Static Analysis (Epic 02 Sprint 4): app/services/ciem_service.py — static permission analysis engine. analyze_dangerous_permissions() inspects an identity's granted_actions and policies against a curated list of 18 dangerous IAM actions (iam:PassRole, iam:CreatePolicyVersion, iam:AttachRolePolicy, sts:AssumeRole, s3:*, iam:*, ec2:*, lambda:UpdateFunctionCode, etc.) and wildcard expansion. find_assume_role_chains() traverses ASSUMES_ROLE edges up to 4 hops to detect privilege escalation paths to high-privilege identities. list_identities_with_ciem() returns paginated identities with dangerous_permissions_count and escalation_paths_count fields, sorted by risk score.

  • Identities API (Epic 02 Sprint 4): GET /api/v1/identities — paginated list of IAM identities with CIEM counters, filterable by provider and only_dangerous=true. GET /api/v1/identities/{key}/permissions — full CIEM analysis for a single identity: dangerous permissions list, AssumeRole escalation chains, policy names, and risk score.

  • Risk Score column in Inventory table (Epic 02 Sprint 4): RiskBadge component displays numeric score (0–100) with color-coded tier (CRITICAL red, HIGH orange, MEDIUM yellow, LOW blue, NONE slate). Inventory DataTable and Findings FindingsTable now render RiskBadge for each resource/finding.

  • Attack Paths card on dashboard (Epic 02 Sprint 4): security overview page now shows a clickable Attack Paths card with total active paths count, critical+high count, and new-in-24h badge. Also added Attack Paths to the quick navigation links.

  • 13 unit tests for ciem_service: cover dangerous permission detection, wildcard expansion (iam:CreateUser → iam:*, s3:DeleteBucket → s3:*), AdministratorAccess policy detection, duplicate prevention, missing field handling, and constant structure validation.

  • 14 integration tests for GET /api/v1/identities and GET /api/v1/identities/{key}/permissions: cover empty list, sorting by risk_score, provider filter, only_dangerous filter, pagination, 404 for missing identity, dangerous permission detection, and response field completeness.

  • 10 component tests for RiskBadge: cover all 5 tiers (CRITICAL/HIGH/MEDIUM/LOW/NONE), null/undefined score rendering, title attribute, and tier CSS class verification.

  • Attack Paths canvas (Epic 02 Sprint 3): interactive React Flow canvas replacing the flat table layout. AttackPathList panel (260px) groups paths by severity with score, entry→target route, hop count, and toxic badge — clicking loads the path in the canvas. AttackPathCanvas renders custom node types (EntryPointNode red, TargetNode yellow, IdentityNode purple, ResourceNode default) with dagre left-to-right auto-layout, animated smoothstep edges with inferred relation labels, zoom/pan controls, and minimap. Bottom info bar shows selected path summary with entry point, target, risk score, hops, and a disabled "Remediate with AI" CTA. NodeDetailPanel slide-over fetches full resource detail from /api/v1/inventory/{key} on node click. graph-layout.ts utility wraps dagre layout and node-type inference. 15 component tests for AttackPathList and AttackPathCanvas. structuredClone polyfill added to jest setup (dagre dependency). @xyflow/react v12 and @dagrejs/dagre installed.

  • Attack Paths API (Epic 02 Sprint 2): three new endpoints under /api/v1/attack-paths. GET /api/v1/attack-paths returns a paginated, risk-score-sorted list of attack paths with filters for severity, is_toxic_combination, and provider (keyset cursor pagination on risk_score DESC). GET /api/v1/attack-paths/stats returns aggregate counts by severity plus a new_24h counter. GET /api/v1/attack-paths/{path_id} returns the full path document, all vertex node documents, and associated findings. Tenant isolation enforced at database level (separate ArangoDB database per tenant) plus tenant_id filter on every query.

  • Attack Paths frontend (Epic 02 Sprint 2): web console page at /attack-paths. Stats bar with per-severity counts and new-in-24h indicator. Filter bar for severity, provider, and toxic-combination toggle. Paginated table sorted by risk score with entry-point → target path display, risk score progress bar, hops count, and toxic combination badge. Slide-over detail panel with entry/target cards, full vertex node list, and associated findings. Sidebar nav item enabled. New types (AttackPath, AttackPathStats, AttackPathDetail, AttackPathFilters) and attackPathsApi client added. 20 component tests.

  • Post-scan inventory extraction (all providers): CSPM scans now automatically populate the inventory after each scan run — no separate discovery task required. prowler_inventory.extract_inventory_from_findings() deduplicates resources by resource_uid, routes to the correct collection (resources, identities, data_assets), and upserts with last_scanned_at and normalized resource_type. Supports all four providers: AWS, Azure, GCP, and Kubernetes.

  • ProwlerService.run_azure_scan, run_gcp_scan, run_kubernetes_scan: new scan methods wrapping AzureProvider, GcpProvider, and KubernetesProvider. run_cspm_scan task now routes to the correct method by provider (was AWS-only).

  • Provider-aware default frameworks in GET /api/v1/scans: when no frameworks are specified, the scan API selects sane defaults per provider (CIS-AWS-2.0 + PCI + SOC2 for AWS, CIS-AZURE-2.0 for Azure, CIS-GCP-2.0 for GCP, CIS-K8S-1.12 for Kubernetes).

  • ScanResult dataclass: ProwlerService scan methods now return ScanResult(findings, raw_outputs) so the task can use raw OutputFinding objects for inventory extraction without a second pass.

  • 43 unit tests for prowler_inventory: cover type normalization (AWS/Azure/GCP/K8s), collection routing (IAM → identities, S3 → data_assets, EC2 → resources), tag extraction, public IP detection, ArangoDB key generation, deduplication, and edge cases.

  • Risk Score engine (Epic 02 Sprint 1): app/services/risk_score.py — contextual risk scoring (0–100) for every resource in the graph. Score formula: severity_base × exposure_factor × blast_factor, capped at 100. Severity base is the weighted sum of FAIL findings (CRITICAL=10, HIGH=7, MEDIUM=4, LOW=1). Exposure factor doubles score for public/internet-facing resources. Blast factor scales with the count of sensitive data assets reachable within 3 hops. Resources in confirmed attack paths have a minimum score of 40. Path risk score uses max-node score amplified by a depth factor (shorter path = more dangerous).

  • Attack Path detection (Epic 02 Sprint 1): app/services/attack_path_service.py — AQL graph traversal queries that discover paths from internet-facing resources to sensitive data assets (up to 4 hops), IAM privilege escalation chains, and toxic combinations. Four toxic combination rules: TC-02 (public bucket with credentials), TC-03 (overpermissioned identity → production database), TC-04 (K8s pod with host network). Results are persisted to the new attack_paths collection with is_toxic_combination flag, risk score, severity, and path metadata. Resources participating in any attack path are marked in_attack_path=true.

  • Post-scan graph enrichment tasks (Epic 02 Sprint 1): app/workers/tasks/attack_paths.py — two Celery tasks (recalculate_risk_scores, detect_attack_paths) enqueued automatically after each CSPM scan completes. recalculate_risk_scores bulk-updates the risk_score field on all resources, identities, and data assets. detect_attack_paths runs the full detection pipeline and persists results.

  • attack_paths ArangoDB collection: added to the tenant schema with persistent indexes on tenant_id + risk_score, tenant_id + is_toxic_combination, and tenant_id + severity for efficient filtering. Added risk_score index on resources, identities, and data_assets for sort performance.

  • Dashboard home (F0.3): replaced / redirect with a real security overview page showing ThreatScore, finding counts by severity (CRITICAL/HIGH/MEDIUM/LOW — each links to /findings?severity=X), last 5 scan jobs with status icons and relative timestamps, and quick-navigation links to all main sections.

  • GET /api/v1/findings/stats: new endpoint returning aggregate counts by_severity and by_status plus total — a single AQL query, scoped by tenant, used by the dashboard.

  • findingsApi.stats(): new client method in frontend/src/lib/api.ts; FindingsStats type added to types.ts.

  • Dev seed endpoint (F0.1): POST /api/v1/dev/seed inserts 19 realistic demo findings (CRITICAL/HIGH/MEDIUM/LOW, mix of FAIL/PASS/MUTED) into ArangoDB without requiring real cloud credentials. Only available when DEV_MODE=true. Endpoint is idempotent. Complementary DELETE /api/v1/dev/seed clears findings and scan_jobs. Standalone script backend/scripts/seed_demo.py and make seed / make seed-clear Makefile targets for local use. DEV_MODE setting added to config.py and .env.example.

  • Discovery job tracking (US-10.09): all four discovery tasks (discover_aws, discover_azure,
    discover_gcp, discover_k8s) now write to the scan_jobs ArangoDB collection so runs appear in
    /admin/jobs. A new _job_tracking helper module (start_discovery_job, complete_discovery_job,
    fail_discovery_job) handles the lifecycle with fire-and-forget semantics — tracking failures never
    abort discovery. Integration tests cover all state transitions and the missing-collection edge case.

  • Admin Jobs API (Epic 10 Sprint 1): cross-tenant job inspection and control endpoints under
    /api/v1/admin/. GET /admin/jobs lists scan jobs across all tenant databases (or scoped to one
    via ?tenant_id=), with optional status filter and keyset cursor pagination. GET /admin/jobs/{id} returns full job detail including findings counts and logs. POST /admin/jobs/{id}/retry re-enqueues a failed job and records the action in admin_audit_log.
    DELETE /admin/jobs/{id} calls Celery revoke. GET /admin/workers inspects live worker state
    via Celery inspect. GET /admin/queue-depth returns pending message counts for all known queues
    via Redis LLEN. admin_audit_log collection added to a dedicated ogum_admin database.
    All routes marked TODO(epic-06) for role enforcement once Auth/RBAC is implemented.

  • App shell navigation (Epic 11 Sprint 1): global layout with fixed sidebar and header wrapping
    all pages. AppShell composes Sidebar + Header and is wired into app/layout.tsx.
    Sidebar organises all platform modules into labelled sections with active links for implemented
    pages (/, /inventory, /findings, /compliance, /providers) and disabled <span> elements
    with a "Soon" badge for modules not yet built (Attack Paths, Side Scanning, Pulse, CDR, AI
    Remediation, Integrations, Agent, Admin). Header shows a dynamic breadcrumb based on the
    current route. NavItem highlights the active route with an orange left border.

  • Makefile (Makefile): covers Docker Compose lifecycle (up, down, up-infra, up-backend,
    restart), code quality (lint, format, typecheck, check, fe-lint, fe-typecheck),
    tests by layer (test-unit, test-integration, test-security, fe-test, agent-test),
    dependency management (install, update-deps), container shells, log tailing, and a
    health target that checks all services. Run make help to list all targets.

  • Terraform test fixtures (infra/terraform/test-fixtures/): deploys intentionally
    misconfigured AWS resources (EC2, S3, IAM, Security Groups, VPC, CloudTrail) for validating
    Ogum.Inventory discovery and Ogum.Static CSPM findings. Includes compliant baselines alongside
    misconfigured scenarios for diff testing. terraform output expected_findings lists the exact
    CSPM findings that should appear after scanning the fixture account.

  • IaC scanning (Ogum.Static Sprint 4): POST /api/v1/scans/iac triggers a Celery task that
    shallow-clones a Git repository, runs Checkov across Terraform, CloudFormation, and Kubernetes
    manifests, and persists findings with source: iac. Repository tokens are injected into the
    HTTPS URL at clone time and never stored or logged. Cloned repos are cleaned up in a finally
    block after each scan.

  • CheckovService (app/services/checkov_service.py): wraps Checkov's programmatic API
    (TFRunner, CFNRunner, K8sRunner), normalizes check results to Finding objects with
    OCSF-aligned severity levels, and maps resource IDs to cloud provider (aws, azure, gcp,
    iac). Raises a graceful [] result on ImportError (checkov optional dependency).

  • GET /api/v1/findings/export: streams all findings matching the active filter set as CSV
    or JSON (OCSF-aligned). Streaming uses a generator with keyset cursor pagination internally —
    safe for exports of 100k+ findings without loading all rows into memory. The /export route is
    registered before /{finding_key} to prevent FastAPI treating the literal "export" as a path
    parameter.

  • Export button (components/findings/ExportButton.tsx): dropdown with CSV and JSON (OCSF)
    options; triggers a browser Blob download with the current filter set applied. Appears in the
    Findings page header next to the severity counter chips.

  • docs/scanning.md: public documentation covering CSPM scan trigger and polling, supported
    compliance frameworks, IaC scan parameters (including token security notes), export formats and
    filter options, and CSV field reference.

  • Findings UI (Ogum.Static Sprint 3): /findings page with a full-featured FindingsTable
    component — severity badge column, MUTED/ACCEPTED/PASS/FAIL status, provider badge,
    keyset cursor pagination (Previous/Next). FindingFilters bar with debounced search, six
    dropdown filters (severity, status, provider, framework, source), and active-filter chips
    with individual clear and "Clear all". FindingDetailPanel slide-in (440px) showing resource
    metadata, risk description, remediation steps, CLI command with copy-to-clipboard, framework
    badges, mute modal with required reason field, and a placeholder "Generate PR ✨" (Ogum.AI,
    Epic 05). Panel closes on Escape key or backdrop click.

  • Compliance dashboard (/compliance): framework score sidebar with pass/fail counts and
    progress bar, main panel with detailed scores, ThreatScore gauge (0–100 weighted by severity
    of open FAIL findings), top-10 failing controls ordered by occurrence count.

  • GET /api/v1/compliance/summary: AQL-powered aggregation endpoint returning per-framework
    pass/fail/total/score, global ThreatScore, and top-10 failing checks by count — all scoped to
    the requesting tenant.

  • SeverityBadge component (components/ui/SeverityBadge.tsx): dedicated wrapper around
    Badge that maps SeverityLevel enum to the correct variant — reused across findings table,
    detail panel, and compliance page.

  • Frontend types and API client extended with Finding, FindingDetail, PagedFindings,
    FindingsFilter, ComplianceSummary, FrameworkScore types; findingsApi (list, get, mute,
    accept) and complianceApi (summary) added to lib/api.ts.

  • Findings API (Ogum.Static Sprint 2): GET /api/v1/findings with server-side filtering
    (provider, severity, status, framework, region, account_id, resource_type, source, full-text q)
    and keyset cursor pagination (stable across large datasets). GET /api/v1/findings/{key} returns
    the full finding enriched with linked resource metadata, attack path placeholders (Epic 02), and
    the best available CLI remediation command. PATCH /api/v1/findings/{key} mutates status to
    MUTED (requires reason) or ACCEPTED; every change is written to the audit_log collection.

  • audit_log vertex collection added to tenant schema with indexes on tenant_id and
    (tenant_id, finding_key) — stores every finding status mutation with actor, reason, and timestamp.

  • app/services/cli_command.py: provider-aware CLI command builder. Priority: stored
    remediation_code from Prowler → known resource-type template → provider fallback.
    Supports AWS, Azure, GCP, and Kubernetes. ARN-style resource IDs are simplified to the last
    segment for ergonomic CLI output.

  • CSPM scan engine (Ogum.Static Sprint 1): Prowler v5 programmatic integration via
    ProwlerService. POST /api/v1/scans triggers a Celery task (run_cspm_scan) that
    executes Prowler checks, normalizes results to OCSF-aligned Finding objects, and persists
    them in ArangoDB with idempotent upserts (check_id + resource_id + tenant_id as key).
    GET /api/v1/scans/{job_id} and GET /api/v1/scans expose job status and history.

  • findings and scan_jobs vertex collections added to tenant schema (init_tenant_schema).
    HAS_FINDING edge collection links resources → findings.
    Persistent indexes on tenant_id, severity, status, provider, and check_id.

  • Finding and ScanJob Pydantic models (app/models/finding.py): OCSF-aligned severity
    levels (CRITICAL/HIGH/MEDIUM/LOW/INFORMATIONAL), finding status (FAIL/PASS/MUTED/ACCEPTED),
    framework mapping list, remediation text and code, scan job lifecycle tracking.

  • Provider delete purges all associated resources: DELETE /api/v1/providers/{id} now
    hard-deletes all documents in resources, identities, data_assets, and
    network_endpoints scoped to the provider's account, then sweeps orphaned edges from all
    eight edge collections. The response includes a purged map with per-collection delete counts.

  • Scan all regions by default: regions: [] (empty list) now means "scan all enabled regions"
    across all providers. For AWS, the task calls ec2:DescribeRegions to resolve the full list
    at runtime. Azure, GCP, and Kubernetes already scanned all resources regardless of region.
    The wizard and edit modal show an empty field by default with a hint explaining the behavior;
    users can still restrict to specific regions by listing them comma-separated.

  • Edit provider: PATCH /api/v1/providers/{id} now accepts role_arn, azure_tenant_id,
    and azure_client_id updates (non-secret, safe to persist); EditProviderModal component
    pre-fills current config values and supports optionally re-triggering discovery with new
    ephemeral credentials (API keys, client secret, SA JSON, kubeconfig) that are forwarded to
    the worker and never stored.

  • DiscoverRequest type exported from @/lib/types; providersApi.triggerDiscovery now
    accepts an optional DiscoverRequest body for re-trigger with static credentials.

  • Edit button (Pencil icon) added to ProvidersTable and ProviderCard — opens
    EditProviderModal with current provider values pre-filled.

  • Multi-method credential support for all providers:

    • AWS: IAM Role (role_arn via sts:AssumeRole) and static API keys
      (aws_access_key_id / aws_secret_access_key, dev only) are both supported in the UI
      and API; ambient worker credentials remain the fallback when neither is provided
    • Azure: Service Principal (azure_tenant_id + azure_client_id + azure_client_secret)
      and DefaultAzureCredential (Managed Identity / ADC) are both selectable in the wizard
    • GCP: Service Account JSON and Application Default Credentials (ADC) are both supported
    • Kubernetes: external kubeconfig (JSON) and in-cluster ServiceAccount are both selectable
  • credential_type field on ProviderConfig: records which credential method was used
    at registration time (role, static, ambient, service_principal, managed_identity,
    service_account, adc, kubeconfig, incluster) — no secrets stored

  • azure_tenant_id / azure_client_id stored on ProviderConfig (non-secret, safe to
    persist) so re-triggered discovery can restore Azure credential context

  • provider_key threading in discover_azure and discover_gcp: both tasks now accept
    and propagate the provider_key parameter so ArangoDB status updates work correctly

  • Status updates for Azure and GCP: _set_provider_status(db, provider_key, "active"|"error")
    now called on task success and failure, consistent with AWS behavior

  • MethodSelector UI component in ConnectWizard: segmented control for choosing
    credential method per provider — no separate page or modal required

  • Provider Management API (/api/v1/providers): new endpoints GET /{id}, PATCH /{id},
    and POST /{id}/discover for fetching, updating, and re-triggering discovery on a specific provider

  • Provider status field: ProviderConfig now tracks pending | active | error | disabled state;
    status transitions automatically on discovery dispatch and enable/disable toggle

  • ProviderUpdateRequest model: supports partial PATCH of display_name, regions, and enabled

  • DiscoverResponse model: typed response for the re-trigger discovery endpoint

  • ProviderType enum: provider field is now validated as a literal type (aws | azure | gcp | k8s)

  • /providers page (frontend): connected accounts management page with table showing provider
    type, display name, account identifier, regions, status badge, last discovery time, and per-row
    actions (re-discover, enable/disable, delete)

  • ProvidersTable component: reusable table and card components for provider management,
    with busy state per row and correct disabled states for actions

  • providersApi client additions: get(), update(), and triggerDiscovery() methods

  • docs/connecting-accounts.md: comprehensive guide covering UI wizard, API reference,
    credential model, discovery schedule, status reference, and troubleshooting

  • Ogum.Inventory Sprint 5 — Onboarding, Export, and Tenant Isolation

    • tenant_config document collection added to init_tenant_schema() — stores connected
      provider metadata per tenant (no credentials — those come from env/Vault at task time)
    • app/models/provider.py — Pydantic v2 schemas: ProviderConfig, ProviderRegisterRequest,
      ProviderRegisterResponse
    • app/services/provider_service.py — CRUD on tenant_config collection:
      register_provider, list_providers, delete_provider, update_provider_last_discovery
    • POST /api/v1/providers — register a cloud provider connection; dispatches discovery
      task (aws/azure/gcp/k8s) and records job_id; 201 with provider_id and discovery_job_id
    • GET /api/v1/providers — list all connected providers for the tenant
    • DELETE /api/v1/providers/{provider_id} — remove a provider config; 404 if not found
    • GET /api/v1/inventory/export?format=csv — streaming CSV export of full tenant inventory
      (capped at 50k rows); Content-Disposition attachment header with timestamped filename
    • GET /api/v1/inventory/export?format=json — streaming OCSF-inspired JSON export with
      ocsf_version, metadata (tenant_id, exported_at, total_resources, product), and
      resources array; 422 for unsupported format values
    • Export endpoint registered before /{resource_key} to prevent route shadowing
    • Frontend: ConnectWizard component (components/providers/ConnectWizard.tsx) — 4-step
      modal (select provider → configure → connecting → done); supports AWS, Azure, GCP, K8s;
      provider-specific fields shown conditionally; error display on failed registration
    • Frontend: /providers/new page — renders ConnectWizard; redirects to /inventory on success
    • Frontend: Inventory page — pristine empty state (zero resources + no active filters)
      now shows "No cloud accounts connected" callout with a "Connect Account" CTA link
    • Frontend: providersApi and inventoryApi.exportCsv/exportJson added to src/lib/api.ts
    • Frontend: ProviderConfig, ProviderRegisterRequest, ProviderRegisterResponse
      interfaces added to src/lib/types.ts
    • 15 backend integration tests — all passing:
      • test_providers_api.py (15): register (AWS, Azure, GCP), missing header → 422,
        idempotency, list empty/populated, delete existing/nonexistent, CSV export content-type
        and header row, JSON export OCSF structure, JSON metadata fields, invalid format → 422,
        CSV with seeded resources contains data rows
    • 2 new tenant isolation tests (TestProviderConfigIsolation): provider config in
      Tenant A not visible in Tenant B; AQL on Tenant B returns zero Tenant A resources
    • 8 frontend component tests for ConnectWizard — provider selection, configure step per
      provider, back navigation, cancel handler; all passing
    • docs/getting-started.md — new "Using the Web Console" and "Using the API" sections
      with curl examples for connect, list, export CSV, and export JSON
  • Ogum.Inventory Sprint 4 — Multi-Provider Discovery + Celery Beat Scheduling

    • AzureResource(ResourceBase) model — adds subscription_id field; arango_key()
      uses {sub_prefix}_{resource_group}_{name} for unique, length-safe ArangoDB keys
    • GCPResource(ResourceBase) model — adds project_id field; inherits arango_key()
      from ResourceBase using {provider}_{type}_{compact_id}
    • K8sResource(ResourceBase) model — adds cluster_name and namespace fields;
      overrides arango_key() as k8s_{cluster}_{type}[_{namespace}]_{uid}
    • app/workers/tasks/scheduling.py — acquire_lock(redis, tenant_id, provider) and
      release_lock(redis, tenant_id, provider) helpers using Redis SET NX EX pattern
      (TTL = 7h); trigger_all_discoveries Celery task routes Beat events to provider tasks
    • app/workers/tasks/azure_discovery.py — discover_azure task: VMs, VNets, NSGs,
      Storage Accounts, AKS clusters, Key Vaults (metadata only); soft-delete of absent
      resources; distributed lock guards against concurrent runs
    • app/workers/tasks/gcp_discovery.py — discover_gcp task: Compute instances
      (aggregated across all zones), GCS buckets, GKE clusters; distributed lock
    • app/workers/tasks/k8s_discovery.py — discover_k8s task: Pods, Deployments,
      Services, Nodes, Namespaces; supports kubeconfig dict or in-cluster config;
      distributed lock; K8s UIDs used as resource_id for stable keys
    • celery_app.py — updated include list to register all new task modules;
      default beat_schedule added (trigger_all_discoveries every 6h for dev)
    • docker-compose.yml — celery-beat service added (single instance, Redis-backed)
    • pyproject.toml — added azure-mgmt-compute, azure-mgmt-network,
      azure-mgmt-storage, azure-mgmt-containerservice, azure-mgmt-keyvault,
      google-cloud-compute, google-cloud-storage, google-cloud-container,
      kubernetes as explicit runtime dependencies
    • 19 integration tests — all passing:
      • test_scheduling.py (7): distributed lock acquisition, release, tenant/provider
        isolation, trigger routing, unknown-provider error handling
      • test_azure_discovery.py (4): VMs persisted, idempotency, soft-delete, lock skip
      • test_gcp_discovery.py (4): compute instances persisted, idempotency, soft-delete, lock skip
      • test_k8s_discovery.py (4): pods persisted, idempotency, soft-delete, lock skip
  • Ogum.Inventory Sprint 3 — REST API + Inventory UI

    • GET /api/v1/inventory — list resources with filters (provider, resource_type,
      account_id, region, status, search), limit/offset pagination, and
      sort_by/sort_dir ordering; returns standard {data, meta, error} envelope
    • GET /api/v1/inventory/stats — aggregate counts by provider, resource type, status;
      also returns identity_count, data_asset_count, and last_discovery_at
    • GET /api/v1/inventory/{resource_key} — full resource detail including all
      relationship edges (inbound + outbound across all 8 edge collections)
    • POST /api/v1/inventory/discover — enqueues discover_aws Celery task and returns
      job_id (Celery task ID); returns 422 for providers not yet implemented
    • app/services/inventory_service.py — ArangoDB query service; AQL built with
      bind_vars only (no string interpolation with user input); sort field validated
      against allowlist before use
    • app/models/api_responses.py — Pydantic v2 response models: ApiResponse[T],
      Meta, ResourceSummary, ResourceDetail, EdgeSummary, InventoryStats,
      DiscoverJobResponse
    • X-Tenant-ID header (dev mode) — replaces JWT tenant extraction until Sprint 7;
      clearly marked as temporary in code comments
    • Frontend: Next.js 15 App Router scaffold — layout.tsx, page.tsx (→ /inventory),
      globals.css, tsconfig.json, next.config.ts, tailwind.config.ts
    • Frontend: InventoryPage (src/app/inventory/page.tsx) — full inventory UI with
      React Query data fetching, provider tabs, filters, paginated DataTable, and DetailPanel
    • Frontend: DataTable component — sortable table with Provider badge, name/ID,
      type, account, region, status badge, pagination controls, and skeleton loading state
    • Frontend: ProviderTabs component — All/AWS/Azure/GCP/K8s tabs with resource counts
    • Frontend: Filters component — search input (name/ARN/ID), resource type dropdown,
      region dropdown
    • Frontend: DetailPanel component — 420px slide-in panel with Metadata section,
      Tags, Relationships (edges with direction arrows), Findings placeholder, and
      "Open in console" deep-link for AWS resources
    • Frontend: Badge component (components/ui/Badge.tsx) — provider, status, and
      severity variants with dark-theme color coding (orange accent, slate palette)
    • Frontend: Skeleton component (components/ui/Skeleton.tsx) — loading placeholder
    • Frontend: src/lib/types.ts — TypeScript interfaces for all API response shapes
    • Frontend: src/lib/api.ts — Axios-based API client with X-Tenant-ID interceptor
    • Jest testing setup — jest.config.js, jest.setup.ts, @testing-library/react,
      @testing-library/jest-dom, @testing-library/user-event added to devDependencies
    • 16 backend API integration tests (100% passing) — happy path, filters, pagination,
      search, 422 validation, 404 for missing resource, stats aggregation, discover dispatch
    • 32 frontend component tests — Badge (11 tests), DataTable (9 tests), DetailPanel
      (12 tests); all passing with @testing-library/react
  • Ogum.Inventory Sprint 2 — Expanded AWS Discovery (discover_aws task)

    • Full coverage: VPC, Subnet, Internet Gateway, Elastic IP, Security Group, RDS, Lambda,
      EKS, EKS, ECR, KMS (customer-managed keys only), Secrets Manager (metadata only),
      CloudTrail, and IAM Groups
    • IdentityType.IAM_GROUP enum value added to inventory model
    • Three new edge collections: BELONGS_TO, ATTACHED_TO, MEMBER_OF
    • _upsert_edge helper: idempotent AQL UPSERT keyed on (_from, _to)
    • _create_resource_edges orchestrator: builds BELONGS_TO (EC2→VPC, Subnet→VPC),
      ATTACHED_TO (SG→EC2, SG→Lambda), ROUTES_TRAFFIC (IGW→VPC),
      ASSUMES_ROLE (EC2/Lambda→IAM Role), and MEMBER_OF (IAM User→IAM Group) edges
    • discover_aws Celery task: full discovery with edge creation, replaces discover_aws_basic
      for production use; discover_aws_basic retained for backwards compatibility
    • pythonpath = ["."] added to pyproject.toml pytest config (no manual PYTHONPATH export needed)
  • Integration tests — Sprint 2 (17 new tests, 50 total passing)

    • TestAWSExpandedDiscovery: VPC discovery, open-ingress SG marked is_public=True,
      RDS instance discovery, Lambda with execution role, ECR repository, soft-delete of removed VPC
    • TestRelationshipEdgeCreation: EC2→VPC (BELONGS_TO), SG→EC2 (ATTACHED_TO),
      IGW→VPC (ROUTES_TRAFFIC), Lambda→Role (ASSUMES_ROLE), 2-hop AQL traversal IGW→VPC→EC2

Changed

  • CI — venv cache optimization: all backend CI jobs (lint, unit tests, integration tests, security audit) now use poetry.lock in the cache key (not just pyproject.toml), add cache-dependency-path: backend/pyproject.toml to the setup-python action, and skip poetry install entirely when the virtualenv cache is hit. The integration and unit test jobs add id: venv-cache + if: steps.venv-cache.outputs.cache-hit != 'true' guards.
  • CI — ArangoDB readiness polling optimized: the backend-integration job now polls ArangoDB with 3s intervals (was 5s), breaking immediately when ready instead of always completing 30 iterations. Redis uses a Docker --health-cmd "redis-cli ping" health check. Note: ArangoDB cannot use a Docker health check because curl is not available in the arangodb:3.12 image — the runner-side polling loop is the correct approach.

Fixed

  • Provider cascade delete: deleting a provider now removes all associated data in the correct order — findings and scan_jobs (by provider_id), then graph vertices (resources, identities, data_assets, network_endpoints scoped by provider+account), then orphaned edges, and finally stale attack_paths whose entry_point or target no longer exists in the graph. Previously, findings, scan_jobs, and attack_paths were left as orphaned records after provider removal.
  • Prowler metadata normalization: result.metadata in OutputFinding is CheckMetadata directly — the previous code was accessing metadata.CheckMetadata (one level too deep), causing all check_id, title, description, severity, and resource_type to resolve as "unknown" on every finding.
  • Finding _key sanitization: arango_key() now uses sha256(check_id|resource_id|tenant_id) instead of string substitution — prowler resource names/UIDs can contain dots, parens, @, and other characters that ArangoDB rejects as _key values, causing ERR 1221 on upsert.
  • Prowler v5 API alignment: rewrote ProwlerService.run_aws_scan to match the actual prowler-core v5 API — AwsProvider now takes flat keyword args (no audit_config dict), and scanning uses the high-level Scan class with compliances=[...] instead of manual check iteration. Framework IDs are mapped to prowler's file slugs (CIS-AWS-2.0 → cis_2.0_aws, etc.). _normalize converts OutputFinding pydantic objects to Ogum's Finding model via field introspection.
  • CSPM scan import path: corrected prowler.providers.aws.provider → prowler.providers.aws.aws_provider — the module was renamed in prowler v5 and the worker was failing with ModuleNotFoundError on every scan trigger.
  • Re-trigger discovery with static credentials: POST /{id}/discover now accepts an optional DiscoverRequest body so providers registered with credential_type: static, service_principal, service_account, or kubeconfig can re-provide their ephemeral credentials without re-registering the provider.
  • discover_aws diagnostic log: the task now logs has_role_arn and has_static_keys at INFO level on start, making it easy to verify whether credentials reached the Celery worker.
  • AWS discovery credential guard: NoCredentialsError was not caught when account_id was already provided at registration — the sts:GetCallerIdentity call was skipped, so missing credentials were only detected mid-discovery inside _list_vpcs. Fixed by always calling STS upfront to validate credentials regardless of whether account_id is known.
  • docker/backend.Dockerfile — base image updated from python:3.11-slim to python:3.13-slim to match the Python 3.13 requirement in pyproject.toml; docker compose up was failing with "currently activated Python version 3.11.15 is not supported by the project"
  • backend/pyproject.toml — Python constraint changed from ^3.13 (>=3.13,<4.0) to >=3.13,<3.14; prowler ^5.31.0 requires Python <3.14 and Poetry was unable to resolve dependencies when the upper bound was open-ended at <4.0
  • backend/pyproject.toml — removed all cloud provider SDK direct declarations (azure-identity, azure-mgmt-security, azure-mgmt-compute, azure-mgmt-network, azure-mgmt-storage, azure-mgmt-containerservice, azure-mgmt-keyvault, google-cloud-securitycenter, google-cloud-compute, google-cloud-storage, google-cloud-container, kubernetes); prowler ^5.31.0 pins specific versions of all these packages and any independent range declaration causes dependency resolution failure. All these SDKs remain available as prowler transitive dependencies; only boto3 is kept as a direct dependency since it is used in discovery tasks that run independently of prowler
  • backend/pyproject.toml — removed truffleHog3 ^3.0.0; all 3.x versions pin attrs==20.3.0 which is incompatible with prowler's required jsonschema==4.23.0 (which needs attrs>=22.2.0); truffleHog3 was planned for Epic 03 (Side-Scanning) and is deferred until a compatible secrets-scanner alternative is chosen
  • backend/pyproject.toml and docker/backend.Dockerfile — downgraded target runtime from Python 3.13 to Python 3.12; Python 3.13 has no pre-built wheels for several prowler transitive dependencies (notably alibabacloud-tea), which forces source compilation; during source compilation, Poetry's mid-install downgrade of packaging (26.2 → 23.2) leaves packaging/tags.py temporarily absent, breaking the build isolation subprocess; Python 3.12 has full wheel coverage for all prowler dependencies, eliminating source builds and the packaging race condition entirely
  • .github/workflows/ci.yml — updated all python-version references from 3.13 to 3.12 to match the runtime constraint in pyproject.toml and backend.Dockerfile
  • .github/workflows/ci.yml — added docker-build job that runs docker compose build --no-cache on every push and PR; validates that the Docker build succeeds before any other job runs; prevents docker compose up regressions from reaching main