Repository navigation
Added
- New
/scanspage (Configuration): registers and lists every CSPM scan job — job id, provider, frameworks, start/end/duration, checks completed, and a findings/assets summary (new/updated/removed, total assets, assets removed) — with status pill filters (queued/running/completed/failed, all selected by default), cursor pagination, and a "Trigger Scan" modal picking any connected provider. Finished jobs (completed/failed) show a "View Logs" button opening the raw execution log. Backend:ScanJobgainedfindings_new/findings_updated/findings_removed/assets_total/assets_removed/duration_seconds, computed inrun_cspm_scan(workers/tasks/cspm_scan.py) —_soft_delete_stalenow returns the deletedresource_ids per collection, a new_cascade_delete_findingshard-deletes findings whose resource was just soft-deleted this run (sweeping theirHAS_FINDINGedges), and_count_new_and_updated_findingsreads the pre-existingfirst_seen_scan_id/scan_countfields.scan_service.py(rewritten) adds keyset-paginatedlist_scan_jobs(status/provider filters) andget_scan_job_logs, both scoped to CSPM jobs only (task_namestarting withcspm_scan/) sincescan_jobsis shared with Side-Scanning's differently-shaped documents. NewGET /api/v1/scans(paginated) andGET /api/v1/scans/{job_id}/logs. - Semantic
id/data-testidattributes and a shared page-header pattern applied across the entire frontend, adopted as the standing convention (documented inCLAUDE.md): structural elements (page root, major sections, cards/panels, tables, forms) carry anidwhen unique per page or adata-testidwhen repeated via.map(); a page's title and one-line description now live in the top bar (Header.tsx'sROUTE_LABELS/ROUTE_SUBTITLES) instead of a local<h1>/<p>duplicated in every page body. - Epic 14 Sprint 4, Compliance Avançado batch (UX Avançado II — score duality, section heatmap, score trend, nested requirement drill-down): a new
compliance_catalog.pyloads the full AWS requirement catalog for every compliance framework at runtime via theprowlerpackage's ownCompliance.get_bulk("aws")(not vendored JSON), computing the same{Framework}-{Version}slug Prowler already embeds inFinding.framework_mapping— verified byte-identical againstresolve_family()'s existing slug table.GET /api/v1/compliance/frameworks/{id}(new) returnsscore_by_control(fraction of controls with zero failing assets, Unscored controls excluded from the denominator) alongside the pre-existing finding-count-based score, now exposed asscore_by_asset, plusunscored_count(catalog controls with no PASS/FAIL finding yet for the tenant) and a section → sub-section → requirement tree (each requirement carries PASS/FAIL/UNSCORED status and afinding_keylink) that backs both the newSectionHeatmap(color-coded ≥80/50–79/<50, reusingScoreGauge's existing thresholds) andRequirementAccordion(plain React state, no shadcn — see Fixed below) components. A newcompliance_score_snapshotscollection, written bysnapshot_compliance_scoresat the end of everyrun_cspm_scan(any provider, upserted by tenant+framework+day — no backfill), backsGET /api/v1/compliance/frameworks/{id}/trend?period=7d|14d|1mand the newScoreTrendChart(first realrechartsusage in the project; categorical color pair validated with the dataviz skill's palette validator against the dark chart surface). The catalog — and thereforescore_by_control/Unscored— is AWS-only (decision #14 scope); non-AWS frameworks still getscore_by_assetand a trend, surfaced viacatalog_available: falserather than a fabricated Unscored count. Follow-up in the same batch:MUTED/ACCEPTEDfindings, previously dropped silently by every compliance query (FILTER f.status IN ["FAIL", "PASS"]), are now tracked explicitly.GET /api/v1/compliance/frameworks/{id}and its section tree carry two parallel breakdowns — Control view (control_pass_count/control_fail_count/control_unscored_count/control_total, where an ACCEPTED-only control folds into Pass and a MUTED-only control folds into Unscored — a control's status is FAIL if any asset fails it, else PASS if any asset passes or has accepted risk, else Unscored) and Findings view (finding_pass_count/finding_fail_count/finding_accepted_count/finding_muted_count, the real per-finding-status counts, both ACCEPTED and MUTED excluded from thescore_by_assetratio) — surfaced via a "By Control / By Findings" toggle above the summary row and eachSectionHeatmaprow, switching client-side against data already fetched in one request rather than refetching per toggle. - Epic 14 Sprint 4, Attack Paths batch (UX Avançado II — hierarchical path list, View Results By toggle, path narrative):
GET /api/v1/attack-pathsand/statsnow enrich every row withtarget_asset_category(server-sideDOCUMENT()join + a newresource_categories.py, a direct Python port ofinventoryCategories.tskept in lockstep) andtarget_crown_jewel_reason(derived from real signals only —is_public/exposed_internet→internet_facing,contains_credentials/contains_secrets→stores_sensitive_data,has_admin_policy→high_privilege_identity, elsemanually_flagged;nullwhen the target isn't a crown jewel), both computed at read time rather than persisted since crown-jewel status can change out-of-band between path-detection runs; both are also new list filters and stats dimensions.AttackPathListis now a 2-level expandable tree (severity → category, both expanded by default with per-level counts), and a "View Results By: Paths / Alerts / Assets" toggle on/attack-pathsreorders the same underlying path list by a different second-level key (target_asset_categoryfor Paths/Assets, detectionrulefor Alerts) — no new endpoint, per the decision to keep one search engine behind all three views.AttackPathCanvasgained a syntheticriskInsightnode type (amber, dotted edge, ⊕/🔒 status badge) attached per-resource from the path's associated findings via a newattachRiskInsightNodeshelper ingraph-layout.ts.build_attack_path_docsnow persistsexposure(internet_facing/public_facing/trusted_access/none, via a newclassify_path_exposureingraph/exposure.py),is_cross_account,is_cross_cloud_provider, andaccount_idson every detected path; a newGET /api/v1/attack-paths/{path_id}/narrativereturns a deterministic 4-step paginated narrative (attack_path_narrative_service.py, same template-onlygenerated_bycontract as US-14.02'sResourceNarrativeSummary— Epic 05/Ogum.AI-RAG still isn't implemented). The path detail bottom bar gained a "Path Info" section (status, created, length, copyable ID,ExposureBadge, cross-account/cross-cloud, accounts — "Status Changed" is an explicit "not available yet", there's no status-transition history onattack_pathsyet) and aTake Actionmenu (Export path and View each resource are functional; Generate remediation PR is disabled, Epic 05 dependency). Removed the Epic 02 Sprint 2 table-based Attack Paths components (AttackPathsTable.tsx,AttackPathFilters.tsx,AttackPathDetailPanel.tsx) — dead code superseded by the React Flow canvas UI, unused by any page since the canvas redesign. - Epic 14 Sprint 3 (UX Avançado II — Software Inventory, Resource Compliance, Findings SLA/Exposure/Timeline): the Inventory detail panel's
Software InventoryandCompliancetabs now render real data.GET /api/v1/inventory/{resource_key}/softwarederives Installed Packages (name, version, linked CVE ids, filesystem path when Trivy exposes it) and Licenses (SPDX id, category via a newlicense_catalog.pypermissive/copyleft/weak-copyleft table, deprecated-id flag) from the CycloneDX SBOM already produced by Side-Scanning (Epic 03); theApplications/Running Servicessub-tabs show an explicit "not available yet" state since no pipeline in the platform produces that data yet (documented gap, not fabricated).GET /api/v1/inventory/{resource_key}/complianceadds a per-resource framework selector + control table, reusingcompliance_service.py's family/section resolution logic scoped to one resource's findings. On Findings: a configurable per-severity remediation SLA (GET/PUT /api/v1/settings/sla, defaults CRITICAL 7d/HIGH 30d/MEDIUM 90d/LOW 180d, new/settings/slapage) backs aGET /api/v1/findings/sla-summaryaggregate panel andSLABadgewired into the findings table and detail panel (the SLA classification logic — overdue past deadline, at-risk within 20% of the remaining window — is a direct Python port of the existing frontendclassifySLAhelper, kept in lockstep insla_service.py);GET /api/v1/findings/{key}/exposure-pathadds a mini blast-radius graph to the finding detail panel, reusinginventory_detail_service.get_blast_radiusrather than duplicating the AQL traversal; and a Timeline section shows first-detection date and a newscan_countre-scan counter.first_seen_scan_id/last_seen_scan_id/scan_countare newFindingfields, populated by a newupsert_finding()helper (workers/tasks/cloud_utils.py) that replaces the ad-hoc upsert previously duplicated incspm_scan.pyandside_scanning.py— a small, deliberately scoped pull-forward of Epic 12's US-12.06 (only the scan-history fields, not the grouped view/is_newbadge/bulk-mute that story also covers). - Epic 14 Sprint 2 (UX Avançado II — Inventory detail panel): the Inventory detail panel is rewritten as a two-tier tab system (
Info | Risk | Network | IAM | Configurations | Software Inventory | Compliance, each with sub-tabs), ~58% viewport width instead of a 420px drawer, with breadcrumb,Scan Now/Open in Console/kebab (Copy ARN/Resource ID) actions, keyboard-navigable tabs (roving tabindex, arrow keys), and deep-linkable state via?tab=&subtab=querystring.Info > Overviewnow renders an AI-shaped but fully deterministic narrative summary (GET /api/v1/inventory/{resource_key}/summary— no LLM is wired up yet, Epic 05 isn't built, so this is the fallback path, not a fallback from something) with clickable deep-link chips, relationships regrouped into per-type mini-tables (replacing the old flat edge list), and an embedded blast-radius mini-graph (GET /api/v1/inventory/{resource_key}/blast-radius, outbound graph traversal capped at 3 hops) that's also the full view underRisk > Blast Radius.AttackPathCanvasgained amode="mini"variant (reduced toolbar, no MiniMap,CenterNodefor the focal resource) reused by both — no forked canvas component, per the established convention. Tabs without a data source yet (Alerts, Vulnerabilities, IAM, Network, Software Inventory, Compliance) show an explicit "not available yet" state rather than fabricated content; they land in later Epic 14 sprints per plan. - Epic 14 Sprint 1 (UX Avançado II — foundation): new
components/ui/primitives —RiskScoreBadge(numeric 0.0–10.0),AlertVulnQuintet,ExposureBadge,CrownJewelBadge,SLABadge(withclassifySLAhelper) — plusPillFilter(search + operator + count popover, apply/clear semantics) andSavedViewSelector(pinned quick-access + all-views dropdown + save-as-new). Backed by a newGET/POST/PATCH/DELETE /api/v1/viewsendpoint (app/api/v1/views.py,app/services/view_service.py) with three curated system views seeded per tenant and per-user scoping via the same dev-mode header pattern asX-Tenant-ID(X-User-Id— Epic 06/RBAC replaces this later). System views are read-only from the API; only the owning user can rename, edit, pin, or delete their own views. docs/qa/awsgoat-detection-coverage.md: maps every vulnerability in the vendored AWSGoat modules (infra/terraform/awsgoat/) to the Ogum module/mechanism that should detect it, marking explicit gaps (application-layer SAST/DAST logic bugs, S3 object-content secret scanning, a missingPASSES_ROLEgraph edge for theiam:PassRole+ec2:RunInstancesescalation pattern) where nothing today does. Use it to plan a live end-to-end validation session and to prioritize backlog items.
Security
h24.3.0 (PYSEC-2026-3628, fix exists in 4.4.1) flagged by the PR-time pip-audit job: h2 arrives transitively viaprowler, which hard-pins it at exactly 4.3.0 across every 5.x release — the same pin pattern as prowler'scryptographypin documented below. An explicith2 = ">=4.4.1"floor inpyproject.tomlwas attempted first and made poetry resolution unsolvable (every candidate prowler version requires h2 4.3.0), so it was reverted; the advisory is now on the pip-audit ignore list with that rationale in the comment, tracked forchore/security-upgradesuntil prowler unpins.pip26.0.1 (PYSEC-2026-3721) flagged for the first time by the audit job: affects pip itself inside the poetry venv on the CI runner, same category as the three pre-existing pip ignores — added to the ignore list with an updated comment grouping all four.langchain-coreknown vulnerability (PYSEC-2026-2193) with no fix in the0.3.xline: upgradedlangchainto^1.0.0andlangchain-communityto^0.4.0(backend/pyproject.toml), resolving tolangchain-core1.4.9 andlangchain-text-splitters1.1.2. This also removes the two priorpip-auditignore entries (GHSA-gr75-jv2w-4656,PYSEC-2026-77) that were waiting on alangchain1.x release to exist on PyPI — it now does. No application code depends onlangchainyet (RAG/GitOps module not yet implemented), so this upgrade carries no migration risk.- Open Dependabot alerts against
langchain(GHSA/PYSECadvisories fixed in1.3.9+): tightened thelangchainfloor from^1.0.0to^1.3.9(backend/pyproject.toml) so the declared manifest range itself no longer permits a vulnerable version — the actual resolution was already patched (see above), but Dependabot evaluates the manifest range directly since this repo doesn't commitpoetry.lock. Confirmedcryptography(pinned to exactly46.0.7byprowleracross all5.3xreleases — not a range, a hard pin) andecdsa(no fix published) cannot be bumped without breaking Prowler compatibility; both remain in thepip-auditCI ignore list, already documented with rationale. Added.github/dependabot.ymlscoping version-update PRs tobackend/(pip) andfrontend/(npm) only — the vendored, deliberately-vulnerableinfra/terraform/awsgoat/fixture is excluded, and its pre-existing alert backlog was bulk-dismissed as accepted risk (test fixture, not deployed via CI).
Changed
- Frontend dependency hygiene: removed the dead
reactflowv11 entry frompackage.json— only@xyflow/reactv12 was ever imported (the v12 migration happened without dropping the old dependency), so nothing changes at runtime and the lockfile loses a whole unused subtree. Added@playwright/testas an actual devDependency plus atest:e2escript: the three E2E specs (e2e/auth|inventory|findings.spec.ts) andplaywright.config.tsexisted since Sprint 4 but imported a package that was never installed, making them unrunnable as committed;tsc --noEmitnow also type-checks them against the real API. - Dependency batch (August 2026 Dependabot triage): widened five backend ranges to their dependabot-proposed ceilings (
ruff <0.16,uvicorn <0.52,pytest-cov <8,redis <7.0,mypy <3.0), refreshedagent/go.mod(cilium/ebpf0.22.0,confluent-kafka-go2.15.0 — also pullingcobra1.10.2 /x/systransitively and creating the previously-missinggo.sumthe CI cache path already referenced), and refreshed the frontend lockfile within existing semver ranges via targetednpm update(next15.5.24,postcss8.5.26,nanoid3.3.18,@eslint/eslintrc3.3.6). Deliberately not taken: TypeScript 7 (the Go-based compiler rewrite — needs a dedicated migration), Recharts 3 (breaking API across every chart component), tailwind-merge 3 (targets Tailwind v4; project is on v3). A blanketnpm updatewas blocked by an unpublished transitive version (@xyflow/system@0.0.81) referenced during resolution hence the targeted form. - Compliance reverted to a single By Control score — the By Asset and By Findings views, and the compliance-wide default-metric setting, are removed. Three parallel scores per framework/section/control (
score_by_control/score_by_asset/score_by_finding, plus a "Default Metric" selector in Compliance Settings) added confusion without adding a decision that changes a customer's behavior — one number, By Control, is the framework's stated compliance posture.compliance_service.pyandComplianceFrameworkDetail/ComplianceSectionNode(backend and frontend) dropped every_by_asset/_by_finding/finding_*_countfield;ComplianceView,SectionHeatmap's toggle, and the settings default-metric field are deleted rather than deprecated.Unscoredstill counts toward the compliant side ofscore_by_control((Pass + Unscored) / Total), unchanged from the prior formula. FrameworkDetailmerges the Sections and Requirements accordion into one expandable component (Sections.tsx, replacingSectionHeatmap.tsx+RequirementAccordion.tsx), sections stay open by default instead of collapsed, and the score trend chart moved above the sections list and shrank (h-48→h-32,h-24→h-16) — the trend is a supporting signal, not the primary read. The "+ Set goal" affordance on a control with no configured target is now always visible (light gray), not hover-only.- Clicking a scored control (Pass/Fail) in
FrameworkDetailopens a two-tab drilldown panel (ControlDrilldownPanel.tsx): Findings (paginated, All/Pass/Fail filter, resource shown per finding — newresource_id/check_idexact-match filters onGET /api/v1/findings) and Assets (GET /api/v1/compliance/frameworks/{id}/control-assets, pass/fail counts per resource; expanding a row loads that resource's own findings with its own All/Pass/Fail filter). - Compliance Settings table: added an
Enabledcolumn (was:Listed) as the first column, ahead ofFramework/Goal(was: "By Control target") —Enabled, Goal, Frameworkleft-to-right. devadopted as the local integration trunk (ADR-014): work branches now branch offdevand merge into it locally (--no-ff, no push, no per-branch CI); a single batcheddev → mainPR triggers the full CI matrix once instead of once per small branch..github/workflows/ci.ymlgained apush: branches: [main, dev]trigger andif: github.event_name == 'pull_request' || github.ref == 'refs/heads/main'on the four heavy jobs (docker-build,backend-integration,backend-openapi-drift,backend-security), so pushes todevonly run the lightweight jobs (backend-lint,backend-testunit,frontend-check,agent-check, ~2–3min, no Docker/ArangoDB/Redis).mainremains protected — no direct commits, full CI still gates every merge into it.infra/terraform/awsgoat/module-1: fixed 5terraform validatedeprecation warnings onaws_api_gateway_deployment.apideploy_ba— surfaced while validating the AMI pin change above. Settingstage_name/variablesdirectly on anaws_api_gateway_deploymentis deprecated by the AWS provider; split into a dedicatedaws_api_gateway_stage.apideploy_baresource (mirroring the pattern already used elsewhere in the same file foraws_api_gateway_stage.api) and repointed the 4.invoke_urlreferences innull_resource.file_replacement_api_gw/_cleanupfrom the deployment to the new stage resource.terraform validatenow returns zero diagnostics for this module.infra/terraform/test-fixtures: pinned thecreate_ec2_instances-gated instances (public_exposed,private_clean) to a fixed Debian 11 AMI (ami-053413bdacb39d8dc,debian-11-amd64-20241202-1949) via newec2_instances_ami_idvariable, instead of adata "aws_ami"lookup that resolved to whatever the latest Amazon Linux 2023 build happened to be on a giventerraform apply— reproducible side-scanning test runs need a fixed OS/package baseline, not a moving target. Region-specific; re-pin ifaws_regionchanges. The DVWA instance invulnerable_apps.tfstill uses the dynamic Amazon Linux lookup (unrelated flag, out of scope for this pin).- Documented the two-tier tab detail panel, embedded exposure/blast-radius mini-graphs, and pill-filter/saved-Views pattern as the standard frontend convention for resource, finding, and path detail views (
CLAUDE.md); added corresponding roadmap items to Phase 2 (README.md).
Fixed
-
OpenAPI drift gate failing on the first PR that exercises it (
docs/api/openapi.json): the schema was last regenerated before Epic 14 Sprints 3–4, the compliance redesign, and the Scans page batch landed — every endpoint added since (GET /scans,/scans/{id}/logs,/views,/settings/sla,/findings/sla-summary,/findings/{key}/exposure-path,/compliance/frameworks/{id}+/trend+/control-assets,/attack-paths/{id}/narrative,/inventory/{key}/software+/compliance+/summary+/blast-radius) was missing from the committed spec. Regenerated viascripts/export_openapi.py; the drift job is green again. -
CI red on
devafter the Scans page batch: two failures in the lightweight jobs. (1)mypyfailed onscan_service.py:52(Returning Any from function declared to return "list[str] | None") —doc.get("logs", [])on adict[str, Any]returnsAny, which the newwarn_return_anyconfig correctly flags; fixed by binding through an annotated local (logs: list[str] = doc.get("logs") or [], the same pattern already used inciem_service.py). (2) The newScanTable.test.tsxrow-render assertion used the loose regex/10/, which matched both the findings-total span and the started-at timestamp cell (10:00:00viatoLocaleString) once real fixture data landed; replaced with exact-match assertions (/^10$/) for both counters. -
Invalid mypy option in
pyproject.tomlsilently skipped since introduction:disallow_redundant_casts = trueis not a valid mypy setting (the flag has always been--warn-redundant-casts; there is no "disallow" variant) — every run printed an "Unrecognized option" warning that was easy to miss in CI noise while the intended check never ran. Renamed towarn_redundant_casts = true; full-codebase mypy confirmed clean under the now-active option. -
Disconnecting a provider silently left its findings behind (
_purge_provider_resources,provider_service.py): the findings-deletion filter useddoc.provider_id == @provider_id, butFindinghas noprovider_idfield — onlyprovider+account_id/cluster_name— so the filter never matched anything and every disconnected provider's findings survived the "delete" untouched. Found while cleaning the dev database for a fresh scan; the test covering cascade-delete had seeded aFindingwith an artificialprovider_idthat real ingestion never writes, masking the bug. Fixed by scoping findings the same way vertices already are (provider+account_id/cluster_name); also addedSTS_ASSUMEROLE_ALLOW,ATTACHED_POLICY, andHAS_FINDINGto the orphaned-edge sweep, which previously missed them entirely. -
GET /api/v1/scans500ing against real data:scan_jobsis shared between CSPM jobs (task_name: cspm_scan/*, hasframeworks) and Side-Scanning jobs (task_name: side_scan/*, noframeworksfield at all) —ScanJob(**doc)raised a PydanticValidationErroron the first side-scan document it hit. Fixed by filtering toSTARTS_WITH(j.task_name, "cspm_scan/")inscan_service.py's AQL and inget_scan_job/get_scan_job_logs's Python-side check — Side-Scanning already has its own dedicated page, so CSPM-only is the correct scope here, not a workaround. -
resource_type: "unknown"on findings/inventory: traced toprowler_service.py::_normalize, which falls back to the literal string"unknown"(an Ogum-invented default, not a Prowler value) wheneverresult.metadatais falsy.compliance_service.py's asset-grouping query now prefers a realresource_typeover"unknown"when both are present for the same resource (SORT g.f.resource_type == "unknown" ASC). Also manually removed 17 orphanedcheck_id == "unknown"findings left over from two stale scan runs superseded by correct re-scans. -
EBS snapshot SBOM generation returning empty on real AWS accounts (Epic 03):
_generate_sbom(app/workers/tasks/side_scanning.py) rantrivy vm ebs:{snapshot_id} --format cyclonedximmediately afterrun_trivy_ebshad already read the same snapshot's blocks through the sametrivy-serversidecar for the vuln/secret scan — a second, back-to-backebs:GetSnapshotBlock/ListSnapshotBlockspass over the same snapshot is prone to EBS Direct API throttling, which Trivy surfaces as exit 1 with empty stdout rather than a distinguishable rate-limit error. This was flagged as a known, unfixed residual gap whenscan_ec2_instance_v2was first verified end-to-end against a real account. Confirmed by manually re-running the sametrivy vm ebs:... --format cyclonedxcommand in isolation (no prior read of the same snapshot) against a real snapshot, which succeeded every time. Fixed by retrying_generate_sbomwith linear backoff (3 attempts, 5s/10s waits) instead of treating the first failure as final; still best-effort and never fails the parent scan. -
Terminated EC2 instances resurrected as
status: "active", permanently breaking "Scan Now" (app/services/prowler_inventory.py): found live indev-tenant— twopublic-exposed/private-cleantest-fixtures EC2 instances, replaced by aterraform applydays earlier, kept showing as active inventory with a "Scan Now" button that always failed 422 ("Resource could not be resolved to a scannable target"). Root cause: AWS'sdescribe_instances(and therefore Prowler's own EC2 collector) keeps reporting a terminated instance for a limited window after termination —extract_inventory_from_findingsunconditionally wrotestatus: "active"for every resource Prowler reported, with no check against the EC2 instance's ownstatefield. This silently undid a correct prior soft-delete (the stale doc still carried adeleted_atfrom days earlier) the next time a scan happened to catch the instance inside that post-termination window, and left it permanently stranded as "active" once AWS's window elapsed and Prowler stopped reporting it too — at that point_soft_delete_stale(app/workers/tasks/cspm_scan.py) never gets a chance to mark it deleted, since that only fires for resources absent from the current scan, not ones that were merely absent from a past one. Fixed by skippingec2_instancefindings withraw_metadata.state == "terminated"during extraction entirely, so_soft_delete_stale's existing absence-based logic catches them on the very next scan instead of a second code path fighting it. Manually corrected the two stale documents found live during this investigation. -
AWSGoat
terraform applyfailed on macOS (infra/terraform/awsgoat):terraform validate/plandon't execute provisioners, so this only surfaced on a real localterraform apply. Upstream'slocal-execprovisioners assume a GNU userland (their own CI runs on Ubuntu): (1)sed -i 'script' filewith no attached suffix is GNU-only — BSD sed (macOS's default/usr/bin/sed) requires an in-place suffix argument and fails withsed: -I or -i may not be used with stdinotherwise. Fixed across all 9sed -icall sites (module-1 and module-2) with the portablesed -i.bak 'script' file && rm -f file.bakidiom, accepted identically by BSD and GNU sed. (2)module-2'sRDS_URL=$${RDS_URL::-5}uses bash ≥ 4.2 negative-length substring syntax; macOS ships bash 3.2 (frozen there for GPL licensing reasons) and fails withsubstring expression < 0. Replaced with$${RDS_URL%:*}(strip from the last:onward), which is bash-3.2-safe and also correct regardless of port digit count, unlike upstream's hardcoded "last 5 characters" assumption. Also documented a pre-existing, unrelated blocker ininfra/terraform/awsgoat/README.md: module-1'spopulate_tableprovisioner needsboto3onpython3— not vendored or auto-installed, now called out as a prerequisite. -
EC2 side-scanning failed against every real AWS account (Epic 03): end-to-end browser verification of the "Scan Now" trigger against a live dev/test AWS account (not moto) surfaced a chain of issues that no prior test caught, because nothing had ever exercised the trigger against a real account before. (1)
_get_aws_session(cloud_utils.py) always built the STS client used forAssumeRolefrom ambient credentials, ignoring any staticaws_access_key_id/aws_secret_access_keyalso present on the tenant's provider config — worked in production (EC2 instance profile provides ambient identity) but failed withNoCredentialsErrorin any environment without one, including local dev. Now signs the STS client with the static keys when both are provided, falling back to ambient credentials otherwise. (2)enqueue_side_scan()resolvedexternal_idfor its ownresolve_ec2_scan_metadatasession but never passed it to thescan_ec2_instance_v2.delay()/scan_lambda_function.delay()calls, and those task signatures didn't even accept the parameter — so the worker's ownAssumeRolecall always omittedExternalId, causing a persistentAccessDeniedagainst any role whose trust policy requires one (confused-deputy protection), even after IAM permissions were otherwise correct. This looked like AWS IAM eventual-consistency lag at first; it was a reproducible code bug. Fixed by threadingexternal_idthrough all three task signatures. (3) Thetrivybinary was never installed indocker/backend.Dockerfileat all — the whole Sprint 2 sidecar architecture assumed it was present. Added the official install script alongside the existing dependency install step. (4) Oncetrivywas present, every call site (run_trivy_ebs,run_trivy_rootfs,run_trivy_image, SBOM/SARIF generation, plus the dead-codescan_with_ebs_direct) used the pre-0.72 syntaxtrivy client --server <url> vm/rootfs/image/sbom ...— current Trivy releases only supportclientfor image scanning (trivy client [flags] IMAGE_NAME);vm,rootfs,image, andsbomare top-level subcommands that each take--serverdirectly, with noclientwrapper. The old form FATALed immediately withunknown flag: --server. Fixed across all 8 call sites. With all four fixes applied,scan_ec2_instance_v2completed successfully against a real EC2 instance and EBS snapshot for the first time this project has been able to verify. Known residual gap, not fixed here: SBOM generation for EBS snapshots (_generate_sbom) still returns nothing on a real account — the underlying vuln/secret scan against the same snapshot in the same task run succeeds, so this looks like an EBS Direct API rate limit or a quirk of Trivy's[EXPERIMENTAL]vmmode rather than a syntax issue; SBOM generation is best-effort by design and does not fail the scan. -
Side-scanning jobs untrackable after the manual/auto trigger (Epic 03): browser-testing the newly-shipped "Scan Now" button surfaced two bugs no prior test caught because nothing had ever populated real side-scanning job data before. (1)
scan_ec2_instance_v2/scan_lambda_functionignored thejob_idthe caller passed in and minted their own viastart_discovery_job()— unlikescan_k8s_container/scan_container_image, which correctly reuse it.enqueue_side_scan()'s pre-createdscan_jobsdoc (the one returned to the UI) was silently orphaned atstatus: "queued"forever, while a second, disconnected doc with a different key received the real status. Fixed by reusing the passedjob_id(scan_job_id = job_id; update_job_to_running(db, scan_job_id)), matching the K8s/ECR convention. (2)/side-scanning'spage.tsxreaddata.items/data.totaldirectly off theuseQueryresult, butsideScanApi.listJobs()returns a raw Axios response — the payload is one level deeper atdata.data.items/data.data.total. The?? []fallback made "wrong shape" indistinguishable from "genuinely zero jobs," so the jobs table, KPI cards, and job count have shown nothing for every job type since Sprint 4 (PR #156), not just EC2/Lambda — the existing test's mock skipped the Axios wrapper too, so it couldn't have caught this either. Both fixes include regression tests exercising non-empty job data, which no test in this area did before. -
EC2 side-scanning silently stopped detecting malware/webshells (Epic 03): Sprint 2's move to EBS Direct API (
scan_ec2_instance_v2) never mounts a filesystem, but YARA needs actual file access — it can't scan raw disk blocks. The Sprint 1 task that did mount and run YARA (scan_ec2_instance) was left in the codebase but never wired to any trigger after Sprint 2 shipped, so malware detection silently went inactive with no error, no log, no test failure. Found while auditing Epic 03 documentation against the shipped code. Fixed by adding an optional scoped-mount step toscan_ec2_instance_v2: whenOGUM_SCANNER_INSTANCE_IDand an availability zone are available, it creates a volume from the same snapshot, attaches+mounts it read-only to the scanner's own instance, and runs YARA in parallel with the EBS Direct API vuln+secret scan — always unmounting/detaching/deleting the volume infinally, and skipping (with a log line, never silently) when the scanner isn't running on EC2. Trivy (vuln+secret) remains the primary analysis engine — its failures still propagate and fail the task; YARA is best-effort and never blocks the primary scan. The now-fully-redundantscan_ec2_instance(Sprint 1 task) was removed, along with its only-used-there_normalise_cve/_normalise_secrethelpers;cleanup_orphan_snapshots(still active) keeps its own test coverage in a trimmedtest_side_scanning_task.py. Also fixed a related bug never previously exercised: the volumedetach_volumecall was missingInstanceId, which only surfaced once a real attach+detach cycle was covered by a test. -
Every Prowler CSPM finding classified as MEDIUM severity, no INFORMATIONAL findings ever surfaced:
ProwlerService._normalizecomputedseverity_str = str(getattr(check_meta, "Severity", "medium")).lower().prowler.lib.check.models.Severity— the real, required type ofCheckMetadata.Severity— subclasses(str, Enum)without overriding__str__, the same shape as theStatusbug fixed earlier (see below):str(Severity.high)returns"Severity.high", not"high". That string never matched_SEVERITY_MAP's keys, so every finding silently fell through to theSeverityLevel.MEDIUMdefault regardless of its real severity — found after a full-catalog scan against the test AWS account showed all 407 findings as MEDIUM with zero at any other level. Fixed by readingseverity_obj.valueinstead ofstr(severity_obj), mirroring the existingstatus_obj.valuefix. Same root-cause class as theStatusbug, different enum, undetected until now because the existing test suite's_make_resulthelper passed a plain Python string forSeverityrather than the real enum member every check actually carries at runtime. -
Attack path detection returning zero results (Epic 02):
find_privilege_escalation_pathsand_detect_tc03_overpermissioned_to_dbinattack_path_service.pywere traversingASSUMES_ROLEedges (resources→identities) for identity→identity paths. Corrected toSTS_ASSUMEROLE_ALLOW, ASSUMES; TC-03 traversal now usesOUTBOUND i STS_ASSUMEROLE_ALLOW, ASSUMES, STORES_SENSITIVE_DATA. End-to-end result: 7 attack paths detected in the Terraform test environment (was 0). -
STORES_SENSITIVE_DATAkey mismatch with Prowler-created identities (Epic 02):_create_data_access_edgesbuilt edge_fromreferences usingarango_key()(structured key). When Prowler CSPM creates an identity document first (hash key), the discovery_upsertupdates it in-place, keeping the hash key. The in-memory structured key produced dangling edge_fromreferences — TC-03 AQL traversal found no outbound edges and returned zero results. Fixed:_create_data_access_edgesnow queriesidentitiesfrom ArangoDB withFILTER has_admin_policy == true, resolving actual_keyvalues regardless of insertion path (discovery vs. Prowler). -
Attack path
STORES_SENSITIVE_DATAedge coverage (Epic 02):_create_data_access_edgesnow queries the fulldata_assetscollection instead of only S3 buckets from the current discovery run. DynamoDB tables, SecretsManager secrets, and any other data assets discovered by Prowler CSPM now correctly receive edges from admin IAM identities, enabling TC-03 detection. -
EC2 → IAM role
ASSUMES_ROLEedge (profile name mismatch):_create_resource_edgesnow uses_build_profile_role_map(one paginatedlist_instance_profilescall) to resolve instance profile names to role arango keys. Previously the lookup silently failed when the profile name differed from the role name (e.g.,foo-profilevsfoo-role). -
S3
is_publicdetection:_list_s3_bucketsnow callsget_public_access_blockper bucket and setsis_public=truewhen any block setting is disabled. Buckets with no public access block configuration (the default prior to 2023) are also markedis_public=true. -
Array query params silently ignored by the backend (
apiClient): axios's defaultparamsSerializerencodes arrays askey[]=a&key[]=b, but FastAPI'sQuery(list[str])only recognizes repeated bare keys (key=a&key=b) — the bracketed form is a different, unrecognized param name, so the filter silently defaulted to "no filter" instead of erroring. Found while browser-testing the new inventory multi-select filters (request succeeded with HTTP 200 but returned unfiltered results). Fixed with a customparamsSerializeron the sharedapiClient(frontend/src/lib/api.ts) that emits repeated bare keys for array values. This also fixes the same latent bug inPOST /inventory/discover?regions=..., which had no test coverage catching the wire format. -
Compliance page listing thousands of duplicate "frameworks" instead of ~45 (Epic 01):
compliance_service.get_compliance_summarygrouped findings by the rawframework_mappingstring, which Prowler stores as{framework_slug}/{control_id}(e.g."CIS-7.0/1.1"). Grouping on the full string turned every individual control into its own top-level "framework" — 2,506 sidebar entries in the dev dataset instead of the 45 real frameworks. Fixed by splittingprefixandcontrolbefore aggregating (backend/app/services/compliance_frameworks.py+ rewrittencompliance_service.py). -
Findings
frameworkfilter dropdown never matched any real data:FindingFilters.tsxhardcoded framework values ("CIS-AWS-2.0","PCI_DSS","NIST_800_53", ...) that don't correspond to any string Prowler actually writes toframework_mapping— selecting any of them silently returned zero results. Replaced with real slugs (CIS-7.0,PCI-4.0,NIST-800-53-Revision-5, ...). -
Findings
frameworkfilter only matched exact strings: a link like/findings?framework=CIS-7.0(family/version id, no control suffix) never matched anything, sinceframework_mappingstores the full"CIS-7.0/1.1"string.list_findings's framework filter now also matches by prefix (fw == sel OR STARTS_WITH(fw, sel + "/")). -
Compliance "Top Failing Controls" ignored the selected framework: the list was always computed globally regardless of which framework card was selected in the sidebar.
GET /api/v1/compliance/summarynow accepts an optional?framework=to scope it. -
identity_type/asset_typemissing on CSPM-enriched identities and data assets (Epic 00):prowler_inventory.py::extract_inventory_from_findingswrote a genericresource_typefield regardless of which ArangoDB collection a document was routed to, butIdentityrequiresidentity_typeandDataAssetrequiresasset_type— neither isresource_type. In the dev dataset, 83% of identities and 100% of data assets created this way were missing their required field. This silently broke attack path TC-02 detection, which filtersdata_assetsbyresource_typeand therefore only ever matched CSPM-enriched assets, never natively-discovered ones. Fixed by classifyingidentity_type(role/user/group/service-account) and settingasset_type = type_nameper collection. -
Every Prowler CSPM finding classified as FAIL, even PASS results (Epic 01):
ProwlerService._normalizecomputedstatus_str = str(getattr(result, "status", "FAIL")).upper().prowler.lib.outputs.common.Statussubclasses(str, Enum)but does not override__str__, sostr(Status.PASS)returns"Status.PASS"— not"PASS". That string never matched_STATUS_MAP's keys, so every result (PASS, FAIL, MANUAL alike) silently fell through to theFindingStatus.FAILdefault. Root-caused after noticing 100% of findings in the dev dataset were FAIL, including checks against resources the Terraform test fixtures document as compliant baselines (e.g.s3_bucket_public_accessreporting FAIL for a bucket with all public access blocked). Fixed by readingstatus_obj.valueinstead ofstr(status_obj). A re-scan of the CIS-AWS-2.0 framework against the test account went from 111/111 FAIL to 60 FAIL / 51 PASS. This bug predates and is unrelated to the Compliance/Findings PRs above — only Prowler CSPM findings were affected; Checkov/IaC findings passFindingStatusenum members directly and were never affected.
Added
- Admin Jobs: job name, start time, duration, and captured logs (Epic 06):
/admin/jobsonly showed job ID, tenant, status, provider, created-at, and worker — no way to tell what a job actually did or how long it took, and clicking a row did nothing. Rows are now clickable, opening a detail panel (JobDetailPanel, mirroring the existing InventoryDetailPanelslide-over pattern) with a human-readable job name, created/started/completed timestamps, duration, checks/findings counts, error message (if failed), and a log viewer. The list table gained Job Name, Started At, and Duration columns. Backend:run_cspm_scannow attaches aJobLogHandler(app/workers/tasks/job_logging.py) to the root logger for the scan's duration, capturing its own logging plus anything that propagates through it (Prowler, boto3 at WARNING+) into the job doc'slogsfield (bounded to the last 500 lines) — theJobDetail.logsfield and its SSE streaming endpoint already existed but nothing had ever populated them. Side-scanning (ec2/lambda) and discovery jobs don't have log capture wired yet — their detail panel shows "No logs available for this job" rather than an error. Along the way, fixed a job-naming bug found while building this:ScanJobhad notask_namefield, so every CSPM-scan-shaped job doc — including IaC scans (provider="iac") and, after a fallback fix, side-scan (ec2/lambda) docs that had noproviderfield at all — displayed ascspm_scan/*regardless of what it actually was.ScanJoband the side-scan job-doc writer (app/services/side_scanning/trigger.py) now settask_nameexplicitly (cspm_scan/{provider},iac_scan/iac,side_scan/ec2,side_scan/lambda);admin_service._infer_task_nameprovides the same correction for already-persisted docs from before this fix. - AWSGoat vendored and unified into
test-fixturesas a single-apply, end-to-end vulnerable-app test target:infra/terraform/awsgoat/module-1(serverless blog: Lambda + API Gateway + DynamoDB + S3, XSS/SQLi/IDOR/SSRF chained into IAM privilege escalation via IMDS credential theft) andmodule-2(ECS/Fargate HR payroll app, SQLi/file-upload chained into container breakout + IAM privilege escalation), vendored from ine-labs/AWSGoat (MIT, commitb24869a) along with its attack manuals and minimum deploy IAM policy. Complementstest-fixtures/(synthetic, no running app) by exercising the full chain Ogum needs to detect end-to-end — Ogum.Dynamic catching the app-layer vulnerability, Ogum.Pulse/CDR the resulting cloud API activity, Ogum.Graph the reconstructed attack path — rather than only CSPM misconfigurations. Both modules are wired intotest-fixtures/awsgoat.tfas toggleable child modules (create_awsgoat_module1/create_awsgoat_module2, both off by default) soterraform applyfromtest-fixtures/deploys everything from one place instead of three separate directories/states. Making them child modules required three documented local patches on top of upstream (seeinfra/terraform/awsgoat/README.md#local-patches-applied-on-top-of-upstream): dropping each module's ownprovider "aws"block (inheritstest-fixtures'~> 5.0provider instead — module-2 was previously pinned to~> 3.27, validated clean under 5.x with only deprecation warnings, no breaking changes), prefixing module-1's local file references with${path.module}(they resolved against the root's working directory, not the module's, once this stopped being a standalone root), and renaming an S3 bucket both modules declared with the identical literal name. Deliberately excludes upstream's GitHub Actions auto-deploy workflows (manualterraform applyonly, consistent with this repo's no-CI-cloud-deploy posture). Seeinfra/terraform/awsgoat/README.mdfor safety notes — these are real, deliberately internet-reachable exploitable applications, not simulated findings, and still require a dedicated disposable sandbox account. - Known-vulnerable test fixtures: Metasploitable + DVWA (
infra/terraform/test-fixtures): two opt-in EC2 instances (create_metasploitable_ec2,create_dvwa_ec2, bothfalseby default) for validating Ogum.Dynamic side-scanning against real CVEs and exposed secrets, complementing the synthetic CSPM misconfigurations already inmain.tf. DVWA runs the officialvulnerables/web-dvwaDocker image viauser_data— no custom AMI needed. Metasploitable requires a self-built AMI (Rapid7 doesn't distribute a public one; license restricts it to isolated local VM use), looked up by tag or supplied directly viametasploitable_ami_id. Both instances deploy into the private subnet behind a new security group with no default ingress —vulnerable_apps_allowed_cidrs(empty by default,0.0.0.0/0blocked by a Terraform validation rule) is the only way to open network access; otherwise reachable only via SSM Session Manager (IAM role/instance profile attached automatically). - EC2/Lambda side-scanning trigger — manual and automatic (Epic 03):
/side-scanningwas empty in every environment, including production — nothing ever calledscan_ec2_instance_v2/scan_lambda_function. K8s and ECR scans only fire from external webhooks (a real DaemonSet, a real ECR push); EC2 and Lambda had no equivalent, no UI action, and no automatic trigger after discovery.POST /jobs/{id}/retryforec2/lambdajobs was also a silent no-op (else: pass). NewPOST /api/v1/side-scans/trigger({resource_key}) — surfaced as a "Scan Now" button in the Inventory detail panel for activeec2_instance/lambda_functionresources — and a first-seen auto-trigger hooked intorun_cspm_scan(AWS only): each EC2/Lambda resource gets exactly one side-scan the first time it's discovered, deduplicated against existingscan_jobsentries, never re-triggered periodically (keeps EBS snapshot cost and Celery/trivy-server load bounded — later re-scans are manual or via the already-free dailyrescan_sboms). Retry forec2/lambdajobs now actually re-enqueues. All three call sites (manual trigger, retry, auto-hook) share one implementation inapp/services/side_scanning/trigger.py. Along the way, confirmed directly against the installedprowler-corepackage (not just its docs) that its EC2Instancemodel has noavailability_zoneor volume/block-device fields at all —volume_idis required just to create the scan snapshot, so the trigger makes one batchedec2:DescribeInstancescall (resolve_ec2_scan_metadata) at scan time. This doesn't reintroduce a second discovery pipeline (the "Prowler as single source of truth" rule applies to inventory sync, not to the act of scanning itself). - Prowler as the single source of truth for Inventory — Part 1/3 (Epic 00): first step of retiring the native AWS discovery pipeline (
discovery.py) in favor of building the resource graph entirely from Prowler scan output, avoiding two parallel, schema-incompatible inventory paths.ProwlerService.run_*_scannow acceptframeworks: list[str] | None = None— omitting frameworks (the new default for the eventual scheduled scan) runs prowler's full check catalog for the provider instead of a curated 3-framework subset, sinceScan()with nocompliances=filter already runs every check and each result still tags itself with every framework it belongs to.extract_inventory_from_findingsnow extracts relationship data already present in Prowler'sresource_metadata(EC2security_groups/subnet_id/iam_instance_profile, IAMtrust_policy/policies) into the fieldsgraph/iam_edges.pyalready reads with zero AWS API calls — so IAM deep-model edges (STS_ASSUMEROLE_ALLOW,ASSUMES,ATTACHED_POLICY) work correctly off Prowler-sourced identities without any change to that module. One confirmed gap: prowler's LambdaFunctionmodel has no execution-role field at all — closed with a small targetedlambda:list_functionscall (reusing Prowler's already-resolved credentials) rather than a broader rediscovery pass.VpcSubnet.public(route-table-derived) is now also recognized as an internet-exposure signal, a more accurate source than themap_public_ip_on_launchflag native discovery used to rely on. Parts 2/3 (decoupled resource-edge builder, replacing_create_resource_edges's in-memory coupling to live discovery) and 3/3 (repoint all call sites, retirediscovery.pyand its Celery Beat schedule) land separately. - Prowler as the single source of truth for Inventory — Part 2/3 (Epic 00): new
graph/resource_edges.py::build_resource_edgesrebuildsBELONGS_TO(EC2→VPC via subnet),ATTACHED_TO(SG→EC2), andASSUMES_ROLE(EC2 and Lambda→IAM role) purely by queryingresources/identitiesalready persisted in ArangoDB — no in-memory coupling to a live discovery run, unlike the_create_resource_edgesit replaces.MEMBER_OF(IAM User→Group) is also rebuilt this way; confirmed against prowler-core thatiam_administrator_access_with_mfa/iam_group_administrator_access_policyauditAwsIamGroupas their primary resource, soGroup.usersis actually populated by real scans, not just a theoretical field._create_data_access_edges(STORES_SENSITIVE_DATA) moved unchanged tograph/data_access_edges.py.run_cspm_scannow runsbuild_iam_edges+build_resource_edges+build_data_access_edges+compute_exposed_internetafter every inventory upsert, and soft-deletes resources/identities/data_assets absent from the scan (scoped to tenant+provider — a single Prowler run covers every resource type in one pass, unlike the old per-type/per-region soft-delete). Two accepted gaps carried over from Part 1:ROUTES_TRAFFIC(Internet Gateway→VPC) has no equivalent in prowler-core (noInternetGatewaymodel) and is not rebuilt; EC2 instance-profile→role resolution is name-based only (the old liveiam:ListInstanceProfileslookup for mismatched names is not reintroduced). New gap found while wiring this up: prowler's IAMRole.inline_policiesislist[str]of names only (no document content), so the inline-policy wildcard-action detectiondiscovery.pyused fordangerous_permissionshas no equivalent —has_admin_policyis still derived correctly (managed-policy ARN suffix match, e.g.AdministratorAccess/PowerUserAccess), only the inline-policy path is gapped. Also fixed a pre-existing, previously-undetected bug found while testing this:prowler_inventory.py's AWS collection-routing regex matched"Group"/"Role"as bare substrings, misroutingAwsEc2SecurityGroup,AwsAthenaWorkGroup,AwsAutoScalingAutoScalingGroup,AwsLogsLogGroup, andAwsWafRuleGroup/AwsWafRegionalRuleGroupinto theidentitiescollection instead ofresources— silently brokeATTACHED_TOedges for every EC2 instance with a security group. AWS routing now matches by service prefix (^Aws(Iam|RolesAnywhere)) instead of bare-word matching. - Prowler as the single source of truth for Inventory — Part 3/3 (Epic 00): repointed every AWS discovery call site to
run_cspm_scan— provider registration and re-trigger (providers.py::_dispatch_discovery), the ad-hoc/inventory/discoverendpoint, the admin panel'sTaskType.DISCOVERYtrigger, andtrigger_job's retry path.POST /api/v1/scansand the scheduled Celery Beat scan (trigger_all_cspm_scans) now default toframeworks=None(full check catalog) instead of a curated subset (CIS-AWS-2.0, or["CIS-AWS-2.0", "PCI_DSS_v4", "SOC2"]depending on the entry point) — the whole point of running the full catalog is that every check still tags its result with every framework it belongs to, so a curated subset was strictly less complete for no benefit.trigger_all_cspm_scansalso stops skippingk8s/kubernetesproviders (Prowler does support Kubernetes CSPM scans; the old skip was a leftover from the curated-framework-per-provider table). Celery Beat'sdev-trigger-aws-every-6hentry (and the 1h offset oncspm-scan-all-tenants-every-6hthat existed to let it finish first) are both removed — a CSPM scan is now the entire discovery pass, so there's nothing left to wait on.ConnectWizard.tsx's "Connecting..." copy now sets the expectation that this runs a full scan (minutes, not moments) instead of implying a quick API listing call. - Manual QA test scenarios (
docs/qa/, Epic 13 US-13.08): structured pre-conditions → steps → expected result → what-to-check-if-it-fails scenarios for the flows automated tests don't fully cover — provider onboarding, a full CSPM scan staying consistent across Findings/Inventory/Compliance/Attack Paths at once (including direct regression checks for theStatus/Severityenum-stringification bugs fixed above), the attack-path graph UI, side-scanning (documents that EC2/Lambda scans have no HTTP trigger endpoint yet, only K8s and ECR webhooks do), and the admin panel.docs/scanning.mdupdated to reflectframeworks=None(full catalog) as the default instead of the old curated 3-framework example. - API reference via Swagger UI without the full stack (Epic 13 US-13.07):
docs/api/openapi.json— the FastAPI-generated OpenAPI schema — is now committed to the repository, generated bybackend/scripts/export_openapi.py(pure route introspection, no ArangoDB/Redis/network access needed since everySettingsfield has a default). Newswagger-uiservice indocker-compose.yml(swaggerapi/swagger-ui, port8085) serves it standalone —docker compose up -d swagger-uiwith nothing else running. New CI jobBackend — OpenAPI Schema Driftruns the export script in--checkmode and fails the build if the committed schema doesn't match what the current code generates, so this can't silently go stale the waydocs/scanning.md's framework example did.
Removed
-
backend/app/workers/tasks/discovery.py: AWS discovery is retired —discover_aws,discover_aws_basic, all_list_*AWS API functions,_create_resource_edges,_create_data_access_edges(moved tograph/data_access_edges.py),_build_profile_role_map,_classify_iam_role, andretry_with_backoff(only used by the now-removed_list_*functions) are gone. Genuinely cross-cutting helpers used by other providers' discovery tasks and by side-scanning —_get_aws_session,_get_tenant_db,_upsert,_mark_stale_deleted,_set_provider_status— moved unchanged to the newbackend/app/workers/tasks/cloud_utils.py. Azure, GCP, and Kubernetes discovery tasks are untouched by this change. -
Compliance page redesign: unified framework families with drill-down sections (Epic 01): replaces the flat, unusable framework list with family/version grouping — near-duplicate framework versions (CIS AWS Foundations Benchmark ships 8 slugs, NIST 800-53 has 2 revisions, ISO 27001 has 2 editions, PCI DSS has 2 versions, KISA ISMS-P has English/Korean slugs, FedRAMP has 3 profiles) now collapse into one sidebar entry with a version switcher, cutting the dev dataset's sidebar from 2,506 entries down to 31. Controls within a version are grouped into sections — NIST 800-53 uses its standard 2-letter control family codes (AC, CM, IA, ...), CIS/ISO27001/PCI use their dot-numbered section prefixes (
backend/app/services/compliance_frameworks.py::derive_section). NewFrameworkSidebar(searchable),FrameworkDetail(version tabs + section breakdown + "View findings" deep link), andTopFailingControls(now scoped to the selected framework) components replace the single-file page. -
Findings: multi-select filters and real stats-backed summary (Epic 01):
/findingspage's severity/status/provider/framework/source dropdowns are now checkbox multi-selects (reusingcomponents/ui/MultiSelectFilter), and the summary cards now show real counts fromGET /api/v1/findings/stats(which gainedby_provider) instead of counting only the current page of results — clicking a card toggles that filter. Backend:GET /api/v1/findingsand/findings/exportaccept repeated query params forseverity/status/provider/source;frameworkmatches if any selected framework is inf.framework_mapping(AQLINTERSECTION). -
Inventory: category and multi-select filters (Epic 00):
/inventorypage now supports selecting multiple providers, categories, regions, and accounts at once (checkbox dropdowns with "All"/individual selection), replacing the old single-select dropdowns. Resource types are grouped into 7 categories (Compute, Containers, Database, Storage, Networking, Security & Identity, Other) via a client-side taxonomy (frontend/src/lib/inventoryCategories.ts); unmapped types fall back to "Other". NewInventorySummarycards show asset counts by provider and by category — clicking a card toggles that filter. Backend:GET /api/v1/inventoryaccepts repeated query params (?provider=aws&provider=azure) filtered via AQLINinstead of equality;InventoryStatsgainedby_regionandby_account_idbreakdowns to populate the region/account filter options.ProviderTabscomponent removed (superseded by the summary cards). -
Signal Score roadmap (Ogum.Pulse, Epic 07 Sprint 7 — planned): documented the high-confidence detection design — an event is only prioritized as high-confidence when it both deviates from the per-principal IAM behavioral baseline (actions, IPs, regions, user-agent) and matches a known TTP, instead of treating either condition alone as sufficient. See
README.mdFeatures/Roadmap. -
Side-scanning EBS Direct API (Epic 03 Sprint 2):
scan_ec2_instance_v2Celery task replaces the volume/mount pipeline withtrivy client vm ebs:{snapshot_id}— no volume creation, no OS mount required. Snapshots are deleted infinallyviadelete_snapshot_safe(now non-raising). Newrun_trivy_ebs()intrivy_analyzer.pyreturns(vulnerabilities, secrets)using Trivy'sSeverityfield as the primary severity source (CVSS only as fallback forUNKNOWN). -
Trivy sidecar server:
trivy-serverservice indocker-compose.yml(port:4954, persistenttrivy-cachevolume,aquasec/trivy:latest). Celery worker gainsTRIVY_SERVER_URLenv var anddepends_on: trivy-server. -
Secret scanning migrated from TruffleHog3 to Trivy:
secret_analyzer.pyremoved; Trivy--scanners vuln,secretcovers both CVEs and secrets in EBS Direct scans.Matchfield (secret value) is never stored — Trivy redacts it as****. Sprint 1scan_ec2_instanceno longer performs secret scanning. -
SBOM generation and persistence:
_generate_sbom()generates CycloneDX viatrivy client vm ebs:{id} --format cyclonedxper scan._persist_sbom()upserts to the newsbomsArangoDB collection with aHAS_SBOMedge from the EC2 resource. Schema additions:sbomsvertex collection,HAS_SBOMedge collection, indexes on(tenant_id)and(tenant_id, resource_id). -
rescan_sbomsCelery Beat task: daily at 03:00 UTC — queries stored SBOMs, runstrivy client sbomagainst each, upserts new CVE findings withdetection_method: sbom_rescan. Added tocelery_app.pybeat schedule. -
scan_lambda_functionCelery task: downloads Lambda deployment package via pre-signed URL to/dev/shm/ogum-{job_id}/(RAM disk), extracts ZIP, runstrivy fs, normalises findings withdetection_method: lambda_zip, and always wipes the RAM disk infinally. Image-based Lambda deferred to Sprint 3. -
scan_with_ebs_direct()insnapshot_manager.py: low-level helper that builds and runs thetrivy clientcommand for EBS Direct API, returns parsed JSON. -
_normalise_cve_v2(),_normalise_trivy_secret(): Sprint 2 normalisation helpers using Trivy Severity as primary and annotatingdetection_methodinraw_output. -
_trivy_severity()helper: maps Trivy severity strings toSeverityLevel; falls back tocvss_to_severity()only forUNKNOWN. -
16 unit tests for Sprint 2:
TestRunTrivyEbs(7 tests — parse vulns, parse secrets, severity precedence, empty output, nonzero exit, ignorefile flag, target field);TestDeleteSnapshotSafe(2 tests — success and non-raising error);TestScanWithEbsDirect(4 tests — command structure, ignorefile, nonzero exit, empty stdout). -
9 integration tests for Sprint 2 in
test_side_scanning_v2.py:TestScanEc2InstanceV2(snapshot lifecycle, 2+1 findings persisted, SBOM+edge stored, cleanup on trivy error, severity from Trivy field not CVSS);TestScanLambdaFunction(ZIP download+scan+finding, cleanup on error);TestRescanSboms(SBOM found+rescanned, empty DB, severity from Trivy field). -
K8s container scan via DaemonSet (Epic 03 Sprint 3):
scan_k8s_containerCelery task scans running containers via/proc/<PID>/rootwithout restarting pods.run_trivy_rootfs()added totrivy_analyzer.py— callstrivy client rootfs {path}against the sidecar server with--skip-dirs /proc,/sys,/dev,/runto avoid virtual filesystem loops. SBOM generated per container scan via_generate_sbom_rootfs(). Findings usedetection_method: k8s_proc_root(CVEs) andtrivy_secret_k8s(secrets), linked to the Pod vertex viaHAS_FINDING. -
DaemonSet manifest:
infra/k8s/ogum-scanner-daemonset.yaml—hostPID: true,privileged: true,hostPath: /procmounted at/host/proc, minimal ClusterRole (get/listonpods/nodesonly — no access to Secrets or ConfigMaps),hostNetwork: false. -
K8s scan webhook:
POST /api/v1/side-scans/webhooks/k8s-scan— DaemonSet posts scan trigger; backend validatesx-ogum-tokenagainsttenant_config.scanner_token, createsscan_jobsrecord, enqueuesscan_k8s_containertask, returns202. -
5 unit tests for
run_trivy_rootfs: parse vulns, parse secrets (no Match field), nonzero exit raises RuntimeError, empty output returns empty lists,--skip-dirsincluded in command. -
5 integration tests (
test_side_scanning_k8s.py): findings persisted in ArangoDB, SBOM+edge stored, FileNotFoundError on missing proc path, tenant isolation (security), severity from Trivy field. -
3 integration tests (
test_k8s_webhook.py): valid token → 202 + task enqueued + scan_jobs created, invalid token → 401, missing header → 422. -
infra/k8s/directory created for Kubernetes manifests. -
Registry image scan (Epic 03 Sprint 4):
scan_container_imageCelery task scans ECR and any OCI-registry image viatrivy client image {uri}@{digest}— nodocker pull, runs vuln + secret + misconfig scanners.run_trivy_image()added totrivy_analyzer.pyreturning(vulns, secrets, misconfigs)._normalise_misconfig()normalises Trivy misconfigurations intoFindingwithresource_type=container_image. SARIF report generated per scan via_generate_sarif()and upserted to the newsarif_reportsArangoDB collection (_persist_sarif()). Schema additions:sarif_reportsvertex collection with persistent index on(tenant_id, image_digest). -
ECR push webhook:
POST /api/v1/side-scans/webhooks/ecr— receives ECR push events, validatesx-ogum-tokenagainsttenant_config.scanner_token, createsscan_jobsrecord (type: ecr), enqueuesscan_container_image, returns202. -
Jobs API:
GET /api/v1/side-scans/jobs(list withstatus/job_typefilters, pagination),GET /api/v1/side-scans/jobs/{job_id}(detail with tenant isolation — wrong tenant → 404),POST /api/v1/side-scans/jobs/{job_id}/retry(re-enqueuesfailedjobs only — non-failed → 422). All endpoints enforceX-Tenant-IDisolation via AQLFILTER j.tenant_id == @tid. -
Image security badge:
GET /api/v1/side-scans/images/{digest}/security— CI/CD badge endpoint returningoverall_status: pass|failand CVE counts by severity (CRITICAL/HIGH/MEDIUM/LOW). Returnsfailwhen CRITICAL or HIGH findings exist. -
/side-scanningUI page: Next.js page with KPI cards (EC2 / Lambda / K8s / Registry job counts), status/type filter dropdowns, jobs table (type, resource, status badge with pulse animation for running, findings count, duration, started-at), and one-click Re-scan button for failed jobs (triggers retry mutation). Sidebar nav updated: "Side Scanning" now routes to/side-scanning(previouslydisabled badge="Soon"). -
sideScanApiadded tofrontend/src/lib/api.ts:listJobs,getJob,retryJob,imageSecurityStatus. -
Types:
SideScanJob,SideScanJobStatus,SideScanJobType,PagedSideScanJobs,ImageSecurityStatusadded tofrontend/src/lib/types.ts. -
5 unit tests (
TestRunTrivyImage): parse vulnerabilities, parse secrets (no Match field), parse misconfigurations, nonzero exit raises RuntimeError, empty output returns empty tuples. -
Integration tests (
test_side_scanning_registry.py): ECR webhook enqueues task + creates scan_jobs record (5 tests covering: valid push, invalid token 401, list jobs tenant isolation, status filter, job detail, wrong-tenant 404, not-found 404, retry creates new job + dispatches task, retry non-failed 422, badge pass/fail). -
Frontend test (
SideScanningPage.test.tsx): 5 unit tests — renders title, KPI cards, status filter, type filter, loading state.
Changed
-
delete_snapshot_safe()now logs but does not raise on failure (previously re-raised). Sprint 1 task cleanup blocks that wrapped it in try/except are unchanged and still safe. -
scan_ec2_instance(Sprint 1) no longer runs TruffleHog3 for secret scanning —secret_countis always 0. Secrets are now detected inscan_ec2_instance_v2via Trivy. -
Sprint 1 integration tests updated to remove
run_trufflehogmocks;test_scan_persists_secret_findingreplaced withtest_scan_v1_has_no_secret_scanning. -
IAM Deep Model + Graph UX (Epic 02 Sprint 6): New
app/services/graph/sub-package with three modules.iam_edges.pymaterialisesSTS_ASSUMEROLE_ALLOW(trust-policy-derived role assumption edges),ASSUMES(active assumption from storedassumed_role_arn), andATTACHED_POLICY(identity-to-policy links) from identity metadata already in ArangoDB — no new cloud API calls.privilege_escalation.pyimplements 10 IAM privilege escalation patterns (PRIVESC-01 through PRIVESC-10) covering direct/chain role assumption, dangerous permission combinations (CreatePolicyVersion, SetDefaultPolicyVersion, AttachRolePolicy, AttachUserPolicy, PutUserPolicy, CreateAccessKey, UpdateFunctionCode), and PassRole+RunInstances to admin instance profile.exposure.pyderivesexposed_internet: boolon resources/data_assets/network_endpoints from existingis_public/is_internet_facingfields. -
Graph API (
GET/PATCH/POST/DELETE /api/v1/graph/...): Crown Jewels endpoints (PATCH /resources/{id}/crown-jewel,GET /crown-jewels) to mark/list crown-jewel assets. AQL Console (POST /aql) executes tenant-scoped read-only queries — blocks INSERT/UPDATE/REMOVE/REPLACE/UPSERT, injects@tenant_idbind var when referenced, caps results at 200 rows. Saved Queries (GET/POST/DELETE /queries) persists favourite AQL queries per tenant (max 50). Shortest Path (GET /paths/{from}/{to}) uses AQLSHORTEST_PATHacross all IAM and graph edges. Exposure Summary (GET /exposure) returns internet-exposed resource counts per collection. -
Celery tasks for graph:
build_iam_graph_edges(tenant_id, provider_id)calls the IAM edge builder and exposure computation;refresh_active_sessions_from_cloudtrail(tenant_id)is a documented stub deferred to Epic 05 (Pulse). -
Schema additions:
saved_queriesvertex collection;STS_ASSUMEROLE_ALLOW,ASSUMES,ATTACHED_POLICY,HAS_ACTIVE_SESSIONedge collections;is_crown_jewelpersistent indexes onresourcesanddata_assets;saved_queriestenant index. -
Frontend Graph UX:
QueryConsolecomponent — AQL editor with template query picker, saved queries sidebar (create/delete), read-only result table with truncation warning and execution time.PathfindingPanelcomponent — input pair for ArangoDB node IDs with shortest-path visualisation showing entry→edge→target chain. Both modals wired intoAttackPathsPagevia "AQL Console" and "Pathfinding" header buttons.graphApiclient inapi.tsexposes all new graph endpoints. -
Frontend types:
SavedQuery,AqlResult,ShortestPathResult,ExposureSummaryinterfaces intypes.ts. -
conftest.py:
ARANGO_URLnow falls back tohttp://{ARANGO_HOST}:{ARANGO_PORT}instead oflocalhost:8529only, enabling integration tests to run inside Docker Compose without manual env override. -
13 unit tests: 8 tests for
_extract_principal_arns()covering single/multi principal, wildcard action, Deny exclusion, non-AssumeRole action, service principal, empty inputs, list of actions; 5 tests for escalation pattern metadata (count, uniqueness, required fields, coverage, lowercase keys). -
12 integration tests for Graph API: Crown Jewels CRUD (not-found, empty list, mark+list cycle); AQL console (valid read query, blocked write statements, invalid AQL 422, @tenant_id injection); Saved Queries (empty list, create+list+delete, write query rejected, nonexistent delete 404); Exposure Summary structure validation.
-
MITRE ATT&CK Intelligence Integration (Epic 02 Sprint 5): New
app/services/mitre_service.pywithMITRE_CHAINSmapping the four built-in Toxic Combination rules (TC-01 through TC-04) to MITRE technique sequences.build_mitre_chain_for_tc()returns independent copies of the chain (mutation-safe).get_techniques_for_path()queriesogum_adminfor full technique/tactic/APT-group details, returning gracefully when MITRE data hasn't been imported yet.enrich_finding_with_mitre()createsMAPPED_TOedges between findings and techniques in the tenant DB. -
MITRE collections in ArangoDB schema:
mitre_techniques,mitre_tactics, andmitre_groupsvertex collections added toogum_adminglobal database;APT_USESedge collection for group→technique relationships;MAPPED_TOandIMPLEMENTSedge collections in tenant DBs. -
GET /api/v1/attack-paths/{path_id}/mitreendpoint: returns techniques (with tactic IDs and platform lists), tactic names, APT groups, andmitre_chainbreadcrumb. Returns 404 on wrong-tenant access. Falls back gracefully to empty technique lists when MITRE bundle hasn't been imported. -
actively_exploitedfilter onGET /api/v1/attack-paths: boolean query parameter filters paths by active runtime exploitation status (set by Ogum.Pulse CDR events). -
mitre_ttpfilter onGET /api/v1/findings: filter findings by MITRE technique ID (checksmitre_ttpsarray). -
mitre_chain,mitre_ttps,actively_exploited,last_runtime_event_atfields on attack path documents: populated bybuild_attack_path_docs()for Toxic Combination rules. -
scripts/import_mitre_bundle.py: idempotent STIX 2.0 import script — downloads enterprise-attack.json from MITRE CTI GitHub, parses techniques/tactics/intrusion-sets/relationships, and upserts intoogum_admin. Supports--local-filefor offline use. Graceful error if download fails. -
make mitre-importtarget: convenience wrapper for the STIX import script. -
MITRE ATT&CK section in
AttackPathDetailPanel: shows technique chips (linked to attack.mitre.org), tactic breadcrumb chain for Toxic Combinations, APT group badges with country, and graceful fallback to raw T-code chips when admin MITRE data isn't imported yet. -
Frontend types:
MitreTechnique,MitreTactic,MitreGroup,MitreIntelligenceinterfaces added totypes.ts;AttackPathextended withmitre_ttps,mitre_chain,actively_exploited,last_runtime_event_at. -
attackPathsApi.getMitre(pathKey)method inapi.ts. -
17 unit tests for
mitre_service: cover all four chain constants,build_mitre_chain_for_tc()for known/unknown rules, mutation safety,get_techniques_for_path()with None admin DB, missing collection, chain preservation, and empty-field handling. -
6 integration tests for MITRE API: 404 on missing/wrong-tenant path, response structure, empty technique list graceful degradation,
actively_exploitedfilter true/false behavior. -
2 integration tests in
test_attack_path_service.py: toxic combination docs include MITRE chain/ttps; non-toxic paths have empty MITRE chain. -
EC2 Side-Scanning Pipeline (Epic 03 Sprint 1):
app/services/side_scanning/sub-package implementing agentless EC2 analysis via ephemeral EBS snapshots.snapshot_manager.pyhandles the full AWS lifecycle:create_scan_snapshot()tags snapshots withogum:scan=true,ogum:tenant_id, andogum:expires_at(+2h);wait_for_snapshot()polls with 10s intervals (max 15 min);create_volume_from_snapshot()creates a gp3 volume;mount_volume_ro()mounts read-only; cleanup (delete_volume_safe,delete_snapshot_safe) always runs infinally. Three analyser wrappers:trivy_analyzer.run_trivy_fs()parses JSON output extracting CVE ID, package, CVSS score;yara_analyzer.run_yara()returns rule+path matches (gracefully skipped if rules dir absent);secret_analyzer.run_trufflehog()returns metadata-only dicts (never logs actual secret values).cvss_to_severity()maps CVSS v3 scores to Ogum severity tiers (CRITICAL ≥9.0, HIGH ≥7.0, MEDIUM ≥4.0, LOW >0.0, INFO =0.0). -
scan_ec2_instanceCelery task (Epic 03 Sprint 1):app/workers/tasks/side_scanning.py. Runs Trivy, YARA, and TruffleHog in parallel viaThreadPoolExecutor. Normalises CVE findings (from Trivy), exposed secrets (CRITICAL severity, path metadata only), and malware detections (CRITICAL). Persists findings to ArangoDB withHAS_FINDINGedges. Creates and completes ascan_jobsrecord per run. Cleanup always runs infinallyregardless of analyser failures.OGUM_SCANNER_INSTANCE_IDenv var controls whether volume attachment is attempted (skipped in dev/test). -
cleanup_orphan_snapshotsCelery Beat task (Epic 03 Sprint 1): lists all snapshots taggedogum:scan=trueand deletes any withogum:expires_atin the past. Provides recovery from worker crashes that leave orphaned snapshots in customer accounts. -
FindingSource.SIDE_SCANNINGenum value: addedside_scanningtoapp/models/finding.pyFindingSourceenum for findings originating from agentless disk analysis. -
15 unit tests for side-scanning: 10 tests for
cvss_to_severity()covering all boundary values (CRITICAL/HIGH/MEDIUM/LOW/INFORMATIONAL thresholds, negative scores); 5 tests for_extract_cvss()covering NVD V3 extraction, vendor scores, missing CVSS block, missing V3 field, and malformed data. -
8 integration tests for
scan_ec2_instanceandcleanup_orphan_snapshots: verify snapshot created and deleted (moto + real ArangoDB), CVE/secret/malware findings persisted with correct severity, cleanup runs after analyser exception,scan_jobsrecord created and completed, and orphan cleanup correctly deletes expired / skips future snapshots. -
CIEM Static Analysis (Epic 02 Sprint 4):
app/services/ciem_service.py— static permission analysis engine.analyze_dangerous_permissions()inspects an identity'sgranted_actionsandpoliciesagainst a curated list of 18 dangerous IAM actions (iam:PassRole,iam:CreatePolicyVersion,iam:AttachRolePolicy,sts:AssumeRole,s3:*,iam:*,ec2:*,lambda:UpdateFunctionCode, etc.) and wildcard expansion.find_assume_role_chains()traversesASSUMES_ROLEedges up to 4 hops to detect privilege escalation paths to high-privilege identities.list_identities_with_ciem()returns paginated identities withdangerous_permissions_countandescalation_paths_countfields, sorted by risk score. -
Identities API (Epic 02 Sprint 4):
GET /api/v1/identities— paginated list of IAM identities with CIEM counters, filterable byproviderandonly_dangerous=true.GET /api/v1/identities/{key}/permissions— full CIEM analysis for a single identity: dangerous permissions list, AssumeRole escalation chains, policy names, and risk score. -
Risk Score column in Inventory table (Epic 02 Sprint 4):
RiskBadgecomponent displays numeric score (0–100) with color-coded tier (CRITICAL red, HIGH orange, MEDIUM yellow, LOW blue, NONE slate). InventoryDataTableand FindingsFindingsTablenow renderRiskBadgefor each resource/finding. -
Attack Paths card on dashboard (Epic 02 Sprint 4): security overview page now shows a clickable Attack Paths card with total active paths count, critical+high count, and new-in-24h badge. Also added Attack Paths to the quick navigation links.
-
13 unit tests for
ciem_service: cover dangerous permission detection, wildcard expansion (iam:CreateUser→iam:*,s3:DeleteBucket→s3:*), AdministratorAccess policy detection, duplicate prevention, missing field handling, and constant structure validation. -
14 integration tests for
GET /api/v1/identitiesandGET /api/v1/identities/{key}/permissions: cover empty list, sorting by risk_score, provider filter,only_dangerousfilter, pagination, 404 for missing identity, dangerous permission detection, and response field completeness. -
10 component tests for
RiskBadge: cover all 5 tiers (CRITICAL/HIGH/MEDIUM/LOW/NONE), null/undefined score rendering, title attribute, and tier CSS class verification. -
Attack Paths canvas (Epic 02 Sprint 3): interactive React Flow canvas replacing the flat table layout.
AttackPathListpanel (260px) groups paths by severity with score, entry→target route, hop count, and toxic badge — clicking loads the path in the canvas.AttackPathCanvasrenders custom node types (EntryPointNodered,TargetNodeyellow,IdentityNodepurple,ResourceNodedefault) with dagre left-to-right auto-layout, animated smoothstep edges with inferred relation labels, zoom/pan controls, and minimap. Bottom info bar shows selected path summary with entry point, target, risk score, hops, and a disabled "Remediate with AI" CTA.NodeDetailPanelslide-over fetches full resource detail from/api/v1/inventory/{key}on node click.graph-layout.tsutility wraps dagre layout and node-type inference. 15 component tests forAttackPathListandAttackPathCanvas.structuredClonepolyfill added to jest setup (dagre dependency).@xyflow/reactv12 and@dagrejs/dagreinstalled. -
Attack Paths API (Epic 02 Sprint 2): three new endpoints under
/api/v1/attack-paths.GET /api/v1/attack-pathsreturns a paginated, risk-score-sorted list of attack paths with filters forseverity,is_toxic_combination, andprovider(keyset cursor pagination onrisk_score DESC).GET /api/v1/attack-paths/statsreturns aggregate counts by severity plus anew_24hcounter.GET /api/v1/attack-paths/{path_id}returns the full path document, all vertex node documents, and associated findings. Tenant isolation enforced at database level (separate ArangoDB database per tenant) plustenant_idfilter on every query. -
Attack Paths frontend (Epic 02 Sprint 2): web console page at
/attack-paths. Stats bar with per-severity counts and new-in-24h indicator. Filter bar for severity, provider, and toxic-combination toggle. Paginated table sorted by risk score with entry-point → target path display, risk score progress bar, hops count, and toxic combination badge. Slide-over detail panel with entry/target cards, full vertex node list, and associated findings. Sidebar nav item enabled. New types (AttackPath,AttackPathStats,AttackPathDetail,AttackPathFilters) andattackPathsApiclient added. 20 component tests. -
Post-scan inventory extraction (all providers): CSPM scans now automatically populate the inventory after each scan run — no separate discovery task required.
prowler_inventory.extract_inventory_from_findings()deduplicates resources byresource_uid, routes to the correct collection (resources,identities,data_assets), and upserts withlast_scanned_atand normalizedresource_type. Supports all four providers: AWS, Azure, GCP, and Kubernetes. -
ProwlerService.run_azure_scan,run_gcp_scan,run_kubernetes_scan: new scan methods wrappingAzureProvider,GcpProvider, andKubernetesProvider.run_cspm_scantask now routes to the correct method by provider (was AWS-only). -
Provider-aware default frameworks in
GET /api/v1/scans: when no frameworks are specified, the scan API selects sane defaults per provider (CIS-AWS-2.0 + PCI + SOC2for AWS,CIS-AZURE-2.0for Azure,CIS-GCP-2.0for GCP,CIS-K8S-1.12for Kubernetes). -
ScanResultdataclass:ProwlerServicescan methods now returnScanResult(findings, raw_outputs)so the task can use rawOutputFindingobjects for inventory extraction without a second pass. -
43 unit tests for
prowler_inventory: cover type normalization (AWS/Azure/GCP/K8s), collection routing (IAM → identities, S3 → data_assets, EC2 → resources), tag extraction, public IP detection, ArangoDB key generation, deduplication, and edge cases. -
Risk Score engine (Epic 02 Sprint 1):
app/services/risk_score.py— contextual risk scoring (0–100) for every resource in the graph. Score formula:severity_base × exposure_factor × blast_factor, capped at 100. Severity base is the weighted sum of FAIL findings (CRITICAL=10, HIGH=7, MEDIUM=4, LOW=1). Exposure factor doubles score for public/internet-facing resources. Blast factor scales with the count of sensitive data assets reachable within 3 hops. Resources in confirmed attack paths have a minimum score of 40. Path risk score uses max-node score amplified by a depth factor (shorter path = more dangerous). -
Attack Path detection (Epic 02 Sprint 1):
app/services/attack_path_service.py— AQL graph traversal queries that discover paths from internet-facing resources to sensitive data assets (up to 4 hops), IAM privilege escalation chains, and toxic combinations. Four toxic combination rules: TC-02 (public bucket with credentials), TC-03 (overpermissioned identity → production database), TC-04 (K8s pod with host network). Results are persisted to the newattack_pathscollection withis_toxic_combinationflag, risk score, severity, and path metadata. Resources participating in any attack path are markedin_attack_path=true. -
Post-scan graph enrichment tasks (Epic 02 Sprint 1):
app/workers/tasks/attack_paths.py— two Celery tasks (recalculate_risk_scores,detect_attack_paths) enqueued automatically after each CSPM scan completes.recalculate_risk_scoresbulk-updates therisk_scorefield on all resources, identities, and data assets.detect_attack_pathsruns the full detection pipeline and persists results. -
attack_pathsArangoDB collection: added to the tenant schema with persistent indexes ontenant_id + risk_score,tenant_id + is_toxic_combination, andtenant_id + severityfor efficient filtering. Addedrisk_scoreindex onresources,identities, anddata_assetsfor sort performance. -
Dashboard home (F0.3): replaced
/redirect with a real security overview page showing ThreatScore, finding counts by severity (CRITICAL/HIGH/MEDIUM/LOW — each links to/findings?severity=X), last 5 scan jobs with status icons and relative timestamps, and quick-navigation links to all main sections. -
GET /api/v1/findings/stats: new endpoint returning aggregate countsby_severityandby_statusplustotal— a single AQL query, scoped by tenant, used by the dashboard. -
findingsApi.stats(): new client method infrontend/src/lib/api.ts;FindingsStatstype added totypes.ts. -
Dev seed endpoint (F0.1):
POST /api/v1/dev/seedinserts 19 realistic demo findings (CRITICAL/HIGH/MEDIUM/LOW, mix of FAIL/PASS/MUTED) into ArangoDB without requiring real cloud credentials. Only available whenDEV_MODE=true. Endpoint is idempotent. ComplementaryDELETE /api/v1/dev/seedclears findings and scan_jobs. Standalone scriptbackend/scripts/seed_demo.pyandmake seed/make seed-clearMakefile targets for local use.DEV_MODEsetting added toconfig.pyand.env.example. -
Discovery job tracking (US-10.09): all four discovery tasks (
discover_aws,discover_azure,
discover_gcp,discover_k8s) now write to thescan_jobsArangoDB collection so runs appear in
/admin/jobs. A new_job_trackinghelper module (start_discovery_job,complete_discovery_job,
fail_discovery_job) handles the lifecycle with fire-and-forget semantics — tracking failures never
abort discovery. Integration tests cover all state transitions and the missing-collection edge case. -
Admin Jobs API (Epic 10 Sprint 1): cross-tenant job inspection and control endpoints under
/api/v1/admin/.GET /admin/jobslists scan jobs across all tenant databases (or scoped to one
via?tenant_id=), with optionalstatusfilter and keyset cursor pagination.GET /admin/jobs/{id}returns full job detail including findings counts and logs.POST /admin/jobs/{id}/retryre-enqueues a failed job and records the action inadmin_audit_log.
DELETE /admin/jobs/{id}calls Celery revoke.GET /admin/workersinspects live worker state
via Celery inspect.GET /admin/queue-depthreturns pending message counts for all known queues
via RedisLLEN.admin_audit_logcollection added to a dedicatedogum_admindatabase.
All routes markedTODO(epic-06)for role enforcement once Auth/RBAC is implemented. -
App shell navigation (Epic 11 Sprint 1): global layout with fixed sidebar and header wrapping
all pages.AppShellcomposesSidebar+Headerand is wired intoapp/layout.tsx.
Sidebarorganises all platform modules into labelled sections with active links for implemented
pages (/,/inventory,/findings,/compliance,/providers) and disabled<span>elements
with a "Soon" badge for modules not yet built (Attack Paths, Side Scanning, Pulse, CDR, AI
Remediation, Integrations, Agent, Admin).Headershows a dynamic breadcrumb based on the
current route.NavItemhighlights the active route with an orange left border. -
Makefile (
Makefile): covers Docker Compose lifecycle (up,down,up-infra,up-backend,
restart), code quality (lint,format,typecheck,check,fe-lint,fe-typecheck),
tests by layer (test-unit,test-integration,test-security,fe-test,agent-test),
dependency management (install,update-deps), container shells, log tailing, and a
healthtarget that checks all services. Runmake helpto list all targets. -
Terraform test fixtures (
infra/terraform/test-fixtures/): deploys intentionally
misconfigured AWS resources (EC2, S3, IAM, Security Groups, VPC, CloudTrail) for validating
Ogum.Inventory discovery and Ogum.Static CSPM findings. Includes compliant baselines alongside
misconfigured scenarios for diff testing.terraform output expected_findingslists the exact
CSPM findings that should appear after scanning the fixture account. -
IaC scanning (Ogum.Static Sprint 4):
POST /api/v1/scans/iactriggers a Celery task that
shallow-clones a Git repository, runs Checkov across Terraform, CloudFormation, and Kubernetes
manifests, and persists findings withsource: iac. Repository tokens are injected into the
HTTPS URL at clone time and never stored or logged. Cloned repos are cleaned up in afinally
block after each scan. -
CheckovService(app/services/checkov_service.py): wraps Checkov's programmatic API
(TFRunner,CFNRunner,K8sRunner), normalizes check results toFindingobjects with
OCSF-aligned severity levels, and maps resource IDs to cloud provider (aws,azure,gcp,
iac). Raises a graceful[]result onImportError(checkov optional dependency). -
GET /api/v1/findings/export: streams all findings matching the active filter set as CSV
or JSON (OCSF-aligned). Streaming uses a generator with keyset cursor pagination internally —
safe for exports of 100k+ findings without loading all rows into memory. The/exportroute is
registered before/{finding_key}to prevent FastAPI treating the literal "export" as a path
parameter. -
Export button (
components/findings/ExportButton.tsx): dropdown with CSV and JSON (OCSF)
options; triggers a browserBlobdownload with the current filter set applied. Appears in the
Findings page header next to the severity counter chips. -
docs/scanning.md: public documentation covering CSPM scan trigger and polling, supported
compliance frameworks, IaC scan parameters (including token security notes), export formats and
filter options, and CSV field reference. -
Findings UI (Ogum.Static Sprint 3):
/findingspage with a full-featuredFindingsTable
component — severity badge column, MUTED/ACCEPTED/PASS/FAIL status, provider badge,
keyset cursor pagination (Previous/Next).FindingFiltersbar with debounced search, six
dropdown filters (severity, status, provider, framework, source), and active-filter chips
with individual clear and "Clear all".FindingDetailPanelslide-in (440px) showing resource
metadata, risk description, remediation steps, CLI command with copy-to-clipboard, framework
badges, mute modal with required reason field, and a placeholder "Generate PR ✨" (Ogum.AI,
Epic 05). Panel closes on Escape key or backdrop click. -
Compliance dashboard (
/compliance): framework score sidebar with pass/fail counts and
progress bar, main panel with detailed scores, ThreatScore gauge (0–100 weighted by severity
of open FAIL findings), top-10 failing controls ordered by occurrence count. -
GET /api/v1/compliance/summary: AQL-powered aggregation endpoint returning per-framework
pass/fail/total/score, global ThreatScore, and top-10 failing checks by count — all scoped to
the requesting tenant. -
SeverityBadgecomponent (components/ui/SeverityBadge.tsx): dedicated wrapper around
Badgethat mapsSeverityLevelenum to the correct variant — reused across findings table,
detail panel, and compliance page. -
Frontend types and API client extended with
Finding,FindingDetail,PagedFindings,
FindingsFilter,ComplianceSummary,FrameworkScoretypes;findingsApi(list, get, mute,
accept) andcomplianceApi(summary) added tolib/api.ts. -
Findings API (Ogum.Static Sprint 2):
GET /api/v1/findingswith server-side filtering
(provider, severity, status, framework, region, account_id, resource_type, source, full-textq)
and keyset cursor pagination (stable across large datasets).GET /api/v1/findings/{key}returns
the full finding enriched with linked resource metadata, attack path placeholders (Epic 02), and
the best available CLI remediation command.PATCH /api/v1/findings/{key}mutates status to
MUTED(requiresreason) orACCEPTED; every change is written to theaudit_logcollection. -
audit_logvertex collection added to tenant schema with indexes ontenant_idand
(tenant_id, finding_key)— stores every finding status mutation with actor, reason, and timestamp. -
app/services/cli_command.py: provider-aware CLI command builder. Priority: stored
remediation_codefrom Prowler → known resource-type template → provider fallback.
Supports AWS, Azure, GCP, and Kubernetes. ARN-style resource IDs are simplified to the last
segment for ergonomic CLI output. -
CSPM scan engine (Ogum.Static Sprint 1): Prowler v5 programmatic integration via
ProwlerService.POST /api/v1/scanstriggers a Celery task (run_cspm_scan) that
executes Prowler checks, normalizes results to OCSF-alignedFindingobjects, and persists
them in ArangoDB with idempotent upserts (check_id + resource_id + tenant_idas key).
GET /api/v1/scans/{job_id}andGET /api/v1/scansexpose job status and history. -
findingsandscan_jobsvertex collections added to tenant schema (init_tenant_schema).
HAS_FINDINGedge collection linksresources → findings.
Persistent indexes ontenant_id,severity,status,provider, andcheck_id. -
FindingandScanJobPydantic models (app/models/finding.py): OCSF-aligned severity
levels (CRITICAL/HIGH/MEDIUM/LOW/INFORMATIONAL), finding status (FAIL/PASS/MUTED/ACCEPTED),
framework mapping list, remediation text and code, scan job lifecycle tracking. -
Provider delete purges all associated resources:
DELETE /api/v1/providers/{id}now
hard-deletes all documents inresources,identities,data_assets, and
network_endpointsscoped to the provider's account, then sweeps orphaned edges from all
eight edge collections. The response includes apurgedmap with per-collection delete counts. -
Scan all regions by default:
regions: [](empty list) now means "scan all enabled regions"
across all providers. For AWS, the task callsec2:DescribeRegionsto resolve the full list
at runtime. Azure, GCP, and Kubernetes already scanned all resources regardless of region.
The wizard and edit modal show an empty field by default with a hint explaining the behavior;
users can still restrict to specific regions by listing them comma-separated. -
Edit provider:
PATCH /api/v1/providers/{id}now acceptsrole_arn,azure_tenant_id,
andazure_client_idupdates (non-secret, safe to persist);EditProviderModalcomponent
pre-fills current config values and supports optionally re-triggering discovery with new
ephemeral credentials (API keys, client secret, SA JSON, kubeconfig) that are forwarded to
the worker and never stored. -
DiscoverRequesttype exported from@/lib/types;providersApi.triggerDiscoverynow
accepts an optionalDiscoverRequestbody for re-trigger with static credentials. -
Edit button (Pencil icon) added to
ProvidersTableandProviderCard— opens
EditProviderModalwith current provider values pre-filled. -
Multi-method credential support for all providers:
- AWS: IAM Role (
role_arnviasts:AssumeRole) and static API keys
(aws_access_key_id/aws_secret_access_key, dev only) are both supported in the UI
and API; ambient worker credentials remain the fallback when neither is provided - Azure: Service Principal (
azure_tenant_id+azure_client_id+azure_client_secret)
andDefaultAzureCredential(Managed Identity / ADC) are both selectable in the wizard - GCP: Service Account JSON and Application Default Credentials (ADC) are both supported
- Kubernetes: external kubeconfig (JSON) and in-cluster ServiceAccount are both selectable
- AWS: IAM Role (
-
credential_typefield onProviderConfig: records which credential method was used
at registration time (role,static,ambient,service_principal,managed_identity,
service_account,adc,kubeconfig,incluster) — no secrets stored -
azure_tenant_id/azure_client_idstored onProviderConfig(non-secret, safe to
persist) so re-triggered discovery can restore Azure credential context -
provider_keythreading indiscover_azureanddiscover_gcp: both tasks now accept
and propagate theprovider_keyparameter so ArangoDB status updates work correctly -
Status updates for Azure and GCP:
_set_provider_status(db, provider_key, "active"|"error")
now called on task success and failure, consistent with AWS behavior -
MethodSelectorUI component inConnectWizard: segmented control for choosing
credential method per provider — no separate page or modal required -
Provider Management API (
/api/v1/providers): new endpointsGET /{id},PATCH /{id},
andPOST /{id}/discoverfor fetching, updating, and re-triggering discovery on a specific provider -
Provider
statusfield:ProviderConfignow trackspending | active | error | disabledstate;
status transitions automatically on discovery dispatch and enable/disable toggle -
ProviderUpdateRequestmodel: supports partial PATCH ofdisplay_name,regions, andenabled -
DiscoverResponsemodel: typed response for the re-trigger discovery endpoint -
ProviderTypeenum: provider field is now validated as a literal type (aws | azure | gcp | k8s) -
/providerspage (frontend): connected accounts management page with table showing provider
type, display name, account identifier, regions, status badge, last discovery time, and per-row
actions (re-discover, enable/disable, delete) -
ProvidersTablecomponent: reusable table and card components for provider management,
with busy state per row and correct disabled states for actions -
providersApiclient additions:get(),update(), andtriggerDiscovery()methods -
docs/connecting-accounts.md: comprehensive guide covering UI wizard, API reference,
credential model, discovery schedule, status reference, and troubleshooting -
Ogum.Inventory Sprint 5 — Onboarding, Export, and Tenant Isolation
tenant_configdocument collection added toinit_tenant_schema()— stores connected
provider metadata per tenant (no credentials — those come from env/Vault at task time)app/models/provider.py— Pydantic v2 schemas:ProviderConfig,ProviderRegisterRequest,
ProviderRegisterResponseapp/services/provider_service.py— CRUD ontenant_configcollection:
register_provider,list_providers,delete_provider,update_provider_last_discoveryPOST /api/v1/providers— register a cloud provider connection; dispatches discovery
task (aws/azure/gcp/k8s) and records job_id; 201 withprovider_idanddiscovery_job_idGET /api/v1/providers— list all connected providers for the tenantDELETE /api/v1/providers/{provider_id}— remove a provider config; 404 if not foundGET /api/v1/inventory/export?format=csv— streaming CSV export of full tenant inventory
(capped at 50k rows);Content-Dispositionattachment header with timestamped filenameGET /api/v1/inventory/export?format=json— streaming OCSF-inspired JSON export with
ocsf_version,metadata(tenant_id, exported_at, total_resources, product), and
resourcesarray; 422 for unsupported format values- Export endpoint registered before
/{resource_key}to prevent route shadowing - Frontend:
ConnectWizardcomponent (components/providers/ConnectWizard.tsx) — 4-step
modal (select provider → configure → connecting → done); supports AWS, Azure, GCP, K8s;
provider-specific fields shown conditionally; error display on failed registration - Frontend:
/providers/newpage — renders ConnectWizard; redirects to /inventory on success - Frontend: Inventory page — pristine empty state (zero resources + no active filters)
now shows "No cloud accounts connected" callout with a "Connect Account" CTA link - Frontend:
providersApiandinventoryApi.exportCsv/exportJsonadded tosrc/lib/api.ts - Frontend:
ProviderConfig,ProviderRegisterRequest,ProviderRegisterResponse
interfaces added tosrc/lib/types.ts - 15 backend integration tests — all passing:
test_providers_api.py(15): register (AWS, Azure, GCP), missing header → 422,
idempotency, list empty/populated, delete existing/nonexistent, CSV export content-type
and header row, JSON export OCSF structure, JSON metadata fields, invalid format → 422,
CSV with seeded resources contains data rows
- 2 new tenant isolation tests (
TestProviderConfigIsolation): provider config in
Tenant A not visible in Tenant B; AQL on Tenant B returns zero Tenant A resources - 8 frontend component tests for
ConnectWizard— provider selection, configure step per
provider, back navigation, cancel handler; all passing docs/getting-started.md— new "Using the Web Console" and "Using the API" sections
withcurlexamples for connect, list, export CSV, and export JSON
-
Ogum.Inventory Sprint 4 — Multi-Provider Discovery + Celery Beat Scheduling
AzureResource(ResourceBase)model — addssubscription_idfield;arango_key()
uses{sub_prefix}_{resource_group}_{name}for unique, length-safe ArangoDB keysGCPResource(ResourceBase)model — addsproject_idfield; inheritsarango_key()
fromResourceBaseusing{provider}_{type}_{compact_id}K8sResource(ResourceBase)model — addscluster_nameandnamespacefields;
overridesarango_key()ask8s_{cluster}_{type}[_{namespace}]_{uid}app/workers/tasks/scheduling.py—acquire_lock(redis, tenant_id, provider)and
release_lock(redis, tenant_id, provider)helpers using RedisSET NX EXpattern
(TTL = 7h);trigger_all_discoveriesCelery task routes Beat events to provider tasksapp/workers/tasks/azure_discovery.py—discover_azuretask: VMs, VNets, NSGs,
Storage Accounts, AKS clusters, Key Vaults (metadata only); soft-delete of absent
resources; distributed lock guards against concurrent runsapp/workers/tasks/gcp_discovery.py—discover_gcptask: Compute instances
(aggregated across all zones), GCS buckets, GKE clusters; distributed lockapp/workers/tasks/k8s_discovery.py—discover_k8stask: Pods, Deployments,
Services, Nodes, Namespaces; supports kubeconfig dict or in-cluster config;
distributed lock; K8s UIDs used asresource_idfor stable keyscelery_app.py— updatedincludelist to register all new task modules;
defaultbeat_scheduleadded (trigger_all_discoveriesevery 6h for dev)docker-compose.yml—celery-beatservice added (single instance, Redis-backed)pyproject.toml— addedazure-mgmt-compute,azure-mgmt-network,
azure-mgmt-storage,azure-mgmt-containerservice,azure-mgmt-keyvault,
google-cloud-compute,google-cloud-storage,google-cloud-container,
kubernetesas explicit runtime dependencies- 19 integration tests — all passing:
test_scheduling.py(7): distributed lock acquisition, release, tenant/provider
isolation, trigger routing, unknown-provider error handlingtest_azure_discovery.py(4): VMs persisted, idempotency, soft-delete, lock skiptest_gcp_discovery.py(4): compute instances persisted, idempotency, soft-delete, lock skiptest_k8s_discovery.py(4): pods persisted, idempotency, soft-delete, lock skip
-
Ogum.Inventory Sprint 3 — REST API + Inventory UI
GET /api/v1/inventory— list resources with filters (provider,resource_type,
account_id,region,status,search),limit/offsetpagination, and
sort_by/sort_dirordering; returns standard{data, meta, error}envelopeGET /api/v1/inventory/stats— aggregate counts by provider, resource type, status;
also returnsidentity_count,data_asset_count, andlast_discovery_atGET /api/v1/inventory/{resource_key}— full resource detail including all
relationship edges (inbound + outbound across all 8 edge collections)POST /api/v1/inventory/discover— enqueuesdiscover_awsCelery task and returns
job_id(Celery task ID); returns 422 for providers not yet implementedapp/services/inventory_service.py— ArangoDB query service; AQL built with
bind_varsonly (no string interpolation with user input); sort field validated
against allowlist before useapp/models/api_responses.py— Pydantic v2 response models:ApiResponse[T],
Meta,ResourceSummary,ResourceDetail,EdgeSummary,InventoryStats,
DiscoverJobResponseX-Tenant-IDheader (dev mode) — replaces JWT tenant extraction until Sprint 7;
clearly marked as temporary in code comments- Frontend: Next.js 15 App Router scaffold —
layout.tsx,page.tsx(→ /inventory),
globals.css,tsconfig.json,next.config.ts,tailwind.config.ts - Frontend:
InventoryPage(src/app/inventory/page.tsx) — full inventory UI with
React Query data fetching, provider tabs, filters, paginated DataTable, and DetailPanel - Frontend:
DataTablecomponent — sortable table with Provider badge, name/ID,
type, account, region, status badge, pagination controls, and skeleton loading state - Frontend:
ProviderTabscomponent — All/AWS/Azure/GCP/K8s tabs with resource counts - Frontend:
Filterscomponent — search input (name/ARN/ID), resource type dropdown,
region dropdown - Frontend:
DetailPanelcomponent — 420px slide-in panel with Metadata section,
Tags, Relationships (edges with direction arrows), Findings placeholder, and
"Open in console" deep-link for AWS resources - Frontend:
Badgecomponent (components/ui/Badge.tsx) — provider, status, and
severity variants with dark-theme color coding (orange accent, slate palette) - Frontend:
Skeletoncomponent (components/ui/Skeleton.tsx) — loading placeholder - Frontend:
src/lib/types.ts— TypeScript interfaces for all API response shapes - Frontend:
src/lib/api.ts— Axios-based API client withX-Tenant-IDinterceptor - Jest testing setup —
jest.config.js,jest.setup.ts,@testing-library/react,
@testing-library/jest-dom,@testing-library/user-eventadded to devDependencies - 16 backend API integration tests (100% passing) — happy path, filters, pagination,
search, 422 validation, 404 for missing resource, stats aggregation, discover dispatch - 32 frontend component tests — Badge (11 tests), DataTable (9 tests), DetailPanel
(12 tests); all passing with@testing-library/react
-
Ogum.Inventory Sprint 2 — Expanded AWS Discovery (
discover_awstask)- Full coverage: VPC, Subnet, Internet Gateway, Elastic IP, Security Group, RDS, Lambda,
EKS, EKS, ECR, KMS (customer-managed keys only), Secrets Manager (metadata only),
CloudTrail, and IAM Groups IdentityType.IAM_GROUPenum value added to inventory model- Three new edge collections:
BELONGS_TO,ATTACHED_TO,MEMBER_OF _upsert_edgehelper: idempotent AQL UPSERT keyed on(_from, _to)_create_resource_edgesorchestrator: builds BELONGS_TO (EC2→VPC, Subnet→VPC),
ATTACHED_TO (SG→EC2, SG→Lambda), ROUTES_TRAFFIC (IGW→VPC),
ASSUMES_ROLE (EC2/Lambda→IAM Role), and MEMBER_OF (IAM User→IAM Group) edgesdiscover_awsCelery task: full discovery with edge creation, replacesdiscover_aws_basic
for production use;discover_aws_basicretained for backwards compatibilitypythonpath = ["."]added topyproject.tomlpytest config (no manualPYTHONPATHexport needed)
- Full coverage: VPC, Subnet, Internet Gateway, Elastic IP, Security Group, RDS, Lambda,
-
Integration tests — Sprint 2 (17 new tests, 50 total passing)
TestAWSExpandedDiscovery: VPC discovery, open-ingress SG markedis_public=True,
RDS instance discovery, Lambda with execution role, ECR repository, soft-delete of removed VPCTestRelationshipEdgeCreation: EC2→VPC (BELONGS_TO), SG→EC2 (ATTACHED_TO),
IGW→VPC (ROUTES_TRAFFIC), Lambda→Role (ASSUMES_ROLE), 2-hop AQL traversal IGW→VPC→EC2
Changed
- CI — venv cache optimization: all backend CI jobs (lint, unit tests, integration tests, security audit) now use
poetry.lockin the cache key (not justpyproject.toml), addcache-dependency-path: backend/pyproject.tomlto thesetup-pythonaction, and skippoetry installentirely when the virtualenv cache is hit. The integration and unit test jobs addid: venv-cache+if: steps.venv-cache.outputs.cache-hit != 'true'guards. - CI — ArangoDB readiness polling optimized: the
backend-integrationjob now polls ArangoDB with 3s intervals (was 5s), breaking immediately when ready instead of always completing 30 iterations. Redis uses a Docker--health-cmd "redis-cli ping"health check. Note: ArangoDB cannot use a Docker health check becausecurlis not available in thearangodb:3.12image — the runner-side polling loop is the correct approach.
Fixed
- Provider cascade delete: deleting a provider now removes all associated data in the correct order — findings and scan_jobs (by
provider_id), then graph vertices (resources, identities, data_assets, network_endpoints scoped by provider+account), then orphaned edges, and finally stale attack_paths whose entry_point or target no longer exists in the graph. Previously, findings, scan_jobs, and attack_paths were left as orphaned records after provider removal. - Prowler metadata normalization:
result.metadatainOutputFindingisCheckMetadatadirectly — the previous code was accessingmetadata.CheckMetadata(one level too deep), causing allcheck_id,title,description,severity, andresource_typeto resolve as"unknown"on every finding. - Finding
_keysanitization:arango_key()now usessha256(check_id|resource_id|tenant_id)instead of string substitution — prowler resource names/UIDs can contain dots, parens,@, and other characters that ArangoDB rejects as_keyvalues, causingERR 1221on upsert. - Prowler v5 API alignment: rewrote
ProwlerService.run_aws_scanto match the actual prowler-core v5 API —AwsProvidernow takes flat keyword args (noaudit_configdict), and scanning uses the high-levelScanclass withcompliances=[...]instead of manual check iteration. Framework IDs are mapped to prowler's file slugs (CIS-AWS-2.0→cis_2.0_aws, etc.)._normalizeconvertsOutputFindingpydantic objects to Ogum'sFindingmodel via field introspection. - CSPM scan import path: corrected
prowler.providers.aws.provider→prowler.providers.aws.aws_provider— the module was renamed in prowler v5 and the worker was failing withModuleNotFoundErroron every scan trigger. - Re-trigger discovery with static credentials:
POST /{id}/discovernow accepts an optionalDiscoverRequestbody so providers registered withcredential_type: static,service_principal,service_account, orkubeconfigcan re-provide their ephemeral credentials without re-registering the provider. discover_awsdiagnostic log: the task now logshas_role_arnandhas_static_keysat INFO level on start, making it easy to verify whether credentials reached the Celery worker.- AWS discovery credential guard:
NoCredentialsErrorwas not caught whenaccount_idwas already provided at registration — thests:GetCallerIdentitycall was skipped, so missing credentials were only detected mid-discovery inside_list_vpcs. Fixed by always calling STS upfront to validate credentials regardless of whetheraccount_idis known. docker/backend.Dockerfile— base image updated frompython:3.11-slimtopython:3.13-slimto match the Python 3.13 requirement inpyproject.toml;docker compose upwas failing with "currently activated Python version 3.11.15 is not supported by the project"backend/pyproject.toml— Python constraint changed from^3.13(>=3.13,<4.0) to>=3.13,<3.14;prowler ^5.31.0requiresPython <3.14and Poetry was unable to resolve dependencies when the upper bound was open-ended at<4.0backend/pyproject.toml— removed all cloud provider SDK direct declarations (azure-identity,azure-mgmt-security,azure-mgmt-compute,azure-mgmt-network,azure-mgmt-storage,azure-mgmt-containerservice,azure-mgmt-keyvault,google-cloud-securitycenter,google-cloud-compute,google-cloud-storage,google-cloud-container,kubernetes);prowler ^5.31.0pins specific versions of all these packages and any independent range declaration causes dependency resolution failure. All these SDKs remain available as prowler transitive dependencies; onlyboto3is kept as a direct dependency since it is used in discovery tasks that run independently of prowlerbackend/pyproject.toml— removedtruffleHog3 ^3.0.0; all 3.x versions pinattrs==20.3.0which is incompatible withprowler's requiredjsonschema==4.23.0(which needsattrs>=22.2.0);truffleHog3was planned for Epic 03 (Side-Scanning) and is deferred until a compatible secrets-scanner alternative is chosenbackend/pyproject.tomlanddocker/backend.Dockerfile— downgraded target runtime from Python 3.13 to Python 3.12; Python 3.13 has no pre-built wheels for severalprowlertransitive dependencies (notablyalibabacloud-tea), which forces source compilation; during source compilation, Poetry's mid-install downgrade ofpackaging(26.2 → 23.2) leavespackaging/tags.pytemporarily absent, breaking the build isolation subprocess; Python 3.12 has full wheel coverage for all prowler dependencies, eliminating source builds and the packaging race condition entirely.github/workflows/ci.yml— updated allpython-versionreferences from3.13to3.12to match the runtime constraint inpyproject.tomlandbackend.Dockerfile.github/workflows/ci.yml— addeddocker-buildjob that runsdocker compose build --no-cacheon every push and PR; validates that the Docker build succeeds before any other job runs; preventsdocker compose upregressions from reachingmain