Repository navigation
Releases
v0.7.1 - Marketplace CI availability
A security release for the publishing pipeline and the dependency floors, and cloudg in CI: a severity gate, SARIF output and a GitHub Action. Upgrade with the extras you use, for example pip install -U "cloudg[aws,gcp]==0.7.1", so pip lifts the dependencies below as well.
Security:
- The release workflow publishes to PyPI with
pypa/gh-action-pypi-publishv1.14.3. It was pinned to v1.14.2, whose bundled trust store still held the compromised emSign root CAs (GHSA-5xjx-3w67-w9x9). The action uses that trust store for Trusted Publishing and the PEP 740 attestations. This is release tooling and is not part of the package. - The extras now require versions of their transitive dependencies with the published advisories fixed.
aws: aiohttp 3.14.4 (through aioboto3).azure: certifi 2026.10.18, which drops the emSign root CAs (PYSEC-2026-4320).gcp: certifi 2026.10.18, opentelemetry-api 1.45.1 (CVE-2026-114603) and pyasn1 0.6.5, which fixes four denial-of-service bugs in the BER/DER decoder (CVE-2026-119194 to CVE-2026-119197).mcp: opentelemetry-api 1.45.1.dev: werkzeug 3.1.9 (through moto, PYSEC-2026-4312). A fresh install already resolved to these versions; the floors stop an upgrade from keeping an older copy that is already installed.
Added:
cloudg gatefails a CI job on findings at or above a severity (--fail-on,highby default). Suppressed findings never count,--maxtolerates a number of blocking findings, and--min-assetsfails a live run that collected too little. It exits 4 when the gate fails and 1 when the input is missing or unreadable.--sarifwrites the findings as SARIF 2.1.0 for GitHub code scanning,--summaryappends a Markdown summary to a file such as$GITHUB_STEP_SUMMARY, and--jsonprints the verdict on stdout. From Python, the same gate iscloudg.gate.evaluate_gateand the SARIF writer iscloudg.renderers.sarif.write_sarif.- A GitHub Action at the repository root:
uses: morpheuslord/cloudg@v0.7.1. It installs cloudg and the requested scanners with uv, runscloudg run(or gates afindings.jsonyou pass it), uploads SARIF to code scanning and the reports as an artifact, and fails the job on the gate. Credentials come fromaws-actions/configure-aws-credentials,azure/loginorgoogle-github-actions/auth. cloudg's own CI runs the action against a fixture on every push. - The CI and automation guide covers the action,
cloudg gateon GitLab CI and on any other runner, and Azure and GCP through OIDC.
Fixed:
- The Docker image copies
docs/PYPI.md, the package readme, so the image builds again after 0.7.0 moved the readme.
- Copy the PyPI readme into the Docker image by @morpheuslord in #34
- Feat/ci action 0.7.1 by @morpheuslord in #35
- Update action description for cloud security scan by @morpheuslord in #36
- Shorten the action description to fit the Marketplace limit by @morpheuslord in #37
Full Changelog: v0.7.0...v0.7.1
v0.7.0 - Scanner Expansion and improvements
What's Changed
Azure and GCP are mapped as deeply as AWS, scanner and collector failures raise exceptions you can import and catch, report.html opens on an animated isometric map of the estate, and the docs site moves to MkDocs Material.
Added:
- Azure depth. Besides the Resource Graph breadth query, cloudg reads the
policyresources,securityresources,recoveryservicesresources,networkresources,maintenanceresources,insightsresourcesandkubernetesconfigurationresourcestables, management-group and tenant-scope role assignments, ARM child resources (APIM, Service Bus, Event Hubs, DNS record sets, Front Door, Virtual WAN, Data Factory and Synapse pipelines, AI Search, Azure OpenAI deployments and more), diagnostic settings, Entra ID through Microsoft Graph (users, groups, app registrations, service principals, app-role assignments, OAuth2 grants, directory roles, PIM eligibility) and the objects inside AKS clusters. The ARM type catalog grows from 125 to 278 types.azure.map_entra,azure.child_resources,azure.diagnostic_settingsandazure.data_planeturn the extra sources off. - GCP depth. 350 Cloud Asset Inventory types now have extractors (123 before), and API sources fill what the inventory does not return: Cloud DNS records, Security Command Center, Cloud Identity groups, IAM deny policies and principal access boundaries, PAM, audit configs, Cloud Monitoring, OS Config, Artifact Analysis, Backup and DR, Apigee, Cloud Scheduler and others.
gcp.deep_sourcespicks which ones run. - Azure and GCP record coverage per task family, as AWS does, show "not enabled" placeholders for Defender plans, Sentinel and Security Command Center services, and follow
--services/inventory.exclude_services. cloudg.exceptions:CloudGErrorand its subclasses (ScannerError,ScannerTimeoutError,ScannerNotInstalledError,ScannerExecutionError,ScannerOutputError,ReportParseError,ScannerFailed,CollectionError,CredentialError,ProviderSDKMissingError,ConfigurationError), also exported fromcloudg. Every scanner wrapper takesstrict=Trueto raise on the first failure, records each failure in.failures, and hasraise_for_errors().parse_report(path, strict=True)raisesReportParseErrorfor an unreadable report.- report.html: an estate map in 3D (provider plates, account and region platforms, network slabs, subnet tiles, a control deck for identity and governance), with every relation drawn as a trace line on a lane of its own, asset selection, tracing from the internet, turning, zoom and pan. The D3 tree and force views stay one toggle away. The page also gains a posture overview, a top-risks list, an Assets tab, CSV export, light and dark themes and print styles. The Anatomy iso-kit and the Hairline kernel (both MIT) ship in
cloudg/templates/vendor/and are always embedded. - The docs site is built with MkDocs Material, with API pages generated from the docstrings, tabbed examples, an Examples section, an Exceptions page, a Credits page and a sample report at
/sample-report.html.
Changed:
- Credential, SDK and collection failures raise
CredentialError,ProviderSDKMissingErrorandCollectionError. They keep their old bases (RuntimeError,ImportError), so existingexceptclauses still match.ResilienceErrorandLiveOperationRejectednow derive fromCloudGError. - The PyPI project page is a short overview (
docs/PYPI.md) that links into the docs site.docs/DOCUMENTATION.mdstays the full reference that the MCP server serves.
- Add script to generate sample HTML report by @morpheuslord in #30
- Add files via upload by @morpheuslord in #31
- Credit Prowler and the other sources in the README by @morpheuslord in #32
- Azure and GCP mapped at AWS depth, importable exceptions, an isometric report and a MkDocs docs site by @morpheuslord in #33
Full Changelog: v0.6.1...v0.7.0
v0.6.1 - Documentation and bug fix
A security fix for report.html, a round of fixes found while documenting every command and API, and the new docs site. Upgrade if you open reports built from scanner output you did not produce yourself. Several commands now behave the way their help text always said, so check the Changed list if you script cloudg report, cloudg scan or the exit codes.
Security:
- report.html no longer runs markup or script that arrives in scanner output. Finding titles, descriptions, evidence, resource IDs, asset names and compliance framework names are escaped wherever the page writes them, the page header is autoescaped, and data embedded in the page's script can no longer close the script element.
- A package can no longer take over a built-in scanner (such as
prowler) by registering acloudg.scannersentry point with the same name. Built-in names always win and the duplicate is ignored with a warning.
Fixed:
- report.html works offline. Chart.js 4.4.0 and D3 7.9.0 ship with cloudg and are embedded in the page when
report.inline_jsis true (the default). Withinline_js: falsethe page loads the same pinned builds from cdn.jsdelivr.net with Subresource Integrity. Clicking a row in a filtered findings table now opens that finding. - Config keys that were accepted but ignored now take effect:
graph.persist_graphml,graph.export_cytoscape,graph.max_nodes_warn,rag.chunk_strategy,rag.max_chunk_tokens,report.formats,report.inline_js,report.output_dir(as the default of theCloudGEnginemethods),rulesets.load_externalandontology.include_raw_metadata. A misspelt key in any section is logged asUnknown config key <section>.<key> is ignored, not only underratelimit.rulesets.rules_dir: nullloads the packaged rulesets instead of failing validation. cloudg deps --jsonwrites only JSON to stdout. Whenever--jsonis used, and forcloudg mcp, the banner and logs go to stderr.cloudg report -ikeeps the scan ID, timestamps, compliance results and edges of a findings.json, rebuilds edges for older files that lack them, and accepts raw-findings.json.scanners.timeout_secondslimits each scanner process incloudg run,cloudg scanandCloudGEngine.scan(). Before, the limits were hard-coded at 3600 and 1800 seconds. A scanner that times out is reported as failed and its partial findings are kept.cloudg scanreads-c config.yaml, no longer scans the current directory unless asked, runs the IAM linter when given--assets, and exits 1 when no requested scanner could run.cloudg collectuses the credentials and regions in config.yaml.cloudg map --sweepfollowsinventory.tagging_sweepunless the flag is given.- Prowler and ScoutSuite get the credentials cloudg was given: session token, the assumed
role_arn, web identity andaws.profilefrom the config, and real regions.AWS_DEFAULT_REGION=ALLis no longer set. Region discovery for--regions alluses the same credentials, and falls back to the 17 regions enabled by default in every account when discovery fails. cloudg runexits 3 when collection failed for every target, including a run with no credentials at all where every service fails on its own, and lists the targets that failed. Partial collection still exits 0, with a warning.--accountswithout--role-name(oraws.role_name) is an error instead of mapping the caller's own account once per listed ID.aws.role_arnis assumed first and the member role from there, instead of replacing the member role. An--outhat matches nothing is an error that lists the organization's OUs.- Scanner plugins registered under
cloudg.scannersrun fromscanners.enabledand--scanners, and unknown scanner names produce a warning. The IAM linter followsscanners.enabledand--scannerstoo. CloudGEngine.run_pipeline()passesaws.profileto the scanners,PipelineResult.errorslists collection, scanner and analysis failures as well,on_findingfires as each scanner finishes and receives copies normalisation does not change, and the*_syncwrappers raise a clear error inside a running event loop.CloudGEngine.scan()falls back to a Trivy filesystem scan when no images are set, like the CLI.- Prowler compliance control IDs name the check (
CIS/iam_root_hardware_mfa_enabled) instead of including the account ID, so one check across several accounts is one control. When findings from two scanners are merged, both scanners' check-ID mappings are kept. - Compliance results are no longer all FAIL. A control fails when a finding maps to it and passes when only Prowler passing checks (ASFF
PASSED, OCSFPASS) map to it; controls nothing assessed are left out.FindingsNormaliser.normalise()takespassed_checks=, and the Prowler parser returns them on the list it gives back. - Prowler OCSF output (
*.ocsf.json, the Prowler 4 default) is parsed: status, severity, resource, check ID, compliance and muted findings. Before, every record became an "Unknown Prowler Finding", passing checks included. The MCP loaders accept OCSF instead of refusing it. ScoutSuite documentation URLs no longer show up as compliance frameworks. build_asset_map()andbuild_compliance_map()match findings to assets the way the ontology and RAG exports do, so a finding no longer attaches to every asset that shares its name.CloudGEngine.map_inventory(findings=...)applies the overlay withoutoutput_dirtoo (result.asset_map,result.compliance_map).- GraphML export (
cloudg run,InventoryResult.export(), the MCP export) no longer fails on assets without an ARN.RelationshipLinker.link()can be called more than once.inventory-map.jsonsaves the collection coverage records andInventoryResult.load()restores them. - Each ontology format gets its own file (
.ttl,.jsonld,.rdf,.nt). N-Triples no longer overwrite the Turtle file, and spellings such asjsonldno longer land inontology.ttl.AnalysisResult.rag_chunk_countis filled in. CloudOntology.query_asset_neighbourhood()works withhopsabove 1 and with IDs containing:or/, andquery()returns None for unbound variables.NACL_RULEedges get the same ontology relations asSECURITY_GROUP_RULEedges.ReachabilityAnalyzer.compute_blast_radius()stops at the internet placeholder instead of following an egress rule out to everything the internet reaches.- findings.json metadata timestamps are ISO 8601, and an unset
completed_atisnullrather than"None".
Changed:
cloudg reportrefuses to write findings.json over its own input; pass-oto write elsewhere, or--overwrite. findings.json now has anedgeslist.cloudg scan --scannersdefaults toscanners.enabledinstead ofprowler,checkov, andcloudg scan -paccepts only aws, azure and gcp.- The Terraform recreation goes to
<output>/terraform, following-o, unlessterraform.output_diris set to another path. The old default./reports/terraformcounts as unset. cloudg runwritestopology-cytoscape.jsononly withgraph.export_cytoscape: true, and skipstopology.graphmlwithgraph.persist_graphml: false.run_pipeline()writes the formats inreport.formatsand now includestopology.svg.- RAG chunks are sized by
rag.max_chunk_tokens(about 4 characters per token) instead of fixed caps of 30 relations, 10 findings and 50 triples. Long lists end with an "... and N more" line, and community chunks are bounded too. - report.html is about 485 KB larger with the libraries embedded.
query_asset_neighbourhood()rows for more than one hop are{neighbour, neighbourName, hops}. D3 graph JSON has"arn": ""instead ofnullfor assets without an ARN.cloudg.mcp.state.check_prowler_inputwas removed.ProwlerScannertakes its newer settings (aws_session_token,aws_regions,timeout_seconds,env) and the access keys as keyword arguments; calls that pass them by keyword work as before.aws_regionis now the fifth positional argument.- Parts of three large modules moved into their own: the engine result types and collection checks to
cloudg.api_results, the rate-limit models tocloudg.config_ratelimit, and the RAG chunk text builders tocloudg.graph.rag_text. Everything public is still importable fromcloudg.api,cloudg.configandcloudg.graph.rag_export.
Documentation:
- The handbook at morpheuslord.github.io/cloudg is now a site of separate pages instead of one long
docs/index.html. Guides, the CLI, the Python API, the MCP server, the reference documents and this changelog each have their own tab and sidebar.site/build.pybuilds it: the CLI options tables come from the click definitions, the API pages from the docstrings, the config pages fromconfig.yamland theCloudGConfigmodels, and the longdocs/*.mdreferences are split into one page per section. Writing rules for new pages are insite/AUTHORING.md. - New guides cover installation, the pipeline phase by phase, authentication for each provider, AWS Organizations, scanners and ingest input formats, compliance mapping, reports, the graph, ontology and RAG exports, rate limits, Docker, CI and Python recipes. Code samples are complete scripts, and processes are drawn as diagrams.
- Search (Ctrl K or /), a dark theme, linked code tabs and copy buttons work without a server. Links to the old
index.html#anchorsections redirect to the new pages.
v0.6.0 - MCP Lib + Security
Two additions and a round of graph fixes. cloudg is now a Model Context Protocol server, so AI agents can query an inventory map, its graph, findings, compliance and ontology under an access and privacy policy. Every cloud API call now goes through a shared rate limiter that slows down when a provider throttles, so an organization-wide map (or an agent asking for one too often) no longer fails or hammers the APIs.
The release also changes what the graph analysis reports for the same input. Reachability follows network-flow edges only, matches ports as numbers and gives findings stable ids; the ontology emits different triples; GraphBuilder merges parallel rule edges; the inventory mapper keeps parallel rules when it deduplicates assets; cloudg.retry.with_retry retries a different set of errors. The modules concerned are public API: cloudg.graph.ports (new), cloudg.graph.builder, cloudg.graph.reachability, cloudg.graph.ontology_rules, the ontology and RAG exporters (cloudg.graph.ontology, cloudg.graph.rag_export), cloudg.inventory.dependencies, cloudg.inventory.mapper and cloudg.retry. Embedders that store reachability findings, count graph edges or query the ontology should read Changed and Fixed below.
Added:
- The MCP layer,
cloudg.mcp, built around one class,CloudGMCPLayer. Its default catalog has 74 tools in ten categories (graph 17, inventory 11, findings 10, workspace 8, ontology 6, compliance 5, live 5, privacy 5, meta 4, export 3), 13 resources, 11 resource templates and 11 prompts (security_posture_review,attack_surface_report,blast_radius_assessment,compliance_gap_analysis,incident_triageand six more). The CLI also listscloudg://metrics. Tools return structured content, resource links and progress notifications; lists are paginated and arguments complete. A list cursor belongs to the query and dataset version that produced it: once the dataset is reloaded or replaced, the old cursor is refused with an invalid-arguments error instead of returning the wrong page. - Graph analysis in the MCP catalog uses the same rules as the library. Attack paths and
find_pathsfollow the reachability network-flow hops plus explicit identity pivots (role assumption, IAM trust, grants);internet_exposureand the edge search read Azure*ports andInternet/Any/*sources, GCP all-port rules and AWS ICMP rules, and skip egress rules;find_paths(source="internet")works on an Azure-only dataset. Findings and asset references resolve throughAssetIndex, so an ambiguous name is never matched to the first asset that has it, andreachability_findingsreturns the analyser's deterministic ids. - Tool errors do not repeat the caller's value or list close matches in their message. Both are in the error data instead (
data.value,data.suggestions), where the privacy policy can transform them. - A dependency-free native server (
cloudg.mcp.native): stdio, streamable HTTP and the deprecated SSE endpoints, bearer tokens mapped to roles,OriginandHostchecks against loopback by default, CORS,/healthz. It serves both protocol eras on the same object: the handshake versions 2024-11-05, 2025-03-26, 2025-06-18 and 2025-11-25, and the stateless 2026-07-28 (server/discover,subscriptions/listen).HTTPConfigcaps sessions per principal (max_sessions_per_principal, 100; at the cap the principal's least recently used idle session is closed) and open connections (max_connections, 512), and gives a client 30 seconds to send its headers (header_timeout) and 120 seconds for the body (body_timeout). - Flavors and adapters:
--flavor native,sdk(the official SDK's low-level server),fastmcporauto.register_into(server)mounts the whole catalog into an mcp 2.xMCPServer, an mcp 1.xFastMCP, a low-levelServeror afastmcp.FastMCP(2.x, 3.x, 4.x) next to the host's own tools;build_server()andexport_definitions()cover the other cases. Verified with mcp 1.30.0 and 2.3.0 and fastmcp 2.14.7, 3.4.8 and 4.0.11. On the SDK and fastmcp HTTP transports a session id is bound to the principal that opened it. - Options objects for embedders:
LayerOptions(CloudGMCPLayer(options=...)) andServeOptions(serve_async(options=...)). The existing keyword arguments still work; an unknown keyword raisesTypeError. - A tool handler can take a single pydantic arguments model,
def tool(ctx, args: Model), instead of one parameter per argument. The input schema and the validation are the same as for the flat form. - Content
_metakeyscloudg/dataandcloudg/render: a resource or prompt that renders structured data as text passes the data itself, so the privacy transforms see it with its keys before it is rendered. - Policies: seven built-in profiles (
open,standard(default),strict,read_only,airgapped,audit,soc-analyst) plus policy files that extend each other, with per-role rules, sensitivity ceilings, capability denials and per-principal rate limits. Hidden primitives look the same as nonexistent ones. The textual graph and ontology exports (cloudg://graph/{format},cloudg://ontology/{format},cloudg://ontology/turtle),sparql_queryandsubgraph_exportare RESTRICTED, because the privacy layer cannot see inside one opaque string; a profile with aconfidentialceiling hides them (strict, andsoc-analystfor every role exceptlead). - Transforms, composed per principal into input and output pipelines:
redact,sanitize(fences attacker-controllable text),project,annotate,substitute,alias,regex_replace,rename_keys,template,guard_secrets(refuses secrets in arguments) anddepseudonymize, with built-in detectors for secrets and tokens, private keys, cloud credential ids, ARNs and resource ids, account ids, IP addresses, hostnames and e-mail addresses. TokenVault: format-preserving pseudonyms derived with HMAC-SHA256, stable across restarts withCLOUDG_MCP_VAULT_KEY, and saved when the server shuts down when the policy setsvault.path. A pseudonym an agent passes back as an argument is reversed before the tool runs;reveal_tokenreverses pseudonyms and the aliases introduced bysubstituteandaliastransforms.- Middleware:
AuditLogMiddleware(JSONL, argument values hashed),MetricsMiddleware(cloudg://metrics),CachingMiddleware,ConcurrencyLimitMiddleware,RetryMiddleware, and your ownasync (info, call_next)callables. - CLI:
cloudg mcp serve | tools | resources | prompts | call | read | config, also installed as thecloudg-mcpconsole script and runnable aspython -m cloudg.mcp.cloudg mcp configprints client configuration for Claude Desktop, Claude Code, Cursor and VS Code. Undercloudg mcp, the banner and UI output go to stderr so stdout stays a clean protocol channel. - The MCP workspace reads only files under its allowed roots. With none configured (
allowed_rootsorCLOUDG_MCP_ALLOWED_ROOTS) they are the working directory and the report directory, and either one is left out when it is/or the home directory. A load error never quotes the file's contents, and the search for report files in a directory is bounded. SPARQL queries stop at the row limit or after 60 seconds, and a slow analysis build on one dataset no longer blocks the others or the workspace status. - Live tools (
map_inventory,collect_assets,run_scanners,run_pipeline) check the configured credentials before any cloud call (preflight, on by default, at most 25 seconds) and run under a live operation guard: identical concurrent calls share one run, a scope collected in the last 120 seconds is refused withretry_after_secondsand a loaded live dataset that covers the same providers, and live operations are capped at one per scope and two overall. A live operation that runs longer thanratelimit.live_operation_timeout_seconds(default 3600) is cancelled, raisesLiveOperationTimeoutand frees its scopes.force=trueskips the cooldown for theadminandoperatorroles. Each live tool reports the throttling of its own run.run_scannersrefuses container image names that start with-. The newrate_limit_statustool andcloudg://ratelimitresource show the guard and throttling state without calling a cloud API. cloudg.resilience: provider-aware error classification (classify,classify_strictfor codes, classes and statuses without message text,is_provider_answer,retry_after,describe_error), adaptive token buckets per provider, account, region and service (per action for EC2) that halve on throttling and recover additively, bulkheads, circuit breakers at the granularity of the rate bucket (per action for EC2, per service elsewhere),call_with_resilience/call_with_resilience_sync/@resilientwith decorrelated-jitter backoff that never undercuts Retry-After, a per-call deadline and a per-provider retry budget, per-run telemetry (stats_scope,ResilienceStats), the process-wide governor (get_governor,configure,reset_governor) andLiveOperationGuardwith itsGuardSettings. The first two retries of every call do not draw on the shared retry budget, so one busy run cannot starve every other call of its retries. A half-open breaker lets one probe through and takes the probe back after one cooldown if it never reports; a provider's non-retryable answer (anAccessDenied, a 404) counts as proof that the service is reachable. Calls refused by an open circuit, and calls given up because the retry budget ran out, do not count as breaker failures. Reconfiguring the limits keeps the breakers, budgets and held concurrency slots of providers whose settings did not change.- Provider wiring, with rate limits, concurrency caps, breakers and the shared retry budget enforced in every SDK: botocore event hooks on every AWS session cloudg creates (the collectors' aioboto3 sessions, the base an...
v0.5.3 - Security fix
What's Changed
Fixes CodeQL py/incomplete-url-substring-sanitization alerts #27 to #33.
- GCP relation extractors (context.py, iam.py, names.py) identified a resource's API service with prefix tests on host-like literals. They now parse the service with anchored regular expressions via new api_service() / asset_type_service() helpers. Behaviour is unchanged: a fuzz of old vs new over 92,147 generated inputs found no differences.
- Three test assertions that checked host-like literals against lists with
innow use set comparison. - Version bumped to 0.5.3 (pyproject.toml, cloudg/init.py, README badge, handbook); CHANGELOG and handbook entries added.
ruff check clean, 382 tests pass.
Full Changelog: v0.5.2...v0.5.3
v0.5.2 - Code Docs
What's Changed
Documentation release, no code changes.
- docs/INVENTORY_REFERENCE.md: every return value and exported file of the inventory mapper, field by field, with annotated examples from a real run.
- docs/INVENTORY_CATALOG.md: all 156 asset types with their native type mappings, metadata keys, declared relations, and the full relationship matrix.
- docs/INVENTORY_INTERNALS.md: architecture, code map, all 133 AWS tasks, linker resolution rules, Azure/GCP extractor frameworks, dependency semantics, extension recipes, security rules, code quality limits, testing.
- DOCUMENTATION.md, the handbook and README: full AssetType/EdgeType sets, NetworkEdge fields, collector option signatures, links to the new documents.
- Handbook changelog shows 0.5.0, 0.5.1 and 0.5.2 as separate releases.
- Version bumped to 0.5.2 (pyproject.toml, cloudg/init.py, README badge, handbook); CHANGELOG has a 0.5.2 entry.
Full Changelog: v0.5.1...v0.5.2
v0.5.1 deeper inventory
A much deeper inventory map. cloudg map now follows workloads into containers and Kubernetes, through serverless and event wiring, across the IAM graph and the security services watching it all, and across every account of an AWS Organization / Control Tower landing zone. Edges say what they mean, and cloudg deps answers what an asset needs and what breaks with it.
Added:
- Deep AWS service collectors in
cloudg.inventory.aws_services, composed intoAWSDeepInventoryCollectorand grouped into service families selectable with--services/--exclude-services(inventory.services,inventory.exclude_services):- containers: ECR repositories (image sample, scan findings summary, registry scan mode, repository policy grants), ECS services and task definitions (images, task and execution roles, secrets, log groups, target groups, subnets), EKS clusters, nodegroups, Fargate profiles, addons, access entries and pod identity associations;
- Kubernetes: namespaces, Deployments, StatefulSets, DaemonSets, CronJobs, Services, Ingresses and ServiceAccounts read from inside EKS clusters with a read-only EKS token (
cloudg.inventory.kubernetes, no new dependency;--kubernetes/--no-kubernetes); - serverless and integration: Lambda event source mappings, resource-policy invokers, function URLs, container images, layers, DLQs and EFS mounts; API Gateway REST and HTTP APIs; SQS; SNS subscriptions; EventBridge buses, rules and targets; Step Functions; Kinesis;
- data, DNS and deployment: deep S3 (real region, notifications, replication, access logging, policy grants), ALB/NLB listeners, certificates and target groups, classic ELB, Auto Scaling groups, launch templates, deep RDS and Aurora, EFS, ElastiCache, OpenSearch, Redshift, Route 53 records, CloudFormation stacks and the resources they manage, log groups, ACM in-use-by, VPC endpoints, flow logs, transit gateway attachments;
- security services and scanners: GuardDuty, Security Hub, Inspector (per-resource coverage), Macie, AWS Config, IAM Access Analyzer, Detective, WAFv2, Network Firewall, Shield Advanced, CloudTrail. Services that are not enabled appear as
enabled: falseplaceholders; - identity: the full IAM graph from
GetAccountAuthorizationDetails: trust policies (cross-account, OIDC / IRSA / GitHub), managed policy attachments, group membership, instance profiles, admin detection, and principal → resource grants.
- AWS Organizations and Control Tower:
cloudg map --org(aws.organizationconfig) discovers the OU tree, every member account, SCPs/RCPs and other org policies, delegated administrators and, when present, the Control Tower landing zone (version, drift, governed regions, log archive / audit / config / backup accounts), enabled controls and baselines, then maps every selected account by assuming--org-role(defaultAWSControlTowerExecution). Filters:--ou,--exclude-account; governed regions replace--regions all.cloudg.inventory.discover_organization()/OrganizationTopologyare public. - Typed relationships: new
EdgeTypevaluesINVOKES,USES_IMAGE,ASSUMES_ROLE,GRANTS_ACCESS,LOGS_TO,PROTECTS,MONITORS,MANAGES,GOVERNS;NetworkEdge.relationship(ontology relation name) andNetworkEdge.properties. Collectors declare relationships inmetadata["relations"], which the linker resolves. - Dependency analysis:
cloudg.inventory.DependencyGraph(depends_on,dependents,tree,shared_dependencies,blast_radius), cross-account edges and security coverage (gaps, workloads without vulnerability scanning, internet-facing endpoints without WAF);InventoryResult.dependency_graph(),analysis(),load();inventory-dependencies.jsonandinventory-organization.jsonexports; thecloudg depscommand. - 47 new
AssetTypevalues covering containers, Kubernetes, messaging, data, DNS, IaC, security services and organization structure, mapped to new OWL classes; richer Azure ARM and GCP Cloud Asset Inventory type maps. cloudg mapflags:--accounts,--role-name,--org,--org-role,--ou,--exclude-account,--ct-home-region,--services,--exclude-services,--kubernetes/--no-kubernetes.inventoryconfig:services,exclude_services,kubernetes,kubernetes_timeout,iam_resource_edges,max_images_per_repository,stack_resources,account_hierarchy.- Coverage expansion after a full audit of every AWS, Azure and GCP service against what cloudg enumerated:
- AWS breadth: a Cloud Control API sweep lists every resource type with a list handler, tagged or not (the tagging API never returns resources that were never tagged), skipping types a dedicated collector covers.
--cloud-control/--no-cloud-control,inventory.cloud_control*. - AWS depth, 133 collectors in all (
cloudg.inventory.aws_services.{governance,network_ext,application,data_ml}): IAM Identity Center (permission sets, assignments, users, groups, provisioned roles), RAM shares, Service Catalog / Account Factory, StackSets, SAML providers, access key metadata, Roles Anywhere, Cognito, deep KMS / Secrets Manager / DynamoDB, snapshot and AMI sharing, S3 access points and account Block Public Access; VPN, customer and virtual private gateways, transit gateway route tables and peering, PrivateLink services, prefix lists, egress-only gateways, API Gateway authorizers / VPC links / custom domains, deep CloudFront, Route 53 Resolver, Direct Connect, Global Accelerator, listener rules, VPC Lattice, Cloud WAN; CodePipeline / CodeBuild / CodeDeploy / CodeConnections, SSM, CloudWatch alarms, AWS Backup, ECS capacity providers and container instances, Cloud Map, Lambda aliases, Scheduler, Pipes, API destinations, archives, Firehose, log subscriptions and destinations, OAM, Batch, App Runner, Elastic Beanstalk; MSK, Amazon MQ, SageMaker, Bedrock, OpenSearch Serverless, Redshift Serverless, RDS proxies and global clusters, MemoryDB, DAX, Glue, Lake Formation, EMR, Athena, Transfer Family, DataSync, FSx, ECR Public. - Azure: collection through Azure Resource Graph with full properties (
azure-mgmt-resourcegraphadded to theazureextra; SDK fallback without it), typed relationships for managed identities, role assignments, private endpoints, AKS, App Service, Container Apps, networking, Key Vault, storage, SQL and Defender plans; every Enabled subscription; management groups, Azure Landing Zone archetypes and Azure Policy (cloudg.inventory.azure_hierarchy,azure.all_subscriptions,azure.map_management_groups). - GCP: collection through Cloud Asset Inventory
list_assetswith full resource JSON, org-wide whengcp.organization_idis set; typed relationships across compute, firewalls, load balancing, GKE / Workload Identity, Cloud Run, Functions, Pub/Sub, Eventarc, logging sinks, CMEK and Shared VPC (cloudg.inventory.gcp_relations); IAM bindings to service accounts and principal nodes; organization → folder → project hierarchy, organization policies and VPC Service Controls (cloudg.inventory.gcp_hierarchy); newgcpoptionscollection_scope,skip_asset_types,iam_policies,map_hierarchy,org_policies,vpc_service_controls. - 62 further
AssetTypevalues (identity federation, provisioning, hybrid networking, operations, data processing, ML), 156 in all.
- AWS breadth: a Cloud Control API sweep lists every resource type with a list handler, tagged or not (the tagging API never returns resources that were never tagged), skipping types a dedicated collector covers.
- Reference catalogs: type maps and skip lists moved out of code into YAML under
cloudg/inventory/catalogs/(aws_cloudcontrol,aws_arn_types,azure_arm_types,gcp_asset_types), validated at load and extensible through$CLOUDG_CATALOG_DIRoverlays.
Full Changelog: v0.5.0...v0.5.1
v0.5.0 - Inventory system with security fixes
Inventory mapping. A dedicated, scanner-independent function that maps everything deployed (or default) in a cloud estate and how it interlinks. It answers what exists and how it is wired together; the scanners keep answering what is wrong.
Added:
cloudg mapCLI command: deep-collects the complete infrastructure across AWS, Azure and GCP without running any scanner, links it into an interconnected map, and writesinventory-map.json,inventory-map.graphmlandinventory-graph.json.--findingsmerges previously generated scanner findings intoasset-map.json(asset → risk) andcompliance-map.json(framework → affected assets).cloudg.inventorypackage, all public API:AWSDeepInventoryCollectoradds route tables, internet/NAT gateways, network interfaces, EBS volumes, Elastic IPs, NACLs, VPC peering, transit gateways and customer-managed IAM policies on top of the standard collector, plus a catch-all sweep of the Resource Groups Tagging API so every taggable resource appears on the map.AzureDeepInventoryCollectoradds NICs, public IPs, managed disks, load balancers and route tables, plus a full-subscription ARMresources.list()sweep covering every service.GCPDeepInventoryCollectorbrings a richer Cloud Asset Inventory type taxonomy and IAM policy bindings as attachment edges.RelationshipLinkerderives interconnection edges purely from asset metadata (attachment, containment, routing, peering, and a generic cross-service reference pass) and works on anylist[CloudAsset].InventoryMapper/InventoryResult: orchestration, summaries, exports, and thebuild_asset_map()/build_compliance_map()/export_merged()merge helpers.
CloudGEngine.map_inventory()/map_inventory_sync()for embedders.inventory:config section (tagging_sweep,link_references).- New schema values:
AssetType.NETWORK_INTERFACE,EdgeType.ATTACHED_TO,EdgeType.REFERENCES. MultiAccountCollectoraccepts per-provider collector class overrides.- Documentation: inventory mapping is covered in the README, the handbook (
cloudg map, the inventory mapping API) and the docs site; a new "deeper toolkit" chapter documents the standalone programmatic APIs (GraphBuilder, CloudOntology, RAGExporter, TerraformExporter, FindingsNormaliser, parsers).
Fixed (pre-release hardening, folded in before publish):
- The engine's Checkov path passed
framework=to a constructor that takesframeworks=(a list) and raised TypeError; it now forwards the configured framework list. - The GCP identifier checks in the inventory linker and mapper are anchored regexes, so
.googleapis.cominside an unrelated ARN no longer classifies an asset as GCP (CodeQL py/incomplete-url-substring-sanitization). - The Pages deploy artifact carries the run attempt in its name; re-running the workflow no longer fails on "Multiple artifacts named github-pages".
- install.sh no longer uses
A && B || Cas if-then-else anywhere; every step is an explicit if/else, with apip_toolhelper for the repeated pip installs. - Dockerfile pins parliament to 1.6.4.
- Codacy complexity findings: the long CLI commands (
run,scan,ingest), the engine phases (scan,analyze,run_pipeline), the SVG/HTML renderers, the ontology relation inference, the RAG exporters, credentials, registry discovery and the Prowler/IAM-linter scanners were split into focused helpers; behavior is unchanged and the CLIruncommand no longer takes 27 named parameters. - Codacy file-length findings: the four oversized modules were split along their natural seams, with every import path preserved — the CLI helpers moved to
cli_helpers.py/cli_run_helpers.pyand theruncommand tocli_commands.py; the AWS per-service collectors moved toaws_services.py/aws_services_extended.pymixins; the ontology relation-inference layer moved toontology_rules.py(re-exported fromontology); the SVG hierarchical layout moved tosvg_layout.py. The SVG output is byte-identical and the whole suite passes unchanged.
What's Changed
- Add a report viewer to the docs site by @morpheuslord in #11
- Feat/inventory mapping by @morpheuslord in #12
- Anchor GCP identifier checks and fix Pages re-run deploys by @morpheuslord in #13
- Fix/codacy findings by @morpheuslord in #14
- Format the inventory modules to satisfy ruff format by @morpheuslord in #15
Full Changelog: v0.4.1...v0.5.0
v0.4.1 Security hardening
Security hardening pass over the Checkov and Codacy findings.
- The Docker image now runs as a non-root
cloudguser and carries a HEALTHCHECK. install.shno longer pipes curl output straight into bash; installer scripts are downloaded to a temp file first (Homebrew, NodeSource, Azure CLI).- Third-party GitHub Actions are pinned to full commit SHAs across all workflows.
- Plugin loading validates the
module:Classpath shape beforeimportlib.import_module. - Retry jitter uses the system RNG.
- Silent
except: passhandlers now log at debug level, so swallowed errors are traceable. - Log messages no longer put words like "credentials" next to runtime values.
- The vulnerability-report email moved out of SECURITY.md in favour of GitHub private reporting.
- A
.codacy.ymlscopes analysis to shipped code and documents suppressed false positives (pytest asserts, taxonomy names such asAssetType.SECRET).
What's Changed
- Version releases from the git tag and show the docs on PyPI by @morpheuslord in #7
- Copy the package readme into the Docker build context by @morpheuslord in #8
- Security hardening: resolve Checkov and Codacy findings (0.4.1) by @morpheuslord in #10
Full Changelog: v0.4.0...v0.4.1
v0.4.0 cloudg ingestion mode
Ingest mode. cloudg can now work entirely from scanner outputs you already have, with no cloud credentials and no scanner binaries on the machine.
Added:
cloudg ingestCLI command. Takes--prowler,--scoutsuite,--checkovand--trivypaths (file or directory, each repeatable, any combination), runs the normaliser and the check-equivalence merge over them, and renders the HTML and JSON reports.parse_report(path)classmethods onProwlerScanner,ScoutSuiteScanner,CheckovScannerandTrivyScanner. Each parses the tool's native output format directly. Trivy image and filesystem reports are told apart by theArtifactTypefield.cloudg.ingestmodule withparse_report(tool, path)andingest_reports(mapping).CloudGEngine.ingest_reports(),CloudGEngine.normalise_findings()andCloudGEngine.run_from_reports()/run_from_reports_sync()for embedders.- Feature documentation:
docs/DOCUMENTATION.md, a rendered handbook atdocs/index.html, and this changelog.
Fixed:
- The publish workflow now sets an explicit least-privilege
permissionsblock (CodeQLactions/missing-workflow-permissions).
Full Changelog: v0.3.2...v0.4.0