Skip to content

v0.5.1 deeper inventory

Choose a tag to compare

@morpheuslord morpheuslord released this 04 Oct 15:20
· 111 commits to main since this release
43e18c8

A much deeper inventory map. cloudg map now follows workloads into containers and Kubernetes, through serverless and event wiring, across the IAM graph and the security services watching it all, and across every account of an AWS Organization / Control Tower landing zone. Edges say what they mean, and cloudg deps answers what an asset needs and what breaks with it.

Added:

  • Deep AWS service collectors in cloudg.inventory.aws_services, composed into AWSDeepInventoryCollector and grouped into service families selectable with --services / --exclude-services (inventory.services, inventory.exclude_services):
    • containers: ECR repositories (image sample, scan findings summary, registry scan mode, repository policy grants), ECS services and task definitions (images, task and execution roles, secrets, log groups, target groups, subnets), EKS clusters, nodegroups, Fargate profiles, addons, access entries and pod identity associations;
    • Kubernetes: namespaces, Deployments, StatefulSets, DaemonSets, CronJobs, Services, Ingresses and ServiceAccounts read from inside EKS clusters with a read-only EKS token (cloudg.inventory.kubernetes, no new dependency; --kubernetes/--no-kubernetes);
    • serverless and integration: Lambda event source mappings, resource-policy invokers, function URLs, container images, layers, DLQs and EFS mounts; API Gateway REST and HTTP APIs; SQS; SNS subscriptions; EventBridge buses, rules and targets; Step Functions; Kinesis;
    • data, DNS and deployment: deep S3 (real region, notifications, replication, access logging, policy grants), ALB/NLB listeners, certificates and target groups, classic ELB, Auto Scaling groups, launch templates, deep RDS and Aurora, EFS, ElastiCache, OpenSearch, Redshift, Route 53 records, CloudFormation stacks and the resources they manage, log groups, ACM in-use-by, VPC endpoints, flow logs, transit gateway attachments;
    • security services and scanners: GuardDuty, Security Hub, Inspector (per-resource coverage), Macie, AWS Config, IAM Access Analyzer, Detective, WAFv2, Network Firewall, Shield Advanced, CloudTrail. Services that are not enabled appear as enabled: false placeholders;
    • identity: the full IAM graph from GetAccountAuthorizationDetails: trust policies (cross-account, OIDC / IRSA / GitHub), managed policy attachments, group membership, instance profiles, admin detection, and principal → resource grants.
  • AWS Organizations and Control Tower: cloudg map --org (aws.organization config) discovers the OU tree, every member account, SCPs/RCPs and other org policies, delegated administrators and, when present, the Control Tower landing zone (version, drift, governed regions, log archive / audit / config / backup accounts), enabled controls and baselines, then maps every selected account by assuming --org-role (default AWSControlTowerExecution). Filters: --ou, --exclude-account; governed regions replace --regions all. cloudg.inventory.discover_organization() / OrganizationTopology are public.
  • Typed relationships: new EdgeType values INVOKES, USES_IMAGE, ASSUMES_ROLE, GRANTS_ACCESS, LOGS_TO, PROTECTS, MONITORS, MANAGES, GOVERNS; NetworkEdge.relationship (ontology relation name) and NetworkEdge.properties. Collectors declare relationships in metadata["relations"], which the linker resolves.
  • Dependency analysis: cloudg.inventory.DependencyGraph (depends_on, dependents, tree, shared_dependencies, blast_radius), cross-account edges and security coverage (gaps, workloads without vulnerability scanning, internet-facing endpoints without WAF); InventoryResult.dependency_graph(), analysis(), load(); inventory-dependencies.json and inventory-organization.json exports; the cloudg deps command.
  • 47 new AssetType values covering containers, Kubernetes, messaging, data, DNS, IaC, security services and organization structure, mapped to new OWL classes; richer Azure ARM and GCP Cloud Asset Inventory type maps.
  • cloudg map flags: --accounts, --role-name, --org, --org-role, --ou, --exclude-account, --ct-home-region, --services, --exclude-services, --kubernetes/--no-kubernetes.
  • inventory config: services, exclude_services, kubernetes, kubernetes_timeout, iam_resource_edges, max_images_per_repository, stack_resources, account_hierarchy.
  • Coverage expansion after a full audit of every AWS, Azure and GCP service against what cloudg enumerated:
    • AWS breadth: a Cloud Control API sweep lists every resource type with a list handler, tagged or not (the tagging API never returns resources that were never tagged), skipping types a dedicated collector covers. --cloud-control/--no-cloud-control, inventory.cloud_control*.
    • AWS depth, 133 collectors in all (cloudg.inventory.aws_services.{governance,network_ext,application,data_ml}): IAM Identity Center (permission sets, assignments, users, groups, provisioned roles), RAM shares, Service Catalog / Account Factory, StackSets, SAML providers, access key metadata, Roles Anywhere, Cognito, deep KMS / Secrets Manager / DynamoDB, snapshot and AMI sharing, S3 access points and account Block Public Access; VPN, customer and virtual private gateways, transit gateway route tables and peering, PrivateLink services, prefix lists, egress-only gateways, API Gateway authorizers / VPC links / custom domains, deep CloudFront, Route 53 Resolver, Direct Connect, Global Accelerator, listener rules, VPC Lattice, Cloud WAN; CodePipeline / CodeBuild / CodeDeploy / CodeConnections, SSM, CloudWatch alarms, AWS Backup, ECS capacity providers and container instances, Cloud Map, Lambda aliases, Scheduler, Pipes, API destinations, archives, Firehose, log subscriptions and destinations, OAM, Batch, App Runner, Elastic Beanstalk; MSK, Amazon MQ, SageMaker, Bedrock, OpenSearch Serverless, Redshift Serverless, RDS proxies and global clusters, MemoryDB, DAX, Glue, Lake Formation, EMR, Athena, Transfer Family, DataSync, FSx, ECR Public.
    • Azure: collection through Azure Resource Graph with full properties (azure-mgmt-resourcegraph added to the azure extra; SDK fallback without it), typed relationships for managed identities, role assignments, private endpoints, AKS, App Service, Container Apps, networking, Key Vault, storage, SQL and Defender plans; every Enabled subscription; management groups, Azure Landing Zone archetypes and Azure Policy (cloudg.inventory.azure_hierarchy, azure.all_subscriptions, azure.map_management_groups).
    • GCP: collection through Cloud Asset Inventory list_assets with full resource JSON, org-wide when gcp.organization_id is set; typed relationships across compute, firewalls, load balancing, GKE / Workload Identity, Cloud Run, Functions, Pub/Sub, Eventarc, logging sinks, CMEK and Shared VPC (cloudg.inventory.gcp_relations); IAM bindings to service accounts and principal nodes; organization → folder → project hierarchy, organization policies and VPC Service Controls (cloudg.inventory.gcp_hierarchy); new gcp options collection_scope, skip_asset_types, iam_policies, map_hierarchy, org_policies, vpc_service_controls.
    • 62 further AssetType values (identity federation, provisioning, hybrid networking, operations, data processing, ML), 156 in all.
  • Reference catalogs: type maps and skip lists moved out of code into YAML under cloudg/inventory/catalogs/ (aws_cloudcontrol, aws_arn_types, azure_arm_types, gcp_asset_types), validated at load and extensible through $CLOUDG_CATALOG_DIR overlays.

Full Changelog: v0.5.0...v0.5.1