Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 44 additions & 2 deletions .github/workflows/main-validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ jobs:
tenant-id: ${{ secrets.AZURE_TENANT_ID }} # Azure AD tenant
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
enable-AzPSSession: false
allow-no-subscription: false
allow-no-subscriptions: false

- name: Test Azure doctor (strict)
run: |
Expand All @@ -174,13 +174,55 @@ jobs:
name: azure-integration-main
path: test-results.json

integration-test-azure-multi-subscription:
name: Integration Test - Azure Multi-Subscription (Required)
runs-on: ubuntu-latest
needs: validate
environment: cleancloud-test

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install CleanCloud
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"

- name: Azure Login via OIDC (no subscription pin — scans all)
uses: azure/login@v2
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
enable-AzPSSession: false
allow-no-subscriptions: true

- name: Test Azure multi-subscription scan
run: |
set -e
cleancloud scan \
--provider azure \
--output json \
--output-file multi-subscription-results.json

- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
with:
name: azure-multi-subscription-main
path: multi-subscription-results.json

# =========================
# NOTIFY ON FAILURE
# =========================
notify-failure:
name: Create Issue on Failure
runs-on: ubuntu-latest
needs: [validate, integration-test-aws, integration-test-aws-multi-account, integration-test-azure]
needs: [validate, integration-test-aws, integration-test-aws-multi-account, integration-test-azure, integration-test-azure-multi-subscription]
if: failure()

steps:
Expand Down
45 changes: 43 additions & 2 deletions .github/workflows/pr-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ jobs:
tenant-id: ${{ secrets.AZURE_TENANT_ID }} # Azure AD tenant
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
enable-AzPSSession: false
allow-no-subscription: false
allow-no-subscriptions: false

- name: Run tests
run: |
Expand Down Expand Up @@ -236,7 +236,7 @@ jobs:
tenant-id: ${{ secrets.AZURE_TENANT_ID }} # Azure AD tenant
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
enable-AzPSSession: false
allow-no-subscription: false
allow-no-subscriptions: false

- name: Test Azure doctor command
run: |
Expand All @@ -255,3 +255,44 @@ jobs:
with:
name: azure-integration-test-results
path: test-results.json

integration-test-azure-multi-subscription:
name: Integration Test - Azure Multi-Subscription
runs-on: ubuntu-latest
continue-on-error: true
environment: cleancloud-test

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install CleanCloud
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"

- name: Azure Login via OIDC (no subscription pin — scans all)
uses: azure/login@v2
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
enable-AzPSSession: false
allow-no-subscriptions: true

- name: Test Azure multi-subscription scan
run: |
cleancloud scan \
--provider azure \
--output json \
--output-file multi-subscription-results.json

- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
with:
name: azure-multi-subscription-pr-results
path: multi-subscription-results.json
63 changes: 60 additions & 3 deletions README.fr.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,9 @@ Hygiène cloud en lecture seule pour les environnements réglementés & souverai
CleanCloud scanne votre environnement cloud et rapporte ce qui gaspille de l'argent. Exécutez-le une fois pour un audit ponctuel, planifiez-le, ou intégrez-le en CI/CD pour bloquer les builds sur des violations de politique.

- **20 règles de détection haut signal :** volumes orphelins, bases de données inactives, load balancers vides, et plus
- **Gaspillage mensuel estimé :** par finding et en agrégat
- **Scan multi-comptes :** scannez des AWS Organizations entières en quelques minutes — fichier de config, IDs inline, ou auto-découverte via `--org`
- **Gaspillage mensuel estimé :** par finding et en agrégat, détaillé par compte et abonnement
- **Scan multi-comptes (AWS) :** scannez des AWS Organizations entières en quelques minutes — fichier de config, IDs inline, ou auto-découverte via `--org`
- **Scan multi-abonnements (Azure) :** scannez tous les abonnements Azure en parallèle avec une seule identité — auto-découverte via Management Group ou tous les accessibles — détail des coûts par abonnement inclus
- **Application de politique CI/CD (opt-in) :** `--fail-on-confidence HIGH` ou `--fail-on-cost 100` gate votre pipeline
- **Formats de sortie multiples :** lisible, JSON, CSV, et markdown (à coller dans vos PRs GitHub ou Slack)
- **Lecture seule par conception :** aucune suppression, aucune modification de tags, aucune mutation — jamais
Expand Down Expand Up @@ -99,6 +100,10 @@ Régions scannées : us-east-1, us-west-2, eu-west-1
--concurrency N Comptes en parallèle (défaut : 3)
--timeout SECONDS Timeout total du scan en secondes (défaut : 3600)

# Multi-abonnements — Azure uniquement (optionnel)
--management-group ID Scanner tous les abonnements d'un Management Group
--subscription ID Scanner un seul abonnement (défaut : tous les accessibles)

# Sortie (optionnel)
--output human|json|csv|markdown Format de sortie (défaut : human)
--output-file FILE Écrit la sortie dans un fichier
Expand All @@ -123,8 +128,18 @@ cleancloud demo # visualisez des findings sans aucun credential cloud
```bash
docker pull getcleancloud/cleancloud
docker run --rm getcleancloud/cleancloud demo

# Avec credentials AWS (Docker n'hérite pas de ~/.aws automatiquement)
docker run --rm \
-e AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN \
-e AWS_REGION=us-east-1 \
getcleancloud/cleancloud scan --provider aws --all-regions
```

> En CI/CD, `aws-actions/configure-aws-credentials` définit les variables `AWS_*` sur le runner — passez-les avec `-e VAR_NAME` et elles sont transmises au conteneur automatiquement. Voir [Guide CI/CD →](docs/ci.md#using-the-docker-image)

Prêt à scanner votre vrai environnement ? Authentifiez-vous d'abord, puis lancez :

```bash
Expand Down Expand Up @@ -385,12 +400,54 @@ Guide complet (politique IAM, trust policy, templates IaC) : [Configuration mult

---

## Scan multi-abonnements (Azure)

Conçu pour les entreprises gérant de grands tenants Azure. Scannez chaque abonnement en parallèle avec une seule identité — findings agrégés dans un rapport unique avec détail des coûts par abonnement.

```bash
# Scanner tous les abonnements accessibles (défaut)
cleancloud scan --provider azure

# Auto-découverte via Management Group
cleancloud scan --provider azure --management-group <MANAGEMENT_GROUP_ID>

# Liste explicite
cleancloud scan --provider azure --subscription <SUB_1> --subscription <SUB_2>
```

**Permissions requises :**

| Périmètre | Rôle |
|---|---|
| Chaque abonnement | Reader (intégré) |
| Management Group (si `--management-group`) | Reader + `Microsoft.Management/managementGroups/read` |

Assignez Reader au niveau du Management Group — il hérite automatiquement à tous les abonnements en dessous :

```bash
az role assignment create \
--assignee <SERVICE_PRINCIPAL_CLIENT_ID> \
--role Reader \
--scope /providers/Microsoft.Management/managementGroups/<MANAGEMENT_GROUP_ID>
```

**Fonctionnement :**

- **Modèle d'identité plat** — un seul service principal, Reader au niveau du Management Group. Pas d'assumption de rôle inter-abonnements, pas de complexité hub-and-spoke.
- **Trois modes de découverte** — tous les accessibles (défaut), `--management-group` pour l'auto-découverte, `--subscription` pour un contrôle explicite.
- **Parallèle avec isolation** — chaque abonnement s'exécute dans son propre thread. Un abonnement en échec (permission refusée, timeout) n'affecte jamais les autres.
- **Gestion gracieuse des permissions** — les règles échouant avec 403 sont signalées comme ignorées (avec la permission manquante nommée), pas comme des échecs de scan.
- **Détail des coûts par abonnement** — la sortie indique le gaspillage mensuel estimé par abonnement pour identifier précisément lequel est problématique.

Guide complet (RBAC, Workload Identity, Management Group) : [Configuration multi-abonnements Azure →](docs/azure.md#multi-subscription-scanning)

---

## Feuille de route

- Règles AWS supplémentaires (cycle de vie S3, instances EC2 arrêtées)
- Policy-as-code dans `cleancloud.yaml` (`fail_on_confidence`, `fail_on_cost` en config)
- Filtrage de règles (flag `--rules`)
- Scan Azure Management Groups (multi-abonnements au niveau org)

---

Expand Down
63 changes: 60 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,9 @@ Read-only cloud hygiene for regulated & sovereign environments.
CleanCloud scans your cloud environment and reports what's wasting money. Run it once for a quick audit, schedule it, or wire it into CI/CD to fail builds on policy violations.

- **20 high-signal detection rules:** orphaned volumes, idle databases, empty load balancers, and more
- **Estimated monthly waste:** per finding and aggregate
- **Multi-account scanning:** scan entire AWS Organizations in minutes — config file, inline IDs, or auto-discovery via `--org`
- **Estimated monthly waste:** per finding and aggregate, broken down per account and subscription
- **Multi-account scanning (AWS):** scan entire AWS Organizations in minutes — config file, inline IDs, or auto-discovery via `--org`
- **Multi-subscription scanning (Azure):** scan all Azure subscriptions in parallel with one identity — auto-discovery via Management Group or all accessible — per-subscription cost breakdown included
- **CI-native enforcement (opt-in):** `--fail-on-confidence HIGH` or `--fail-on-cost 100` gates your pipeline
- **Multiple output formats:** human-readable, JSON, CSV, and markdown (paste into GitHub PRs or Slack)
- **Read-only by design:** no deletions, no tag changes, no mutations — ever
Expand Down Expand Up @@ -92,8 +93,18 @@ cleancloud demo # see sample findings without any cloud credentials
```bash
docker pull getcleancloud/cleancloud
docker run --rm getcleancloud/cleancloud demo

# With AWS credentials (Docker doesn't inherit local ~/.aws automatically)
docker run --rm \
-e AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN \
-e AWS_REGION=us-east-1 \
getcleancloud/cleancloud scan --provider aws --all-regions
```

> In CI/CD, `aws-actions/configure-aws-credentials` sets `AWS_*` env vars on the runner — pass them with `-e VAR_NAME` and they forward into the container automatically. See [CI/CD guide →](docs/ci.md#using-the-docker-image)

When you're ready to scan your real environment, authenticate first — then run:

```bash
Expand Down Expand Up @@ -124,6 +135,10 @@ Not sure if your credentials have the right permissions? Run `cleancloud doctor
--concurrency N Parallel accounts (default: 3)
--timeout SECONDS Total scan timeout in seconds (default: 3600)

# Multi-subscription — Azure only (optional)
--management-group ID Scan all subscriptions under a Management Group
--subscription ID Scan a single subscription (default: all accessible)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have : Multi-Account Scanning (AWS only) section currently..
do we need a separate one for azure for parity ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for highlighting.. just added it

# Output (optional)
--output human|json|csv|markdown Output format (default: human)
--output-file FILE Write output to file instead of stdout
Expand Down Expand Up @@ -387,12 +402,54 @@ Full setup guide (IAM policy, trust policy, IaC templates): [AWS multi-account s

---

## Multi-Subscription Scanning (Azure)

Built for enterprises running large Azure tenants. Scan every subscription in parallel with one identity — findings aggregated into one report with a per-subscription cost breakdown.

```bash
# Scan all subscriptions the service principal can access (default)
cleancloud scan --provider azure

# Auto-discover via Management Group
cleancloud scan --provider azure --management-group <MANAGEMENT_GROUP_ID>

# Explicit list
cleancloud scan --provider azure --subscription <SUB_1> --subscription <SUB_2>
```

**Permissions required:**

| Scope | Role |
|---|---|
| Each subscription | Reader (built-in) |
| Management Group (if using `--management-group`) | Reader + `Microsoft.Management/managementGroups/read` |

Assign Reader at the Management Group level and it inherits to all subscriptions underneath — no per-subscription role assignment needed:

```bash
az role assignment create \
--assignee <SERVICE_PRINCIPAL_CLIENT_ID> \
--role Reader \
--scope /providers/Microsoft.Management/managementGroups/<MANAGEMENT_GROUP_ID>
```

**How it works:**

- **Flat identity model** — one service principal, Reader at Management Group level. No cross-subscription role assumption, no hub-and-spoke complexity.
- **Three discovery modes** — all accessible (default), `--management-group` for auto-discovery, `--subscription` for explicit control.
- **Parallel with isolation** — each subscription runs in its own thread. One subscription failing (permission denied, timeout) never affects the others.
- **Graceful permission handling** — rules that fail with 403 are reported as skipped (with the missing permission named), not as scan failures.
- **Per-subscription cost breakdown** — output shows estimated monthly waste per subscription so you can see exactly which subscription is dirty.

Full setup guide (RBAC, Workload Identity, Management Group): [Azure multi-subscription setup →](docs/azure.md#multi-subscription-scanning)

---

## Roadmap

- Additional AWS rules (S3 lifecycle, stopped EC2 instances)
- Policy-as-code in `cleancloud.yaml` (`fail_on_confidence`, `fail_on_cost` in config)
- Rule filtering (`--rules` flag)
- Azure Management Group scanning (multi-subscription org-level)

---

Expand Down
32 changes: 30 additions & 2 deletions cleancloud/output/summary.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,17 +78,22 @@ def _print_summary(summary: dict, region_selection_mode: str = None, multi_accou
# Use provider-aware label
provider = summary.get("provider", "aws")
if provider == "azure":
subscriptions_scanned = summary.get("subscriptions_scanned", [])
label = "Subscriptions scanned"
regions_str = ", ".join(subscriptions_scanned) if subscriptions_scanned else regions_str
else:
label = "Regions scanned"

click.echo(f"\n{label}: {regions_str}", nl=False)

# Selection mode annotations
if provider == "azure":
if region_selection_mode == "all":
mode = summary.get("subscription_selection_mode", "")
if mode == "all":
click.echo(" (all accessible)")
elif region_selection_mode == "explicit":
elif mode == "management-group":
click.echo(" (management group)")
elif mode == "explicit":
click.echo(" (explicit)")
else:
click.echo()
Expand Down Expand Up @@ -179,6 +184,29 @@ def _print_summary(summary: dict, region_selection_mode: str = None, multi_accou
for r in timed_out:
click.echo(f" [timeout] {r.account_name} ({r.account_id})")

# Azure multi-subscription breakdown
per_sub = summary.get("per_subscription")
if per_sub:
failed_subs = summary.get("subscriptions_failed", [])
click.echo()
click.echo(f"Subscriptions scanned: {len(per_sub) - len(failed_subs)}")
if failed_subs:
click.echo(f"Subscriptions failed: {len(failed_subs)}")
click.echo()
click.echo("Per-subscription breakdown:")
for r in per_sub:
cost = r.get("estimated_monthly_cost_usd", 0)
cost_str = f" ~${cost:,.0f}/month" if cost else ""
status = "" if r["status"] == "success" else f" [{r['status']}]"
click.echo(
f" {r['name']:<30} ({r['id']}):" f" {r['findings']} findings{cost_str}{status}"
)
if failed_subs:
click.echo()
click.echo("Failed subscriptions:")
for r in failed_subs:
click.echo(f" [failed] {r['name']} ({r['id']}): {r.get('error', '')}")

# Success message
if summary["total_findings"] == 0:
click.echo()
Expand Down
Loading
Loading