[REVIEW] iam-review: add effective privilege evidence matrix

[yunrongy424-oss]

Skill Being Reviewed

Skill name: iam-review
Skill path: skills/identity/iam-review/

False Positive Analysis

Benign configuration that can be over-reported without evidence source context:

{
  "Principal": {"AWS": "arn:aws:iam::123456789012:role/ci-deploy"},
  "Action": "sts:AssumeRole",
  "Condition": {
    "StringEquals": {"sts:ExternalId": "vendor-prod-2026"},
    "Bool": {"aws:MultiFactorAuthPresent": "true"}
  }
}

Why this is a false positive risk:
The skill correctly flags cross-account access and privileged roles, but it does not require a source/evidence matrix that distinguishes permanent admin, time-bound JIT, external-ID constrained federation, and observed activity. A cross-account role can be acceptable when constrained by external ID, MFA/conditional access, session duration, SCP/permission boundary, and owner evidence.

Coverage Gaps

Missed variant 1: Effective privilege differs from attached policy

Attached policy: AdministratorAccess
Permission boundary: denies iam:CreateUser and iam:PutRolePolicy
SCP: denies access outside approved regions

Why it should be caught:
The current skill can flag wildcard admin policies, but it does not force reviewers to document permission boundaries, SCPs, IAM conditions, or cloud-provider deny layers before assigning severity.

Missed variant 2: Last activity evidence is incomplete

User: build-bot
PasswordLastUsed: N/A
AccessKeyLastUsed: 2026-05-30
CloudTrail retention: 7 days

Why it should be caught:
Stale account conclusions depend on log retention, credential type, and provider-specific last-used semantics. The skill should require timestamp source and confidence instead of treating missing or N/A activity as proof of inactivity.

Missed variant 3: Conditional access policy exists but report-only

Policy: Require phishing-resistant MFA for admins
State: reportOnly
Scope: excludes break-glass and service principals

Why it should be caught:
The skill checks MFA and conditional access, but it should require enforcement mode and exclusion evidence. Report-only policies are not control enforcement.

Edge Cases

• Identity inventories often span IdP users, cloud IAM users, groups, roles, service accounts, app registrations, managed identities, workload identities, API keys, and external guests.
• Provider activity fields have different meanings and retention limits. Findings should state whether evidence comes from credential reports, sign-in logs, audit logs, Access Analyzer, IAM Recommender, or policy exports.
• Permission boundaries, SCPs, IAM Conditions, deny policies, and JIT/PIM may materially reduce effective privilege.
• Break-glass accounts can legitimately bypass some controls, but need owner, monitoring, test cadence, and alert evidence.

Remediation Quality

• Fix resolves the vulnerability
• Fix doesn't introduce new security issues
• Fix doesn't break functionality
• Issues found: The skill has strong IAM categories, but needs evidence confidence, effective permission context, enforcement-mode fields, and Not Evaluable reason codes so findings are reproducible and not purely checklist-driven.

Comparison to Other Tools

Tool	Catches this?	Notes
IAM Access Analyzer	Partial	Strong for AWS external access and policy validation, but not a full cross-provider inventory/evidence matrix.
Azure PIM / Entra reports	Partial	Strong for privileged role state, but needs report-only/enforced and exclusion context in a portable review.
GCP IAM Recommender / Policy Analyzer	Partial	Useful for recommendations and policy analysis, but still needs owner, JIT, condition, and log-retention context.
Semgrep / CodeQL	No	Not designed to evaluate effective IAM privilege across provider control planes.

Overall Assessment

Strengths:

• Broad, useful coverage of identity inventory, authentication, least privilege, service accounts, stale accounts, JIT, and zero trust.
• Good cross-provider examples for AWS, Azure/Entra ID, and GCP.
• Clear severity model and remediation priority matrix.

Needs improvement:

• Add an IAM evidence matrix for identity type, provider, owner, assigned privilege, effective privilege modifiers, MFA/conditional access state, activity source, last activity timestamp, log retention, and confidence.
• Add Not Evaluable reason codes for missing inventory, missing activity logs, unknown enforcement mode, unexpanded groups, unknown boundaries/SCPs/conditions, missing owner, and missing break-glass monitoring.
• Update output format to include confidence and evidence gaps for each finding and summary category.

Priority recommendations:

1. Add an "IAM Evidence and Effective Privilege Matrix" section.
2. Add confidence levels and Not Evaluable reason codes.
3. Add pitfalls for treating attached policy as effective privilege, treating N/A last-used as inactivity, and counting report-only conditional access as enforcement.

Bounty Info

• I have read and agree to the CONTRIBUTING.md bounty terms
• Preferred payment method: PayPal - 1005150221@qq.com

[yunrongy424-oss]

Improvement PR submitted for this review: #69

Preferred payment method remains PayPal: 1005150221@qq.com

[Alex-Parejo]

Implemented in #393