Skip to content

[REVIEW] aws-review: refresh CIS AWS v5 control mapping #213

@mlodygbelmondo

Description

@mlodygbelmondo

Skill Being Reviewed

Skill name: aws-review
Skill path: skills/cloud/aws-review/

False Positive Analysis

Benign AWS posture report that can be over-reported under the old baseline:

Framework: CIS Amazon Web Services Foundations Benchmark v3.0.0
Total CIS recommendations evaluated: 62/62
Section scores:
  1 Identity and Access Management: 20/22
  2 Storage: 9/10
  3 Logging: 10/11
  4 Monitoring: 15/16
  5 Networking: 5/6
Overall compliance: 95%

Why this can be misleading:

The skill is hard-coded to CIS AWS Foundations Benchmark v3.0.0 and a fixed 62 recommendation denominator. AWS Security Hub CSPM now supports CIS AWS Foundations Benchmark v5.0.0 and recommends using v5.0.0 to stay current. AWS also publishes a version comparison showing different control-to-requirement mappings across v5.0.0, v3.0.0, v1.4.0, and v1.2.0.

A report generated today should not present v3.0.0 recommendation counts and mappings as current CIS AWS coverage unless legacy v3.0.0 was explicitly requested. Otherwise a team may get stale compliance percentages or miss controls that exist in v5.0.0, such as the v5 EFS encryption-at-rest control listed in AWS Security Hub CSPM.

Coverage Gaps

Missed variant 1: the benchmark version and denominator are pinned

Framework: CIS Amazon Web Services Foundations Benchmark v3.0.0
Total CIS recommendations evaluated: <N>/62

The output should record benchmark_version, benchmark_source_date, security_hub_standard_arn_or_version, and legacy_baseline. The denominator should come from the selected benchmark/control mapping, not from the skill's v3.0.0 constant.

Missed variant 2: v5-only or remapped controls can be omitted from current scoring

Skill section map:
  2 Storage: 10 recommendations

AWS's v5.0.0 control list includes current supported controls and the version-comparison table maps controls differently across benchmark versions. If the skill only walks the v3.0.0 section map, it can omit or mislabel current v5.0.0 controls and produce a compliance score that does not match Security Hub CSPM's current CIS standard.

Missed variant 3: removed or unsupported requirements can be reported as current failures

Finding: CIS 3.3 - CloudTrail log bucket is publicly accessible
Status: Fail under current benchmark

AWS's version-comparison guidance includes cases where controls are not supported for later versions because CIS removed a requirement. The skill should distinguish:

  • current v5.0.0 supported control;
  • legacy v3.0.0/v1.x control;
  • requirement removed by CIS;
  • not supported by Security Hub CSPM; and
  • custom/manual evidence outside Security Hub coverage.

Edge Cases

  • Some organizations may intentionally keep v3.0.0 enabled during migration. That should be explicit legacy mode, not the default current posture claim.
  • Security Hub CSPM can enable multiple CIS benchmark versions simultaneously, so output should identify which standard produced each finding.
  • IaC-only evidence may not prove every Security Hub CSPM control. The skill should keep not evaluable from IaC separate from Security Hub pass/fail evidence.
  • Existing issue [REVIEW] aws-review: add organization-wide evidence scope #190 covers AWS Organizations scope evidence. This review is specifically about benchmark-version freshness and current Security Hub control mapping.

Remediation Quality

  • Fix resolves the vulnerability
  • Fix doesn't introduce new security issues
  • Fix doesn't break functionality
  • Issues found: Refresh aws-review to support CIS AWS Foundations v5.0.0-aware output, add version/source/standard fields, remove the hard-coded v3.0.0 denominator as the default, and distinguish current, legacy, removed, unsupported, and not-evaluable controls.

Comparison to Other Tools

Tool Catches this? Notes
AWS Security Hub CSPM CIS docs Yes Supports CIS AWS Foundations v5.0.0 and provides version comparison/mapping.
AWS Security Hub standard ARN/version Partial Identifies enabled CIS version, but the skill must map review evidence to it.
Current skill output No Hard-codes v3.0.0, five sections, and 62 recommendations.

Overall Assessment

Strengths:

  • Good AWS IaC discovery coverage across Terraform, CloudFormation, CDK, IAM policies, Security Hub, and AWS Config files.
  • Useful severity calibration and AWS-specific pitfalls.
  • Good handling for not-evaluable controls in principle.

Needs improvement:

  • Update the skill from v3.0.0-only CIS AWS assessment to v5.0.0-aware benchmark handling.
  • Add explicit benchmark version, source date, and Security Hub standard version fields.
  • Avoid hard-coded v3.0.0 section denominators in current reports.
  • Add control status categories for current, legacy, removed, unsupported, manual, and not evaluable from supplied evidence.

Priority recommendations:

  1. Add a preflight step requiring requested CIS AWS benchmark version and whether Security Hub CSPM evidence is present.
  2. Default current reports to v5.0.0-aware mapping, preserving v3.0.0 as explicit legacy mode.
  3. Replace <N>/62 and fixed section counts with denominator values derived from the selected version.
  4. Add a control-mapping table with control_id, cis_v5_requirement, cis_v3_requirement, evidence_source, support_status, and assessment_status.

References

Bounty Info

  • I have read and agree to the CONTRIBUTING.md bounty terms.
  • Preferred payment method: To be provided privately after acceptance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions