Summary Junit Reporting

[stevecjones]

When using the junit outputs tests get recorded correctly however there is a slight issue with the following line.

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="3.10" timestamp="2020-09-03T12:17:00Z"> <----- THIS ONE
  <properties>
    <property name="prowler.version" value="2.3.0RC3"/>
    <property name="aws.profile" value="splunk"/>
    <property name="aws.accountNumber" value=""/>
    <property name="check.id" value="3.10"/>
    <property name="check.scored" value="Scored"/>
    <property name="check.level" value="Level 2"/>
    <property name="check.asff.type" value="Software and Configuration Checks/Industry and Regulatory Standards/CIS AWS Foundations Benchmark"/>
    <property name="check.asff.resourceType" value="AwsCloudTrailTrail"/>
  </properties>
  <testcase name="[check310] Ensure a log metric filter and alarm exist for security group changes (Scored) (1)" classname="3.10" time="3.000">
    <failure message="No CloudWatch group found for CloudTrail events"/>
  </testcase>
</testsuite>

When passing the output in tools like Azure Devops or AWS CodeBuild the engine expects a summary of all tests within the junit file like the following:-

<testsuite name="7.31" tests="4" failures="0" skipped="0" errors="0" timestamp="2020-10-21T15:11:59Z">

I have created a PR that addresses the creation of this line after enumerating the test cases within. So in the case of AWS CodeBuild the following graph should then populate.

Sorry I didn't raise this issue first. #671

Thanks!

[toniblyx]

Ok, I responded in the PR to see if you can give me some quick instructions to reproduce it and test it.

[stevecjones]

I can go one better and provide the cloudformation to create the codebuild project to audit the AWS account 👍

---
AWSTemplateFormatVersion: 2010-09-09
Description: Creates a CodeBuild project to audit the AWS account with prowler https://github.com/toniblyx/prowler

Parameters:
  ServiceName:
    Description: 'Specifies the service name used within component naming'
    Type: String
    Default: 'prowler'

  LogsRetentionInDays:
    Description: 'Specifies the number of days you want to retain codebuild run log events in the specified log group. Junit reports are kept for 30 days'
    Type: Number
    Default: 3
    AllowedValues: [1, 3, 5, 7, 14, 30, 60]

Resources:
  ArtifactBucket:
    Type: AWS::S3::Bucket
    Properties:
      Tags:
        - Key: Name
          Value: !Join ['-', ['AP2', 'INF', !Ref 'ServiceName', !Ref 'AWS::AccountId', 'S3', 'Prowler']]
      BucketName: !Sub '${ServiceName}-${AWS::Region}-prowler-${AWS::AccountId}'
      AccessControl: LogDeliveryWrite
      VersioningConfiguration:
        Status: Enabled
      LoggingConfiguration:
        DestinationBucketName: !ImportValue 'ProviderLogBucket'
        LogFilePrefix: !Sub '${ServiceName}-${AWS::Region}-prowler-${AWS::AccountId}/'
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true

  ArtifactBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref 'ArtifactBucket'
      PolicyDocument:
        Id: Content
        Version: '2012-10-17'
        Statement:
          - Action: '*'
            Condition:
              Bool:
                aws:SecureTransport: 'false'
            Effect: Deny
            Principal: '*'
            Resource:
              - !Join ['', ['arn:aws:s3:::', !Ref 'ArtifactBucket', '/*']]
            Sid: S3ForceSSL
          - Action: 's3:PutObject'
            Condition:
              'Null':
                s3:x-amz-server-side-encryption: 'true'
            Effect: Deny
            Principal: '*'
            Resource:
              - !Join ['', ['arn:aws:s3:::', !Ref 'ArtifactBucket', '/*']]
            Sid: DenyUnEncryptedObjectUploads

  # Codebuild Projects
  CodeBuildServiceRole:
    Type: AWS::IAM::Role
    Metadata:
      cfn_nag:
        rules_to_suppress:
          - id: W28
            reason: "Explicit name is required for this resource to avoid circular dependencies."
    Properties:
      RoleName: prowler-codebuild-role
      Path: '/service-role/'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/job-function/SupportUser'
        - 'arn:aws:iam::aws:policy/job-function/ViewOnlyAccess'
        - 'arn:aws:iam::aws:policy/SecurityAudit'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          -
            Action: 'sts:AssumeRole'
            Effect: Allow
            Principal:
              Service:
                - codebuild.amazonaws.com
      Policies:
        - PolicyName: LogGroup
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Effect: Allow
                Resource: !Sub 'arn:aws:logs:ap-southeast-2:${AWS::AccountId}:log-group:/aws/codebuild/*'
        - PolicyName: S3
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                  - s3:PutObject
                  - s3:GetObject
                  - s3:GetObjectVersion
                  - s3:GetBucketAcl
                  - s3:GetBucketLocation
                Effect: Allow
                Resource: !Sub 'arn:aws:s3:::${ArtifactBucket}/*'
        - PolicyName: CodeBuild
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                  - codebuild:CreateReportGroup
                  - codebuild:CreateReport
                  - codebuild:UpdateReport
                  - codebuild:BatchPutTestCases
                  - codebuild:BatchPutCodeCoverages
                Effect: Allow
                Resource: !Sub 'arn:aws:codebuild:ap-southeast-2:${AWS::AccountId}:report-group/*'
        - PolicyName: AssumeRole
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                  - sts:AssumeRole
                Effect: Allow
                Resource: !Sub 'arn:aws:iam::${AWS::AccountId}:role/service-role/prowler-codebuild-role'

  ProwlerCodeBuild:
    Type: AWS::CodeBuild::Project
    Properties:
      Artifacts:
        Type: NO_ARTIFACTS
      Source:
        Type: NO_SOURCE
        BuildSpec: |
          version: 0.2
          phases:
            install:
              runtime-versions:
                python: 3.8
              commands:
                - pip3 install detect-secrets
                - curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
                - unzip awscliv2.zip
                - ./aws/install
                - git clone https://github.com/toniblyx/prowler
            build:
              commands:
                - cd prowler
                - ./prowler -r ap-southeast-2 -f ap-southeast-2 -M junit-xml -g cislevel1 -E check24,check12,check120,check113
          reports:
            prowler:
              files:
                - '**/*'
              base-directory: 'prowler/junit-reports'
              file-format: JunitXml
      Environment:
        ComputeType: "BUILD_GENERAL1_SMALL"
        Image: "aws/codebuild/amazonlinux2-x86_64-standard:3.0"
        Type: "LINUX_CONTAINER"
      Description: Run Prowler audit for the SMC
      ServiceRole: !GetAtt CodeBuildServiceRole.Arn
      TimeoutInMinutes: 300

  ProwlerCodeBuildReportGroup:
    Type: AWS::CodeBuild::ReportGroup
    Properties:
      Name: prowler
      Type: TEST
      ExportConfig:
        ExportConfigType: NO_EXPORT

  ProwlerLogGroup:
    Type: 'AWS::Logs::LogGroup'
    Properties:
      LogGroupName: !Sub '/aws/codebuild/${ProwlerCodeBuild}'
      RetentionInDays: !Ref LogsRetentionInDays

Outputs:
  ArtifactBucketName:
    Description: Artifact Bucket Name
    Value: !Ref 'ArtifactBucket'
    Export:
      Name: !Sub 'ArtifactBucketName-${ServiceName}'

[toniblyx]

This is awesome! Thanks @stevecjones

[toniblyx]

Fixed in #671