Merge pull request #269 from toniblyx/devel

New checks, documentation and fixes:
Added extra739 ELB logging and typos
Added extra740 EBS snapshots are encrypted and HIPAA
Added info about GDPR and HIPAA
Improved Prowler description
fixed issue #268

LIST_OF_CHECKS_AND_GROUPS.md

@@ -1,3 +1,4 @@
 ```
 ./prowler -l # to see all available checks and groups.
+./prowler -L # to see all available groups only.
 ```

README.md

@@ -11,14 +11,20 @@
 - [Troubleshooting](#troubleshooting)
 - [Extras](#extras)
 - [Forensics Ready Checks](#forensics-ready-checks)
+- [GDPR Checks](#gdpr-checks)
+- [HIPAA Checks](#hipaa-checks)
 - [Add Custom Checks](#add-custom-checks)
 - [Third Party Integrations](#third-party-integrations)
 - [Full list of checks and groups](/LIST_OF_CHECKS_AND_GROUPS.md)
 - [License](#license)
 
 ## Description
 
-Tool based on AWS-CLI commands for AWS account security assessment and hardening, following guidelines of the [CIS Amazon Web Services Foundations Benchmark v1.2.0 - 05-23-2018](https://d0.awsstatic.com/whitepapers/compliance/AWS_CIS_Foundations_Benchmark.pdf)
+Prowler is a command line tool for AWS Security Best Practices Assessment, Auditing, Hardening and Forensics Readiness Tool.
+
+It follows guidelines of the CIS Amazon Web Services Foundations Benchmark (49 checks) and has 40 additional checks including related to GDPR and HIPAA.
+
+Read more about [CIS Amazon Web Services Foundations Benchmark v1.2.0 - 05-23-2018](https://d0.awsstatic.com/whitepapers/compliance/AWS_CIS_Foundations_Benchmark.pdf)
 
 ## Features
 
@@ -30,10 +36,11 @@ It covers hardening and security best practices for all AWS regions related to t
 - Networking (4 checks) [group4]
 - CIS Level 1 [cislevel1]
 - CIS Level 2 [cislevel2]
-- Extras (37 checks) *see Extras section* [extras]
+- Extras (39 checks) *see Extras section* [extras]
 - Forensics related group of checks [forensics-ready]
-- GDPR [gdpr] Read more [here](https://github.com/toniblyx/prowler/issues/189)
-- HIPAA [hipaa]
+- GDPR [gdpr] Read more [here](#gdpr-checks)
+- HIPPA [hippa] Read more [here](#hipaa-checks)
+
 
 For a comprehensive list and resolution look at the guide on the link above.
 
@@ -419,6 +426,8 @@ At this moment we have 37 extra checks:
 - 7.36 (`extra736`) Check exposed KMS keys (Not Scored) (Not part of CIS benchmark)
 - 7.37 (`extra737`) Check KMS keys with key rotation disabled (Not Scored) (Not part of CIS benchmark)
 - 7.38 (`extra738`) Check if CloudFront distributions are set to HTTPS (Not Scored) (Not part of CIS benchmark)
+- 7.38 (`extra739`) Check if ELBs have logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.40 (`extra740`) Check if EBS snapshots are encrypted (Not Scored) (Not part of CIS benchmark)
 
 To check all extras in one command:
 
@@ -454,17 +463,90 @@ With this group of checks, Prowler looks if each service with logging or audit c
 - 7.20  Check if Lambda functions are being recorded by CloudTrail (Not Scored) (Not part of CIS benchmark)
 - 7.21  Check if Redshift cluster has audit logging enabled (Not Scored) (Not part of CIS benchmark)
 - 7.22  Check if API Gateway has logging enabled (Not Scored) (Not part of CIS benchmark)
-- 7.23 [extra723] Check if RDS Snapshots are public (Not Scored) (Not part of CIS benchmark)
-- 7.24 [extra724] Check if ACM certificates have Certificate Transparency logging enabled (Not Scored) (Not part of CIS benchmark)
-- 7.25 [extra725] Check if S3 buckets have Object-level logging enabled in CloudTrail (Not Scored) (Not part of CIS benchmark)
-
+- 7.23  Check if RDS Snapshots are public (Not Scored) (Not part of CIS benchmark)
+- 7.24  Check if ACM certificates have Certificate Transparency logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.25  Check if S3 buckets have Object-level logging enabled in CloudTrail (Not Scored) (Not part of CIS benchmark)
+- 7.38  Check if ELBs have logging enabled (Not Scored) (Not part of CIS benchmark)
 
 The `forensics-ready` group of checks uses existing and extra checks. To get a forensics readiness report, run this command:
 
 ```sh
 ./prowler -g forensics-ready
 ```
 
+## GDPR Checks
+
+With this group of checks, Prowler shows result of checks related to GDPR, more information [here](https://github.com/toniblyx/prowler/issues/189). The list of checks showed by this group is as follows:
+
+- 7.18 [extra718] Check if S3 buckets have server access logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.25 [extra725] Check if S3 buckets have Object-level logging enabled in CloudTrail (Not Scored) (Not part of CIS benchmark)
+- 7.27 [extra727] Check if SQS queues have policy set as Public (Not Scored) (Not part of CIS benchmark)
+- 1.2  [check12] Ensure multi-factor authentication (MFA) is enabled for all IAM users that have a console password (Scored)
+- 1.13  [check113] Ensure MFA is enabled for the root account (Scored)
+- 1.14  [check114] Ensure hardware MFA is enabled for the root account (Scored)
+- 7.1 [extra71] Ensure users with AdministratorAccess policy have MFA tokens enabled (Not Scored) (Not part of CIS benchmark)
+- 7.31 [extra731] Check if SNS topics have policy set as Public (Not Scored) (Not part of CIS benchmark)
+- 7.32 [extra732] Check if Geo restrictions are enabled in CloudFront distributions (Not Scored) (Not part of CIS benchmark)
+- 7.33 [extra733] Check if there are SAML Providers then STS can be used (Not Scored) (Not part of CIS benchmark)
+- 2.5  [check25] Ensure AWS Config is enabled in all regions (Scored)
+- 3.9  [check39] Ensure a log metric filter and alarm exist for AWS Config configuration changes (Scored)
+- 2.1  [check21] Ensure CloudTrail is enabled in all regions (Scored)
+- 2.2  [check22] Ensure CloudTrail log file validation is enabled (Scored)
+- 2.3  [check23] Ensure the S3 bucket CloudTrail logs to is not publicly accessible (Scored)
+- 2.4  [check24] Ensure CloudTrail trails are integrated with CloudWatch Logs (Scored)
+- 2.6  [check26] Ensure S3 bucket access logging is enabled on the CloudTrail S3 bucket (Scored)
+- 2.7  [check27] Ensure CloudTrail logs are encrypted at rest using KMS CMKs (Scored)
+- 3.5  [check35] Ensure a log metric filter and alarm exist for CloudTrail configuration changes (Scored)
+- 7.26 [extra726] Check Trusted Advisor for errors and warnings (Not Scored) (Not part of CIS benchmark)
+- 7.14 [extra714] Check if CloudFront distributions have logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.15 [extra715] Check if Elasticsearch Service domains have logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.17 [extra717] Check if Elastic Load Balancers have logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.19 [extra719] Check if Route53 hosted zones are logging queries to CloudWatch Logs (Not Scored) (Not part of CIS benchmark)
+- 7.20 [extra720] Check if Lambda functions invoke API operations are being recorded by CloudTrail (Not Scored) (Not part of CIS benchmark)
+- 7.21 [extra721] Check if Redshift cluster has audit logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.22 [extra722] Check if API Gateway has logging enabled (Not Scored) (Not part of CIS benchmark)
+- 4.3  [check43] Ensure the default security group of every VPC restricts all traffic (Scored)
+- 2.5  [check25] Ensure AWS Config is enabled in all regions (Scored)
+- 7.14 [extra714] Check if CloudFront distributions have logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.29 [extra729] Ensure there are no EBS Volumes unencrypted (Not Scored) (Not part of CIS benchmark)
+- 7.34 [extra734] Check if S3 buckets have default encryption (SSE) enabled and policy to enforce it (Not Scored) (Not part of CIS benchmark)
+- 7.35 [extra735] Check if RDS instances storage is encrypted (Not Scored) (Not part of CIS benchmark)
+- 7.36 [extra736] Check exposed KMS keys (Not Scored) (Not part of CIS benchmark)
+- 7.38 [extra738] Check if CloudFront distributions are set to HTTPS (Not Scored) (Not part of CIS benchmark)
+
+The `gdpr` group of checks uses existing and extra checks. To get a GDPR report, run this command:
+
+```sh
+./prowler -g gdpr
+```
+
+## HIPAA Checks
+
+With this group of checks, Prowler shows result of checks related to HIPAA, more information [here](https://github.com/toniblyx/prowler/issues/227). The list of checks showed by this group is as follows:
+
+- 1.13  [check113] Ensure MFA is enabled for the root account (Scored)
+- 2.3  [check23] Ensure the S3 bucket CloudTrail logs to is not publicly accessible (Scored)
+- 2.6  [check26] Ensure S3 bucket access logging is enabled on the CloudTrail S3 bucket (Scored)
+- 2.7  [check27] Ensure CloudTrail logs are encrypted at rest using KMS CMKs (Scored)
+- 2.9  [check29] Ensure VPC Flow Logging is Enabled in all VPCs (Scored)
+- 7.18 [extra718] Check if S3 buckets have server access logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.25 [extra725] Check if S3 buckets have Object-level logging enabled in CloudTrail (Not Scored) (Not part of CIS benchmark)
+- 7.2 [extra72] Ensure there are no EBS Snapshots set as Public (Not Scored) (Not part of CIS benchmark)
+- 7.5 [extra75] Ensure there are no Security Groups not being used (Not Scored) (Not part of CIS benchmark)
+- 7.39 [extra739] Check if ELBs have logging enabled (Not Scored) (Not part of CIS benchmark)
+- 7.29 [extra729] Ensure there are no EBS Volumes unencrypted (Not Scored) (Not part of CIS benchmark)
+- 7.34 [extra734] Check if S3 buckets have default encryption (SSE) enabled and policy to enforce it (Not Scored) (Not part of CIS benchmark)
+- 3.8  [check38] Ensure a log metric filter and alarm exist for S3 bucket policy changes (Scored)
+- 7.3 [extra73] Ensure there are no S3 buckets open to the Everyone or Any AWS user (Not Scored) (Not part of CIS benchmark)
+- 7.40 [extra740] Check if EBS snapshots are encrypted (Not Scored) (Not part of CIS benchmark)
+- 7.35 [extra735] Check if RDS instances storage is encrypted (Not Scored) (Not part of CIS benchmark)
+
+The `hipaa` group of checks uses existing and extra checks. To get a HIPAA report, run this command:
+
+```sh
+./prowler -g hipaa
+```
+
 ## Add Custom Checks
 
 In order to add any new check feel free to create a new extra check in the extras group or other group. To do so, you will need to follow these steps:
@@ -501,8 +583,8 @@ The link to the license terms can be found at
 Any other piece of code is licensed as Apache License 2.0 as specified in each file. You may obtain a copy of the License at
 <http://www.apache.org/licenses/LICENSE-2.0>
 
-NOTE: If you are interested in using Prowler for commercial purposes remember that due to the CC4.0 license “The distributors or partners that are interested and using Prowler would need to enrol as CIS SecureSuite Members to incorporate this product, which includes references to CIS resources, in their offering.". Information about CIS pricing for vendors here: <https://www.cisecurity.org/cis-securesuite/pricing-and-categories/product-vendor/>
+NOTE: If you are interested in using Prowler for commercial purposes remember that due to the CC4.0 license “The distributors or partners that are interested and using Prowler would need to enroll as CIS SecureSuite Members to incorporate this product, which includes references to CIS resources, in their offering.". Information about CIS pricing for vendors here: <https://www.cisecurity.org/cis-securesuite/pricing-and-categories/product-vendor/>
 
-**I'm not related anyhow with CIS organisation, I just write and maintain Prowler to help companies over the world to make their cloud infrastructure more secure.**
+**I'm not related anyhow with CIS organization, I just write and maintain Prowler to help companies over the world to make their cloud infrastructure more secure.**
 
 If you want to contact me visit <https://blyx.com/contact>

checks/check_extra73

@@ -18,6 +18,30 @@ CHECK_ALTERNATE_extra703="extra73"
 CHECK_ALTERNATE_check73="extra73"
 CHECK_ALTERNATE_check703="extra73"
 
+# Improved and simplified check on Nov 18th 2018 due to a new bucket attribute
+# called PolicyStatus, not available in all regions yet.
+
+# extra73(){
+#   ALL_BUCKETS_LIST=$($AWSCLI s3api list-buckets --query 'Buckets[*].{Name:Name}' $PROFILE_OPT --region $REGION --output text)
+#   for bucket in $ALL_BUCKETS_LIST; do
+#     BUCKET_LOCATION=$($AWSCLI s3api get-bucket-location --bucket $bucket $PROFILE_OPT --region $REGION --output text)
+#     if [[ "None" == $BUCKET_LOCATION ]]; then
+#       BUCKET_LOCATION="us-east-1"
+#     fi
+#     if [[ "EU" == $BUCKET_LOCATION ]]; then
+#       BUCKET_LOCATION="eu-west-1"
+#     fi
+#
+#     BUCKET_POLICY_STATUS=$($AWSCLI s3api get-bucket-policy-status --bucket $bucket --query PolicyStatus.IsPublic --output text | grep False)
+#     if [[ $BUCKET_POLICY_STATUS ]];then
+#       textFail "$BUCKET_LOCATION: $bucket bucket is Public!" "$regx"
+#     else
+#       textPass "$BUCKET_LOCATION: $bucket bucket is not Public" "$regx"
+#     fi
+#   done
+# }
+
+
 extra73(){
   textInfo "Looking for open S3 Buckets (ACLs and Policies) in all regions...  "
   ALL_BUCKETS_LIST=$($AWSCLI s3api list-buckets --query 'Buckets[*].{Name:Name}' $PROFILE_OPT --region $REGION --output text)

checks/check_extra739

@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+
+# Prowler - the handy cloud security tool (copyright 2018) by Toni de la Fuente
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may not
+# use this file except in compliance with the License. You may obtain a copy
+# of the License at http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed
+# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+# CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations under the License.
+CHECK_ID_extra739="7.39"
+CHECK_TITLE_extra739="[extra739] Check if ELBs have logging enabled (Not Scored) (Not part of CIS benchmark)"
+CHECK_SCORED_extra739="NOT_SCORED"
+CHECK_TYPE_extra739="EXTRA"
+CHECK_ALTERNATE_check739="extra739"
+
+extra739(){
+  for regx in $REGIONS; do
+    LIST_OF_ELB=$($AWSCLI elb describe-load-balancers --region $regx $PROFILE_OPT --query LoadBalancerDescriptions[*].LoadBalancerName --output text)
+    if [[ $LIST_OF_ELB ]];then
+      for elb_id in $LIST_OF_ELB; do
+        CHECK_LOG_STATUS=$($AWSCLI elb describe-load-balancer-attributes --region $regx $PROFILE_OPT --load-balancer-name $elb_id --query LoadBalancerAttributes.AccessLog.Enabled --output text|grep False)
+        if [[ $CHECK_LOG_STATUS ]]; then
+          textFail "$regx: ELB $elb_id has login disabled!" "$regx"
+        else
+          textPass "$regx: ELB $elb_id has login enabled" "$regx"
+        fi
+      done
+    else
+      textInfo "$regx: No ELBs found" "$regx"
+    fi
+  done
+}

checks/check_extra740

@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+
+# Prowler - the handy cloud security tool (copyright 2018) by Toni de la Fuente
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may not
+# use this file except in compliance with the License. You may obtain a copy
+# of the License at http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed
+# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+# CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations under the License.
+CHECK_ID_extra740="7.40"
+CHECK_TITLE_extra740="[extra740] Check if EBS snapshots are encrypted (Not Scored) (Not part of CIS benchmark)"
+CHECK_SCORED_extra740="NOT_SCORED"
+CHECK_TYPE_extra740="EXTRA"
+CHECK_ALTERNATE_check740="extra740"
+
+extra740(){
+  textInfo "Looking for EBS Snapshots in all regions...  "
+  for regx in $REGIONS; do
+    LIST_OF_EBS_SNAPSHOTS=$($AWSCLI ec2 describe-snapshots $PROFILE_OPT --region $regx --owner-ids $ACCOUNT_NUM --output text --query 'Snapshots[*].{ID:SnapshotId}' --max-items $MAXITEMS | grep -v None 2> /dev/null)
+    if [[ $LIST_OF_EBS_SNAPSHOTS ]];then
+      for snapshot in $LIST_OF_EBS_SNAPSHOTS; do
+        SNAPSHOT_IS_ENCRYPTED=$($AWSCLI ec2 describe-snapshots $PROFILE_OPT --region $regx --output text --snapshot-id $snapshot --query Snapshots[*].Encrypted|grep False)
+        if [[ $SNAPSHOT_IS_ENCRYPTED ]];then
+          textFail "$regx: $snapshot is currently not encrypted!" "$regx"
+        else
+          textPass "$regx: $snapshot is encrypted" "$regx"
+        fi
+      done
+    else
+      textInfo "$regx: No EBS Snapshots found" "$regx" 
+    fi
+  done
+}

groups/group10_hipaa

@@ -13,9 +13,9 @@
 
 GROUP_ID[10]='hipaa'
 GROUP_NUMBER[10]='10.0'
-GROUP_TITLE[10]='HIPAA Compliance - WORK IN PROGRESS!! - [hipaa] *******************'
+GROUP_TITLE[10]='HIPAA Compliance - ONLY AS REFERENCE - [hipaa] ****************'
 GROUP_RUN_BY_DEFAULT[10]='N' # run it when execute_all is called
-GROUP_CHECKS[10]='check12,check113,check23,check26,check27,check29,extra718,extra725,extra72,extra75'
+GROUP_CHECKS[10]='check12,check113,check23,check26,check27,check29,extra718,extra725,extra72,extra75,extra739,extra729,extra734,check38,extra73,extra740,extra735'
 
 # Resources:
 # https://d0.awsstatic.com/whitepapers/compliance/AWS_HIPAA_Compliance_Whitepaper.pdf

groups/group5_cislevel1

@@ -12,4 +12,4 @@ GROUP_ID[5]='cislevel1'
 GROUP_NUMBER[5]='5.0'
 GROUP_TITLE[5]='CIS Level 1 - [cislevel1] **************************************'
 GROUP_RUN_BY_DEFAULT[5]='N' # run it when execute_all is called
-GROUP_CHECKS[5]='check11,check12,check13,check14,check15,check16,check17,check18,check19,check110,check111,check112,check113,check115,check116,check117,check118,check119,check120,check122,check21,check23,check24,check25,check26,check31,check32,check33,check34,check35,check38,check312,check313,check314,check315,check41,check42'
+GROUP_CHECKS[5]='check11,check12,check13,check14,check15,check16,check17,check18,check19,check110,check111,check112,check113,check115,check116,check117,check118,check119,check120,check122,check21,check23,check24,check25,check26,check31,check32,check33,check34,check35,check38,check312,check313,check314,check41,check42'

groups/group8_forensics

@@ -15,4 +15,4 @@ GROUP_ID[8]='forensics-ready'
 GROUP_NUMBER[8]='8.0'
 GROUP_TITLE[8]='Forensics Readiness - [forensics-ready] ************************'
 GROUP_RUN_BY_DEFAULT[8]='N' # run it when execute_all is called
-GROUP_CHECKS[8]='check21,check22,check23,check24,check25,check26,check27,check29,extra712,extra713,extra714,extra715,extra717,extra718,extra719,extra720,extra721,extra722,extra725'
+GROUP_CHECKS[8]='check21,check22,check23,check24,check25,check26,check27,check29,extra712,extra713,extra714,extra715,extra717,extra718,extra719,extra720,extra721,extra722,extra725,extra739'

groups/group9_gdpr

@@ -13,7 +13,7 @@
 
 GROUP_ID[9]='gdpr'
 GROUP_NUMBER[9]='9.0'
-GROUP_TITLE[9]='GDPR Readiness - WORK IN PROGRESS!! - [gdpr] *******************'
+GROUP_TITLE[9]='GDPR Readiness - ONLY AS REFERENCE - [gdpr] ********************'
 GROUP_RUN_BY_DEFAULT[9]='N' # run it when execute_all is called
 GROUP_CHECKS[9]='extra718,extra725,extra727,check12,check113,check114,extra71,extra731,extra732,extra733,check25,check39,check21,check22,check23,check24,check26,check27,check35,extra726,extra714,extra715,extra717,extra719,extra720,extra721,extra722,check43,check25,extra714,extra729,extra734,extra735,extra736,extra738'