Skip to content

Commit

Permalink
update to v1.5.0
Browse files Browse the repository at this point in the history
  • Loading branch information
gsingh04 committed Jun 2, 2022
1 parent 8dbc803 commit 8cfa35d
Show file tree
Hide file tree
Showing 180 changed files with 21,011 additions and 7,613 deletions.
Binary file added .DS_Store
Binary file not shown.
14 changes: 11 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,20 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.5.0] - 2022-05-31

### Added
- New remediations - see Implementation Guide

### Changed
- Improved cross-region remediation using resource region from Resources[0].Id
- Added custom resource provider for SSM documents to allow in-place stack upgrades

## [1.4.2] - 2022-01-14

### Changed
- Fix to correct the generator id pattern for CIS 1.2.0 Ruleset.


## [1.4.1] - 2022-01-05

### Changed
Expand Down Expand Up @@ -52,14 +60,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- New PCI-DSS v3.2.1 Playbook support for 17 controls (see IG for details)
- Library of remediation SSM Automation runbooks
- NEWPLAYBOOK as a template for custom playbook creation

### Changed
- Updated to CDK v1.117.0
- Reduced duplicate code
- Updated CIS playbook to Orchestrator architecture
- Single Orchestrator deployment to enable multi-standard remediation with a single click
- Custom Actions now consolidated to one: "Remediate with SHARR"

### Removed
- AWS Service Catalog for Playbook deployment

Expand Down
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,11 @@ Detailed instructions for creating a new automated remediation in an existing Pl

### Prerequisites for Customization

- a Linux client with the AWS CLI v2 installed and python 3.7+, AWS CDK 1.114.0+
- a Linux client with the following software
- AWS CLI v2
- Python 3.7+ with pip
- AWS CDK 1.155.0+
- Node.js with npm
- source code downloaded from GitHub
- two S3 buckets (minimum): 1 global and 1 for each region where you will deploy
- An Amazon S3 Bucket for solution templates - accessed globally via https.
Expand Down Expand Up @@ -179,6 +183,10 @@ build-s3-dist.sh -b <bucketname> -v <version>

#### Run Unit Tests

Some Python unit tests execute AWS API calls. The calls that create, read, or modify resources are stubbed, but some
calls to APIs that do not require any permissions execute against the real AWS APIs (e.g. STS GetCallerIdentity). The
recommended way to run the unit tests is to configure your credentials for a no-access console role.

```bash
cd ./deployment
chmod +x ./run-unit-tests.sh
Expand Down
41 changes: 28 additions & 13 deletions deployment/build-s3-dist.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@

# Important: CDK global version number
# This controls the CDK and AWS Solutions Constructs version. Solutions
# Constructs versions map 1:1 to CDK versions. When setting this value,
# Constructs versions map 1:1 to CDK versions. When setting this value,
# choose the latest AWS Solutions Constructs version.
required_cdk_version=1.132.0
required_cdk_version=1.155.0

# Get reference for all important folders
template_dir="$PWD"
Expand Down Expand Up @@ -74,25 +74,25 @@ do
b ) bucket=${OPTARG};;
v ) version=${OPTARG};;
t ) devtest=1;;
c)
c)
clean
exit 0
;;
*)
echo "Usage: $0 -b <bucket> [-v <version>] [-t]"
echo "Version must be provided via a parameter or ../version.txt. Others are optional."
echo "Version must be provided via a parameter or ../version.txt. Others are optional."
echo "-t indicates this is a pre-prod build and instructs the build to use a non-prod Solution ID, DEV-SOxxxx"
echo "Production example: ./build-s3-dist.sh -b solutions -v v1.0.0"
echo "Dev example: ./build-s3-dist.sh -b solutions -v v1.0.0 -t"
exit 1
echo "Dev example: ./build-s3-dist.sh -b solutions -v v1.0.0 -t"
exit 1
;;
esac
done

#------------------------------------------------------------------------------
# DISABLE OVERRIDE WARNINGS
#------------------------------------------------------------------------------
# Use with care: disables the warning for overridden properties on
# Use with care: disables the warning for overridden properties on
# AWS Solutions Constructs
export overrideWarningsEnabled=false

Expand Down Expand Up @@ -127,7 +127,7 @@ echo "export DIST_VERSION=$version" >> ./setenv.sh
#
# It takes precedence over the command line (oddly backwards, but to prevent
# errors)
#
#
# Ex:
# #!/bin/bash
# SOLUTION_ID='SO0111'
Expand Down Expand Up @@ -162,7 +162,7 @@ fi
if [[ -z "$SOLUTION_TRADEMARKEDNAME" ]]; then
echo "SOLUTION_TRADEMARKEDNAME is missing from ../solution_env.sh"
exit 1
else
else
export SOLUTION_TRADEMARKEDNAME
echo "export DIST_SOLUTION_NAME=$SOLUTION_TRADEMARKEDNAME" >> ./setenv.sh
fi
Expand Down Expand Up @@ -196,7 +196,7 @@ export PATH=$(npm bin):$PATH
# Check cdk version
cdkver=`cdk --version | grep -Eo '^[0-9]{1,2}\.[0-9]+\.[0-9]+'`
echo CDK version $cdkver
if [[ $cdkver != $required_cdk_version ]]; then
if [[ $cdkver != $required_cdk_version ]]; then
echo Required CDK version is $required_cdk_version, found $cdkver
exit 255
fi
Expand All @@ -215,10 +215,19 @@ find . -name package-lock.json | while read file;do rm $file; done

mkdir -p $temp_work_dir/source/solution_deploy/lambdalayer/python
cp ${template_dir}/${source_dir}/LambdaLayers/*.py $temp_work_dir/source/solution_deploy/lambdalayer/python
pip install -r $template_dir/requirements.txt -t $temp_work_dir/source/solution_deploy/lambdalayer/python
do_cmd pip install -r $template_dir/requirements.txt -t $temp_work_dir/source/solution_deploy/lambdalayer/python
cd $temp_work_dir/source/solution_deploy/lambdalayer
zip --recurse-paths ${build_dist_dir}/lambda/layer.zip python

echo "------------------------------------------------------------------------------"
echo "[Pack] Member Stack Lambda Layer (used by custom resources)"
echo "------------------------------------------------------------------------------"
do_cmd mkdir -p $temp_work_dir/source/solution_deploy/memberlambdalayer/python
do_cmd cp ${template_dir}/${source_dir}/LambdaLayers/cfnresponse.py $temp_work_dir/source/solution_deploy/memberlambdalayer/python
do_cmd cp ${template_dir}/${source_dir}/LambdaLayers/logger.py $temp_work_dir/source/solution_deploy/memberlambdalayer/python
do_cmd cd $temp_work_dir/source/solution_deploy/memberlambdalayer
do_cmd zip --recurse-paths ${build_dist_dir}/lambda/memberLayer.zip python

echo "------------------------------------------------------------------------------"
echo "[Pack] Custom Action Lambda"
echo "------------------------------------------------------------------------------"
Expand All @@ -229,12 +238,18 @@ zip ${build_dist_dir}/lambda/createCustomAction.py.zip createCustomAction.py
# These are not packaged with the Lambda
do_cmd cp ../../LambdaLayers/*.py .

echo "------------------------------------------------------------------------------"
echo "[Pack] Updatable Runbook Provider Lambda"
echo "------------------------------------------------------------------------------"
do_cmd cd $temp_work_dir/source/solution_deploy/source
do_cmd zip ${build_dist_dir}/lambda/updatableRunbookProvider.py.zip updatableRunbookProvider.py

echo "------------------------------------------------------------------------------"
echo "[Pack] Orchestrator Lambdas"
echo "------------------------------------------------------------------------------"
# cd $template_dir
cd $temp_work_dir/source/Orchestrator
ls | while read file; do
ls | while read file; do
if [ ! -d $file ]; then
zip ${build_dist_dir}/lambda/${file}.zip ${file}
fi
Expand All @@ -247,7 +262,7 @@ echo "--------------------------------------------------------------------------
echo "[Create] Playbooks"
echo "------------------------------------------------------------------------------"
for playbook in `ls ${template_dir}/${source_dir}/playbooks`; do
if [ $playbook == 'NEWPLAYBOOK' ] || [ $playbook == '.coverage' ]; then
if [ $playbook == 'NEWPLAYBOOK' ] || [ $playbook == '.coverage' ] || [ $playbook == 'common' ]; then
continue
fi
echo Create $playbook playbook
Expand Down
2 changes: 1 addition & 1 deletion deployment/requirements.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
requests>=2.22.0
requests>=2.25.0
14 changes: 9 additions & 5 deletions deployment/run-unit-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -115,13 +115,17 @@ echo "--------------------------------------------------------------------------
run_pytest "${source_dir}/remediation_runbooks/scripts" "RemediationRunbooks"

echo "------------------------------------------------------------------------------"
echo "[Test] Python Scripts for Playbook Scripts"
echo "[Test] Python Scripts for Playbook common scripts"
echo "------------------------------------------------------------------------------"
run_pytest "${source_dir}/playbooks/common" "PlaybookCommon"

echo "------------------------------------------------------------------------------"
echo "[Test] Python Scripts for Playbooks"
echo "------------------------------------------------------------------------------"
for playbook in `ls ${source_dir}/playbooks`; do
# if [ $playbook == 'NEWPLAYBOOK' ]; then
# continue
# fi
run_pytest "${source_dir}/playbooks/${playbook}/ssmdocs/scripts" "Playbook${playbook}"
if [ -d ${source_dir}/playbooks/${playbook}/ssmdocs/scripts/tests ]; then
run_pytest "${source_dir}/playbooks/${playbook}/ssmdocs/scripts" "Playbook${playbook}"
fi
done

# The pytest --cov with its parameters and .coveragerc generates a xml cov-report with `coverage/sources` list
Expand Down
2 changes: 2 additions & 0 deletions deployment/testing_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,5 @@ pytest>=4.2.1
pytest-cov
pytest-env
bandit
boto3==1.23.9
requests==2.27.1
Binary file added source/.DS_Store
Binary file not shown.
Binary file added source/LambdaLayers/.DS_Store
Binary file not shown.
44 changes: 44 additions & 0 deletions source/LambdaLayers/cfnresponse.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0

from __future__ import print_function
import urllib3
import json

SUCCESS = "SUCCESS"
FAILED = "FAILED"

http = urllib3.PoolManager()

def send(event, context, responseStatus, responseData, physicalResourceId=None, noEcho=False, reason=None):
responseUrl = event['ResponseURL']

print(responseUrl)

responseBody = {
'Status' : responseStatus,
'Reason' : reason or "See the details in CloudWatch Log Stream: {}".format(context.log_stream_name),
'PhysicalResourceId' : physicalResourceId or context.log_stream_name,
'StackId' : event['StackId'],
'RequestId' : event['RequestId'],
'LogicalResourceId' : event['LogicalResourceId'],
'NoEcho' : noEcho,
'Data' : responseData
}

json_responseBody = json.dumps(responseBody)

print("Response body:")
print(json_responseBody)

headers = {
'content-type' : '',
'content-length' : str(len(json_responseBody))
}

try:
response = http.request('PUT', responseUrl, headers=headers, body=json_responseBody)
print("Status code:", response.status)

except Exception as e:
print("send(..) failed executing http.request(..):", e)
Binary file added source/LambdaLayers/test/.DS_Store
Binary file not shown.
88 changes: 88 additions & 0 deletions source/LambdaLayers/test/test_cfnresponse.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
#!/usr/bin/python
###############################################################################
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. #
# #
# Licensed under the Apache License Version 2.0 (the "License"). You may not #
# use this file except in compliance with the License. A copy of the License #
# is located at #
# #
# http://www.apache.org/licenses/LICENSE-2.0/ #
# #
# or in the "license" file accompanying this file. This file is distributed #
# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express #
# or implied. See the License for the specific language governing permis- #
# sions and limitations under the License. #
###############################################################################

import cfnresponse
from pytest_mock import mocker
from unittest.mock import ANY
import pytest
import json
import os

os.environ['AWS_REGION'] = 'us-east-1'
os.environ['AWS_PARTITION'] = 'aws'

@pytest.fixture()
def urllib_mock(mocker):
yield mocker.patch('cfnresponse.http')

@pytest.fixture()
def event():
yield {
'ResponseURL': 'response_url',
'StackId': 'stack_id',
'RequestId': 'request_id',
'LogicalResourceId': 'logical_resource_id'
}

class Context:
def __init__(self, log_stream_name):
self.log_stream_name = log_stream_name

@pytest.fixture()
def context():
yield Context('log_stream_name')

def body_correct(body, event, context, status, response_data, physical_resource_id = None, no_echo = False, reason = None):
assert body['Status'] == status
assert body['StackId'] == event['StackId']
assert body['RequestId'] == event['RequestId']
assert body['LogicalResourceId'] == event['LogicalResourceId']
if physical_resource_id is not None:
assert body['PhysicalResourceId'] == physical_resource_id
else:
assert body['PhysicalResourceId'] == context.log_stream_name
assert body['NoEcho'] == no_echo
if reason is not None:
assert body['Reason'] == reason
else:
assert context.log_stream_name in body['Reason']
assert body['Data'] == response_data
return True

def test_send(urllib_mock, event, context):
status = cfnresponse.SUCCESS
response_data = {}
cfnresponse.send(event, context, status, response_data)
urllib_mock.request.assert_called_once_with('PUT', event['ResponseURL'], body = ANY, headers = ANY)
_, _, call_kwargs = urllib_mock.request.mock_calls[0]
assert body_correct(json.loads(call_kwargs['body']), event, context, status, response_data)
assert call_kwargs['headers']['content-length'] == str(len(call_kwargs['body']))

def test_send_with_reason(urllib_mock, event, context):
status = cfnresponse.FAILED
response_data = {'some': 'data', 'key': 'value'}
physical_resource_id = 'some_id'
no_echo = True
reason = 'some_reason'
cfnresponse.send(event, context, status, response_data, physical_resource_id, no_echo, reason)
urllib_mock.request.assert_called_once_with('PUT', event['ResponseURL'], body = ANY, headers = ANY)
_, _, call_kwargs = urllib_mock.request.mock_calls[0]
assert body_correct(json.loads(call_kwargs['body']), event, context, status, response_data, physical_resource_id, no_echo, reason)
assert call_kwargs['headers']['content-length'] == str(len(call_kwargs['body']))

def test_send_exception(urllib_mock, event, context):
urllib_mock.request.side_effect = Exception()
cfnresponse.send(event, context, cfnresponse.FAILED, {})
Binary file added source/Orchestrator/.DS_Store
Binary file not shown.
Loading

0 comments on commit 8cfa35d

Please sign in to comment.