Skip to content

Commit

Permalink
added the single_bucket input and updated the README
Browse files Browse the repository at this point in the history
Signed-off-by: GitHub <noreply@github.com>
  • Loading branch information
aaronlippold committed Apr 29, 2022
1 parent 90f5c1d commit 4836da1
Show file tree
Hide file tree
Showing 4 changed files with 88 additions and 40 deletions.
108 changes: 69 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,20 @@
# aws-s3-baseline
# aws-s3-baseline

A micro-baseline to check for insecure or public S3 buckets and bucket objects in your AWS Environment. This [InSpec](https://github.com/chef/inspec) compliance profile verifies that you do not have any insure or open to public S3 Bucket or Bucket Objects in your AWS Environment in an automated way.

NOTE: Small Word of Warning
#### Warning: Large amounts of Bucket Objects

In this InSpec profile implementation, the `s3-objects-no-public-access` control iterates through and verifies every objects in each bucket in your AWS Environment, thus its runtime will depend on the number of objects in your S3 Buckets.
The `s3-objects-no-public-access` control iterates through and verifies every objects in each bucket in your AWS Environment, thus its runtime will depend on the number of objects in your S3 Buckets.

If you have buckets with large numbers of objects, we suggest you script a loop and use the `single_bucket` input to parallelize the workload.

Then you can load all your HDF JSON results into [Heimdall Lite](https://heimdall-lite.mitre.org) to easily review all your scan results.

## Getting Started
It is intended and recommended that InSpec and this profile be run from a __"runner"__ host (such as a DevOps orchestration server, an administrative management system, or a developer's workstation/laptop) against the target remotely over __AWS CLI__.

__For the best security of the runner, always install on the runner the _latest version_ of InSpec and supporting Ruby language components.__
It is intended and recommended that InSpec and this profile be run from a **"runner"** host (such as a DevOps orchestration server, an administrative management system, or a developer's workstation/laptop) against the target remotely over **AWS CLI**.

**For the best security of the runner, always install on the runner the _latest version_ of InSpec and supporting Ruby language components.**

The latest versions and installation options are available at the [InSpec](http://inspec.io/) site.

Expand All @@ -26,27 +31,38 @@ You will need to ensure your AWS CLI environment has the right system environmen

### Notes on MFA

In any AWS MFA enabled environment - you need to use `derived credentials` to use the CLI. Your default `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` will not satisfy the MFA Policies in AWS environments.
In any AWS MFA enabled environment - you need to use `derived credentials` to use the CLI. Your default `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` will not satisfy the MFA Policies in AWS environments.

- The AWS documentation is here: https://docs.aws.amazon.com/cli/latest/reference/sts/get-session-token.html
- The AWS profile documentation is here: https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html
- A useful bash script for automating this is here: https://gist.github.com/dinvlad/d1bc0a45419abc277eb86f2d1ce70625
- The AWS documentation is here: <https://docs.aws.amazon.com/cli/latest/reference/sts/get-session-token.html>
- The AWS profile documentation is here: <https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html>
- A useful bash script for automating this is here: <https://gist.github.com/dinvlad/d1bc0a45419abc277eb86f2d1ce70625>

To generate credentials using an AWS Profile you will need to use the following AWS CLI commands.

a. `aws sts get-session-token --serial-number arn:aws:iam::<$YOUR-MFA-SERIAL> --token-code <$YOUR-CURRENT-MFA-TOKEN> --profile=<$YOUR-AWS-PROFILE>`
a. `aws sts get-session-token --serial-number arn:aws:iam::<$YOUR-MFA-SERIAL> --token-code <$YOUR-CURRENT-MFA-TOKEN> --profile=<$YOUR-AWS-PROFILE>`

b. Then export the `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_SESSION_TOKEN` that was generated by the above command.
b. Then export the `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_SESSION_TOKEN` that was generated by the above command.

## Tailoring to Your Environment

The following inputs must be configured in an inputs ".yml" file for the profile to run correctly for your specific environment. More information about InSpec inputs can be found in the [InSpec Profile Documentation](https://www.inspec.io/docs/reference/profiles/).

```
# Description: List of buckets exempted from inspection.
exception_bucket_list: []

```yaml
# List of buckets exempted from inspection.
exception_bucket_list:
- bucket1
- bucket2
...

# Test only one bucket
single_bucket: 'my-bucket'
```

## Usage
## Note

When you use the `single_bucket` input, the profile will _***ONLY scan***_ that bucket.

# Usage

```
# Set required ENV variables
Expand All @@ -55,72 +71,86 @@ $ export AWS_SECRET_ACCESS_KEY=access-key
$ export AWS_SESSION_TOKEN=session-token # if MFA is enabled
```

# Running This Baseline Directly from Github
## Running This Baseline Directly from Github

```
# How to run
inspec exec https://github.com/mitre/aws-s3-baseline/archive/master.tar.gz --target aws:// --input-file=<path_to_your_inputs_file/name_of_your_inputs_file.yml> --reporter=cli json:<path_to_your_output_file/name_of_your_output_file.json>
```
### Testing all your buckets except those defined in your `excluded buckets`

`inspec exec https://github.com/mitre/aws-s3-baseline/archive/master.tar.gz --target aws:// --input-file=your_inputs_file.yml --reporter=cli json:your_output_file.json`

### Testing a single bucket

`inspec exec https://github.com/mitre/aws-s3-baseline/archive/master.tar.gz --target aws:// --input single_bucket=your_bucket --reporter=cli json:your_output_file.json`

### Different Run Options

[Full exec options](https://docs.chef.io/inspec/cli/#options-3)
[Full exec options](https://docs.chef.io/inspec/cli/#options-3)

## Running This Baseline from a local Archive copy
## Running This Baseline from a local Archive copy

If your runner is not always expected to have direct access to GitHub, use the following steps to create an archive bundle of this baseline and all of its dependent tests:

(Git is required to clone the InSpec profile using the instructions below. Git can be downloaded from the [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) site.)

When the __"runner"__ host uses this profile baseline for the first time, follow these steps:
When the **"runner"** host uses this profile baseline for the first time, follow these steps:

```
### Create your Archieve of the Profile

```bash
mkdir profiles
cd profiles
git clone https://github.com/mitre/aws-s3-baseline
inspec archive aws-s3-baseline
inspec exec <name of generated archive> --target aws:// --input-file=<path_to_your_inputs_file/name_of_your_inputs_file.yml> --reporter=cli json:<path_to_your_output_file/name_of_your_output_file.json>
```

### Run your scan using the Archieved Copy

`inspec exec <name of generated archive> --target aws:// --input-file=<path_to_your_inputs_file/name_of_your_inputs_file.yml> --reporter=cli json:<path_to_your_output_file/name_of_your_output_file.json>`

### Updating your Archieved Copy

For every successive run, follow these steps to always have the latest version of this baseline:

```
```bash
cd aws-s3-baseline
git pull
cd ..
inspec archive aws-s3-baseline --overwrite
inspec exec <name of generated archive> --target aws:// --input-file=<path_to_your_inputs_file/name_of_your_inputs_file.yml> --reporter=cli json:<path_to_your_output_file/name_of_your_output_file.json>
```

### Run your updated Archieved Copy

`inspec exec <name of generated archive> --target aws:// --input-file=<path_to_your_inputs_file/name_of_your_inputs_file.yml> --reporter=cli json:<path_to_your_output_file/name_of_your_output_file.json>`

## Using Heimdall for Viewing the JSON Results

The JSON results output file can be loaded into __[heimdall-lite](https://heimdall-lite.mitre.org/)__ for a user-interactive, graphical view of the InSpec results.
The JSON results output file can be loaded into **[heimdall-lite](https://heimdall-lite.mitre.org/)** for a user-interactive, graphical view of the InSpec results.

The JSON InSpec results file may also be loaded into a __[full heimdall server](https://github.com/mitre/heimdall)__, allowing for additional functionality such as to store and compare multiple profile runs.
The JSON InSpec results file may also be loaded into a **[full heimdall server](https://github.com/mitre/heimdall)**, allowing for additional functionality such as to store and compare multiple profile runs.

## Authors

* Rony Xavier - [rx294](https://github.com/rx294)
* Aaron Lippold - [aaronlippold](https://github.com/aaronlippold)
* Matthew Dromazos - [dromazmj](https://github.com/dromazmj)
- Rony Xavier - [rx294](https://github.com/rx294)
- Aaron Lippold - [aaronlippold](https://github.com/aaronlippold)
- Matthew Dromazos - [dromazmj](https://github.com/dromazmj)

### Special Thanks

* Shivani Karikar - [karikarshivani](https://github.com/karikarshivani)
- Shivani Karikar - [karikarshivani](https://github.com/karikarshivani)

### NOTICE

© 2018-2020 The MITRE Corporation.
© 2018-2021 The MITRE Corporation.

Approved for Public Release; Distribution Unlimited. Case Number 18-3678.

### NOTICE
### NOTICE

MITRE hereby grants express written permission to use, reproduce, distribute, modify, and otherwise leverage this software to the extent permitted by the licensed terms provided in the LICENSE.md file included with this project.

### NOTICE
### NOTICE

This software was produced for the U. S. Government under Contract Number HHSM-500-2012-00008I, and is subject to Federal Acquisition Regulation Clause 52.227-14, Rights in Data-General.
This software was produced for the U. S. Government under Contract Number HHSM-500-2012-00008I, and is subject to Federal Acquisition Regulation Clause 52.227-14, Rights in Data-General.

No other use other than that granted to the U. S. Government, or to those acting on behalf of the U. S. Government under that Clause is authorized without the express written permission of The MITRE Corporation.

For further information, please contact The MITRE Corporation, Contracts Management Office, 7515 Colshire Drive, McLean, VA 22102-7539, (703) 983-6000.
For further information, please contact The MITRE Corporation, Contracts Management Office, 7515 Colshire Drive, McLean, VA 22102-7539, (703) 983-6000.
4 changes: 4 additions & 0 deletions controls/aws_s3_bucket.rb
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@
describe 'This control is Non Applicable since no S3 buckets were found.' do
skip 'This control is Non Applicable since no S3 buckets were found.'
end
elsif !input('single_bucket').to_s.empty?
describe aws_s3_bucket(input('single_bucket').to_s) do
it { should_not be_public }
end
else
aws_s3_buckets.bucket_names.each do |bucket|
next if exception_bucket_list.include?(bucket)
Expand Down
10 changes: 10 additions & 0 deletions controls/aws_s3_bucket_objects.rb
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,16 @@
describe 'This control is Non Applicable since no S3 buckets were found.' do
skip 'This control is Non Applicable since no S3 buckets were found.'
end
elsif !input('single_bucket').to_s.empty?
my_items = aws_s3_bucket_objects(bucket_name: input('single_bucket')).contents_keys
describe "#{input('single_bucket')} object" do
my_items.each do |key|
describe key.to_s do
subject { aws_s3_bucket_object(bucket_name: input('single_bucket'), key: key) }
it { should_not be_public }
end
end
end
else
aws_s3_buckets.bucket_names.each do |bucket|
next if exception_bucket_list.include?(bucket)
Expand Down
6 changes: 5 additions & 1 deletion inspec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ copyright: MITRE, 2022
copyright_email: inspec@mitre.org
license: Apache-2.0
summary: "InSpec validation example baseline profile for AWS S3 - to test if you have public buckets"
version: 1.2.0
version: 1.3.0

inspec_version: ">= 4.0"

Expand All @@ -22,3 +22,7 @@ inputs:
type: array
value:
- ""
- name: single_bucket
description: "The name of the single bucket you wish to scan"
type: string
value: ""

0 comments on commit 4836da1

Please sign in to comment.