Merge branch 'replace-variables'

stelligent · Apr 18, 2018 · 336ab90 · 336ab90
2 parents d150611 + a154b48
commit 336ab90
Show file tree

Hide file tree

Showing 11 changed files with 316 additions and 169 deletions.
diff --git a/README.md b/README.md
@@ -19,12 +19,16 @@ Alternatively, you can install manually from the [releases](https://github.com/s
 # Build Command Line tool
 
 ```
-make config-lint
+make all
 ```
 
 # Run
 
-The program currently supports scanning of the following types of files:
+The program has a set of built-in rules for scanning the following types of files:
+
+* Terraform
+
+The program can also read files from a separate YAML file, and can scan these types of files:
 
 * Terraform
 * Kubernetes
@@ -36,114 +40,58 @@ And also the scanning of information from AWS Descibe API calls for:
 * Security Groups
 * IAM Users
 
+
 ## Example invocations
 
-### Validate Terraform files
+### Validate Terraform files with built-in rules
 
 ```
-./config-lint --rules example-files/rules/terraform.yml example-files/config/*
+config-lint -terraform example-files/config/*
+```
+
+### Validate Terraform files with custom rules
+
+```
+config-lint -rules examples-files/rules/terraform.yml example-files/config/*
 ```
 
 ### Validate Kubernetes files
 
 ```
-./config-lint --rules example-files/rules/kubernetes.yml example-files/config/*
+config-lint -rules example-files/rules/kubernetes.yml example-files/config/*
 ```
 
 ### Validate LintRules files
 
 This type of linting allows the tool to lint its own rules.
 
 ```
-./config-lint --rules example-files/rules/lint-rules.yml example-files/rules/*
+config-lint -rules example-files/rules/lint-rules.yml example-files/rules/*
 ```
 
 ### Validate Existing Security Groups
 
 ```
-./config-lint --rules example-files/rules/security-groups.yml
+config-lint -rules example-files/rules/security-groups.yml
 ```
 
 ### Validate Existing IAM Users
 
 ```
-./config-lint --rules example-files/rules/iam-users.yml
+config-lint -rules example-files/rules/iam-users.yml
 ```
 
-# Rules File
+# Rules
+
+A YAML file that specifies what kinds of files to process, and what validations to perform, [documented here](docs/rules.md).
 
-A YAML file that specifies what kinds of files to process, and what validations to perform.
+# Operations
 
-[Documented Here](docs/rules.md)
+The rules contain expressions that can use expresssions [documented here](docs/operations.md).
 
 ## Examples
 
-To test that an AWS instance type has one of two values:
-```
-Version: 1
-Description: Example rules
-Type: Terraform
-Files:
-  - "*.tf"
-Rules:
-  - id: EC2_INSTANCE_TYPE
-    message: Instance type should be t2.micro or m3.medium
-    resource: aws_instance
-    assertions:
-      - key: instance_type
-        op: in
-        value: t2.micro,m3.medium
-    severity: WARNING
-```
-
-This could also be done by using the or operation with two different assertions:
-
-```
-Version: 1
-Description: Example rules
-Type: Terraform
-Files:
-  - "*.tf"
-Rules:
-  - id: EC2_INSTANCE_TYPE
-    message: Instance type should be t2.micro or m3.medium
-    resource: aws_instance
-    assertions:
-      or:
-        - key: instance_type
-          op: eq
-          value: t2.micro
-        - key: instance_type
-          op: eq
-          value: m3.medium
-    severity: WARNING
-```
-
-And this could also be done by looking up the valid values in an S3 object (HTTP endpoints are also supported)
-
-```
-Version: 1
-Description: Example rules
-Type: Terraform
-Files:
-  - "*.tf"
-Rules:
-  - id: EC2_INSTANCE_TYPE
-    message: Instance type should be t2.micro or m3.medium
-    resource: aws_instance
-    assertions:
-      - key: instance_type
-        op: eq
-        value_from: s3://your-bucket/instance-types.txt
-    severity: FAILURE
-```
-
-The assertions and operations were inspired by those in Cloud Custodian: http://capitalone.github.io/cloud-custodian/docs/
-
-
-## Valid Operations
-
-[Documented Here](docs/operations.md)
+See [here](docs/example-rules.md) for examples of custom rules.
 
 # Output
 
@@ -179,32 +127,6 @@ This example will scan the example terraform file and print the "ami" attribute
 If you specify --search, the rules files is only used to determine the type of configuration files.
 The files will *not* be scanned for violations.
 
-
-# Support for AWS Config Custom Rules
-
-It is also possible to use a rules files in a Lambda that handles events from AWS Config.
-
-[Documented Here](docs/lambda.md)
-
 # Releasing
 To release a new version, run `make bumpversion` to increment the patch version and push a tag to GitHub to start the release process.
 
-# TODO
-
-* Add an optional YAML file for project settings, such as ignoring certain rules for certain resources
-* Figure out how dependency management works in go
-* The lambda function does not handle OverSizedChangeNotification
-* The lambda function name is hard-coded in the Makefile
-* Region is hard-coded to us-east-1 for GetValueFromS3
-* Use type switch as more idiomatic way to handle multiple types in match.go
-* Start using go testing coverage tools
-* Use log package for error reporting
-* Deal with a few FIXME comments in code, mostly error handling
-* Should there be some pre-defined RuleSets?
-* Would it be useful to have helper utilities to send output to CloudWatch/SNS/Kinesis?
-* Add variable interpolation for Terraform files
-* Update value_from to handle JSON return values
-* Create a Provider interface for AWS calls, create a mock for testing SecurityGroupLinter
-* Starting to have inconsistent naming in ops: is-true, is-false, has-properties vs. present, absent, empty, null
-* Add options to Assertion type, for things like 'ignore-case' for string compares? Or just use a regex?
-* Provide a default -query of 'Violations[]', and add an option for a full report
diff --git a/TODO.md b/TODO.md
@@ -0,0 +1,15 @@
+# TODO
+
+* Add an optional YAML file for project settings, such as ignoring certain rules for certain resources
+* Region is hard-coded to us-east-1 for GetValueFromS3
+* Use type switch as more idiomatic way to handle multiple types in match.go
+* Use log package for error reporting
+* Deal with a few FIXME comments in code, mostly error handling
+* Would it be useful to have helper utilities to send output to CloudWatch/SNS/Kinesis?
+* Update value_from to handle JSON return values
+* Create a Provider interface for AWS calls, create a mock for testing SecurityGroupLinter
+* Starting to have inconsistent naming in ops: is-true, is-false, has-properties vs. present, absent, empty, null
+* Add options to Assertion type, for things like 'ignore-case' for string compares? Or just use a regex?
+* Provide a default -query of 'Violations[]', and add an option for a full report
+* Document conditions
+
diff --git a/docs/example-rules.md b/docs/example-rules.md
@@ -0,0 +1,71 @@
+# Example Rules 
+
+Add these rules to a YAML file, and pass the filename to config-lint using the -rules option.
+Each rule contains a list of assertions, and these assertions use operations that are [documented here](operations.md)
+
+To test that an AWS instance type has one of two values:
+
+```
+Version: 1
+Description: Example rules
+Type: Terraform
+Files:
+  - "*.tf"
+Rules:
+  - id: EC2_INSTANCE_TYPE
+    message: Instance type should be t2.micro or m3.medium
+    resource: aws_instance
+    assertions:
+      - key: instance_type
+        op: in
+        value: t2.micro,m3.medium
+    severity: WARNING
+```
+
+This could also be done by using the or operation with two different assertions:
+
+```
+Version: 1
+Description: Example rules
+Type: Terraform
+Files:
+  - "*.tf"
+Rules:
+  - id: EC2_INSTANCE_TYPE
+    message: Instance type should be t2.micro or m3.medium
+    resource: aws_instance
+    assertions:
+      or:
+        - key: instance_type
+          op: eq
+          value: t2.micro
+        - key: instance_type
+          op: eq
+          value: m3.medium
+    severity: WARNING
+```
+
+And this could also be done by looking up the valid values in an S3 object (HTTP endpoints are also supported)
+
+```
+Version: 1
+Description: Example rules
+Type: Terraform
+Files:
+  - "*.tf"
+Rules:
+  - id: EC2_INSTANCE_TYPE
+    message: Instance type should be t2.micro or m3.medium
+    resource: aws_instance
+    assertions:
+      - key: instance_type
+        op: eq
+        value_from: s3://your-bucket/instance-types.txt
+    severity: FAILURE
+```
+
+The assertions and operations were inspired by those in Cloud Custodian: http://capitalone.github.io/cloud-custodian/docs/
+
+
+
+
diff --git a/docs/lambda.md b/docs/lambda.md
diff --git a/docs/operations.md b/docs/operations.md
@@ -251,7 +251,7 @@ Example:
     assertions:
       - every:
           key: Location
-          assertions:
+          expresssions:
             - key: latitude
               op: present
             - key: longitude
@@ -277,7 +277,7 @@ Example:
     assertions:
       - some:
           key: Location
-          assertions:
+          expresssions:
             - key: latitude
               op: present
             - key: longitude
@@ -303,7 +303,7 @@ Example:
     assertions:
       - none:
           key: "ipPermissions[]"
-          assertions:
+          expresssions:
             - key: "fromPort"
               op: eq
               value: 22

diff --git a/docs/rules.md b/docs/rules.md
@@ -19,6 +19,7 @@ Each rule contains the following attributes:
 |id         | A unique identifier for the rule                                                   |
 |message    | A string to be printed when a validation error is detected                         |
 |resource   | The resource type to which the rule will be applied                                |
+|conditions | Expressions (in addition to resource) that determine if a rule should apply        |
 |except     | An optional list of resource ids that should not be validated                      |
 |severity   | FAILURE, WARNING, NON_COMPLIANT                                                    |
 |assertions | A list of expressions used to detect validation errors, see next section           |
@@ -43,4 +44,3 @@ Each expression contains the following attributes:
 |Url        | HTTP endpoint to invoke                                                            |
 |Payload    | Optional JMESPATH to use for payload, default is '@'                               |
 
-
diff --git a/example-files/config/variables.tf b/example-files/config/variables.tf
@@ -0,0 +1,32 @@
+variable "instance_type" {
+  default = "t2.micro"
+}
+
+variable "ami" {
+  default = "ami-f2d3638a"
+}
+
+variable "project" {
+  default = "demo"
+}
+
+resource "aws_instance" "first" {
+  ami = "${var.ami}"
+  instance_type = "${var.instance_type}"
+  tags = {
+    project = "${var.project}"
+  }
+}
+
+variable "instance_type_2" {
+  default = "c4.large"
+}
+
+resource "aws_instance" "second" {
+  ami = "${var.ami}"
+  instance_type = "${var.instance_type_2}"
+  tags = {
+    project = "${var.project}"
+  }
+}
+