Merge pull request #388 from therasec/docs/1.2.0

Documentation update for 1.2.0 release

CHANGELOG.md

@@ -1,5 +1,66 @@
 # Changelog
 
+## [v1.2.0](https://github.com/accurics/terrascan/tree/v1.2.0) (2020-11-16)
+
+[Full Changelog](https://github.com/accurics/terrascan/compare/v1.1.0...v1.2.0)
+
+**Implemented enhancements:**
+
+- Add support for Helm [\#353](https://github.com/accurics/terrascan/issues/353)
+- Add 'git' to container image, or run container as 'root' user by default [\#349](https://github.com/accurics/terrascan/issues/349)
+- Add policy for checking insecure\_ssl configuration for github\_organization\_webhook in GitHub provider [\#339](https://github.com/accurics/terrascan/issues/339)
+- Rule for github\_repository seems to be wrongly placed under gcp [\#325](https://github.com/accurics/terrascan/issues/325)
+
+**Fixed bugs:**
+
+- Fail to validate when there are multiple properties with the same name in a resource [\#1](https://github.com/accurics/terrascan/issues/1)
+
+**Closed issues:**
+
+- Deep modules location mis-proccessed.  [\#365](https://github.com/accurics/terrascan/issues/365)
+- 20MB binary file included in repo now [\#364](https://github.com/accurics/terrascan/issues/364)
+- Private GitHub repositories are not recognized with version 3.0.0+ of GitHub provider [\#326](https://github.com/accurics/terrascan/issues/326)
+- Terrascan -var-file=../another dir [\#144](https://github.com/accurics/terrascan/issues/144)
+- Error in test\_aws\_security\_group\_inline\_rule\_open and test\_aws\_security\_group\_rule\_open [\#138](https://github.com/accurics/terrascan/issues/138)
+- Intial setup after installation [\#136](https://github.com/accurics/terrascan/issues/136)
+- Add support for data sources [\#3](https://github.com/accurics/terrascan/issues/3)
+- Support from modules [\#2](https://github.com/accurics/terrascan/issues/2)
+
+**Merged pull requests:**
+
+- Bring Go to 1.15 in Github Actions [\#384](https://github.com/accurics/terrascan/pull/384) ([gliptak](https://github.com/gliptak))
+- Bring Go to 1.15 in Github Actions [\#383](https://github.com/accurics/terrascan/pull/383) ([gliptak](https://github.com/gliptak))
+- fix a bug when rendering subcharts [\#381](https://github.com/accurics/terrascan/pull/381) ([williepaul](https://github.com/williepaul))
+- Added kustomize support [\#378](https://github.com/accurics/terrascan/pull/378) ([dev-gaur](https://github.com/dev-gaur))
+- Adds support for Helm v3 [\#377](https://github.com/accurics/terrascan/pull/377) ([williepaul](https://github.com/williepaul))
+- Update mkdocs-material to 6.1.4 [\#374](https://github.com/accurics/terrascan/pull/374) ([pyup-bot](https://github.com/pyup-bot))
+- properly handle nested submodules \(\#365\) [\#373](https://github.com/accurics/terrascan/pull/373) ([acc-jon](https://github.com/acc-jon))
+- Address \#365 by properly handling submodule path [\#372](https://github.com/accurics/terrascan/pull/372) ([acc-jon](https://github.com/acc-jon))
+- Update mkdocs-material to 6.1.3 [\#371](https://github.com/accurics/terrascan/pull/371) ([pyup-bot](https://github.com/pyup-bot))
+- Update mkdocs-material to 6.1.2 [\#370](https://github.com/accurics/terrascan/pull/370) ([pyup-bot](https://github.com/pyup-bot))
+- Allow use of multiple policy types \(scan -t x,y or scan -t x -t y\) [\#368](https://github.com/accurics/terrascan/pull/368) ([acc-jon](https://github.com/acc-jon))
+- Remove large binary that was included in the repo [\#366](https://github.com/accurics/terrascan/pull/366) ([cesar-rodriguez](https://github.com/cesar-rodriguez))
+- fix send request method, previously hardcoded [\#361](https://github.com/accurics/terrascan/pull/361) ([kanchwala-yusuf](https://github.com/kanchwala-yusuf))
+- Add git binary to terrascan docker image, required by downloader [\#360](https://github.com/accurics/terrascan/pull/360) ([kanchwala-yusuf](https://github.com/kanchwala-yusuf))
+- Adds new policies/regos for AWS serverless services  [\#357](https://github.com/accurics/terrascan/pull/357) ([cesar-rodriguez](https://github.com/cesar-rodriguez))
+- Update mkdocs-material to 6.1.0 [\#356](https://github.com/accurics/terrascan/pull/356) ([pyup-bot](https://github.com/pyup-bot))
+- Allow configuration of global policy config, fix some typos [\#354](https://github.com/accurics/terrascan/pull/354) ([acc-jon](https://github.com/acc-jon))
+- Feature/support resolve variable references [\#351](https://github.com/accurics/terrascan/pull/351) ([kanchwala-yusuf](https://github.com/kanchwala-yusuf))
+- Add new policy for checking insecure\_ssl on github\_organization\_webhook [\#350](https://github.com/accurics/terrascan/pull/350) ([HorizonNet](https://github.com/HorizonNet))
+- Update mkdocs-material to 6.0.2 [\#348](https://github.com/accurics/terrascan/pull/348) ([pyup-bot](https://github.com/pyup-bot))
+- Add support for colorized output [\#347](https://github.com/accurics/terrascan/pull/347) ([acc-jon](https://github.com/acc-jon))
+- Update mkdocs-material to 6.0.1 [\#346](https://github.com/accurics/terrascan/pull/346) ([pyup-bot](https://github.com/pyup-bot))
+- Adds support for remote Terraform modules and scanning remotely for other IaC tools [\#345](https://github.com/accurics/terrascan/pull/345) ([kanchwala-yusuf](https://github.com/kanchwala-yusuf))
+- fix supported providers unit test, sort the wanted result [\#344](https://github.com/accurics/terrascan/pull/344) ([kanchwala-yusuf](https://github.com/kanchwala-yusuf))
+- Fix typo on AWS IAM account password policy rego name [\#343](https://github.com/accurics/terrascan/pull/343) ([kmonticolo](https://github.com/kmonticolo))
+- Update mkdocs-material to 5.5.14 [\#340](https://github.com/accurics/terrascan/pull/340) ([pyup-bot](https://github.com/pyup-bot))
+- Adds docs section for GitHub policies [\#337](https://github.com/accurics/terrascan/pull/337) ([cesar-rodriguez](https://github.com/cesar-rodriguez))
+- Automatically populate usage with supported IaC providers, versions, and policies [\#336](https://github.com/accurics/terrascan/pull/336) ([kanchwala-yusuf](https://github.com/kanchwala-yusuf))
+- Add line about kubernetes YAML/JSON support [\#335](https://github.com/accurics/terrascan/pull/335) ([williepaul](https://github.com/williepaul))
+- Add policy set for GitHub provider [\#334](https://github.com/accurics/terrascan/pull/334) ([HorizonNet](https://github.com/HorizonNet))
+- Add check for visibility for github\_repository [\#333](https://github.com/accurics/terrascan/pull/333) ([HorizonNet](https://github.com/HorizonNet))
+- Add instructions for booting terrascan demo [\#319](https://github.com/accurics/terrascan/pull/319) ([kklin](https://github.com/kklin))
+
 ## [v1.1.0](https://github.com/accurics/terrascan/tree/v1.1.0) (2020-09-16)
 
 [Full Changelog](https://github.com/accurics/terrascan/compare/v1.0.0...v1.1.0)

README.md

@@ -18,14 +18,14 @@ Detect compliance and security violations across Infrastructure as Code to mitig
 ## Features
 * 500+ Policies for security best practices
 * Scanning of Terraform 12+ (HCL2)
-* Scanning of Kubernetes (JSON/YAML), and Helm v3
+* Scanning of Kubernetes (JSON/YAML), Helm v3, and Kustomize v3
 * Support for AWS, Azure, GCP, Kubernetes and GitHub
 
 ## Installing
 Terrascan's binary for your architecture can be found on the [releases](https://github.com/accurics/terrascan/releases) page. Here's an example of how to install it:
 
 ```sh
-$ curl --location https://github.com/accurics/terrascan/releases/download/v1.1.0/terrascan_1.1.0_Darwin_x86_64.tar.gz --output terrascan.tar.gz
+$ curl --location https://github.com/accurics/terrascan/releases/download/v1.2.0/terrascan_1.2.0_Darwin_x86_64.tar.gz --output terrascan.tar.gz
 $ tar -xvf terrascan.tar.gz
   x CHANGELOG.md
   x LICENSE
@@ -39,8 +39,8 @@ If you have go installed, Terrascan can be installed with `go get`
 ```
 $ export GO111MODULE=on
 $ go get -u github.com/accurics/terrascan/cmd/terrascan
-  go: downloading github.com/accurics/terrascan v1.1.0
-  go: found github.com/accurics/terrascan/cmd/terrascan in github.com/accurics/terrascan v1.1.0
+  go: downloading github.com/accurics/terrascan v1.2.0
+  go: found github.com/accurics/terrascan/cmd/terrascan in github.com/accurics/terrascan v1.2.0
   ...
 $ terrascan
 ```
@@ -72,7 +72,7 @@ $ ./bin/terrascan
 
 ## Getting started
 
-To scan your code for security issues you can run the following
+To scan your code for security issues you can run the following (defaults to scanning Terraform).
 
 ```sh
 $ terrascan scan
@@ -110,7 +110,7 @@ Use "terrascan [command] --help" for more information about a command.
 ```
 
 ## Documentation
-To learn more about Terrascan check out the documentation https://docs.accurics.com where we include a getting started guide, Terrascan's architecture, a break down of it's commands, and a deep dive into policies.
+To learn more about Terrascan check out the documentation https://docs.accurics.com where we include a getting started guide, Terrascan's architecture, a breakdown of it's commands, and a deep dive into policies.
 
 ## Developing Terrascan
 To learn more about developing and contributing to Terrascan refer to the [contributing guide](CONTRIBUTING.md).

docs/getting-started.md

@@ -80,10 +80,10 @@ Use "terrascan [command] --help" for more information about a command.
 ```
 
 ### Initializing
-The initialization process downloads the latest policies from the [repository](https://github.com/accurics/terrascan) into `~/.terrascan`. The policies are located at `~/.terrascan/pkg/policies/opa/rego` and are fetched when scanning the IaC. This command is implicitly executed if the `scan` command doesn't found policies while executing.
+The initialization process downloads the latest policies from the [repository](https://github.com/accurics/terrascan) into `~/.terrascan`. The policies are located at `~/.terrascan/pkg/policies/opa/rego` and are fetched when scanning the IaC. This command is implicitly executed if the `scan` command doesn't find policies while executing.
 
 ### Scanning
-The CLI will default to scanning all supported cloud providers on Terraform HCL files if the `scan` command is used with no arguments. For example, the below two commands will scan the current directory containing Terraform HCL2 files for supported cloud providers (AWS, GCP, and Azure) resources:
+The CLI will default to scanning all supported cloud providers on Terraform HCL files if the `scan` command is used with no arguments. For example, the below two commands will scan the current directory containing Terraform HCL2 files for supported providers (AWS, GCP, Azure, and GitHub) resources:
 
 ``` Bash
 $ terrascan scan
@@ -98,10 +98,10 @@ $ terrascan scan -t aws
 By default Terrascan defaults to scanning Terraform HCL files, you can change the IaC provider using the -i flag. Here's an example of scanning kubernetes yaml files:
 
 ``` Bash
-$ terrascan scan -t k8s -i k8s
+$ terrascan scan -i k8s
 ```
 
-The `scan` command support flags to configure: the directory being scanned, scanning of a specific file, IaC provider type, path to policies, and policy type. The full list of flags can be found by typing `terrascan scan -h`
+The `scan` command supports flags to configure: the directory being scanned, scanning of a specific file, IaC provider type, path to policies, and policy type. The full list of flags can be found by typing `terrascan scan -h`
 
 ``` Bash
 $ terrascan scan -h
@@ -117,8 +117,8 @@ Flags:
   -h, --help                      help for scan
   -d, --iac-dir string            path to a directory containing one or more IaC files (default ".")
   -f, --iac-file string           path to a single IaC file
-  -i, --iac-type string           iac type (helm, k8s, terraform)
-      --iac-version string        iac version (helm: v3, k8s: v1, terraform: v12)
+  -i, --iac-type string           iac type (helm, k8s, kustomize, terraform)
+      --iac-version string        iac version (helm: v3, k8s: v1, kustomize: v3, terraform: v12)
   -p, --policy-path stringArray   policy path directory
   -t, --policy-type strings       policy type (all, aws, azure, gcp, github, k8s) (default [all])
   -r, --remote-type string        type of remote backend (git, s3, gcs, http)
@@ -151,14 +151,27 @@ The URLs for the remote should follow similar naming as the source argument for
 Helm chart can be scanned by specifying "helm" on the -i flag as follows:
 
 ```
-$ terrascan scan -t k8s -i helm
+$ terrascan scan -i helm
 ```
 
 This command will recursively look for Chart.yaml files in the current directory and scans rendered .yaml, .yml, .tpl template files found under the corresponding /templates directory.
 
 A specific directory to scan can be specified using the `-d` flag. The Helm IaC provider does not support scanning of individual files using the `-f` flag.
 
 
+#### Kustomize
+
+Kustomize chart can be scanned by specifying "kustomize" on the -i flag as follows:
+
+```
+$ terrascan scan -i kustomize
+```
+
+This command will look for a kustomization.yaml file in the current directory and scans rendered .yaml or .yml template files.
+
+A specific directory to scan can be specified using the `-d` flag. The Kustomize IaC provider does not support scanning of individual files using the `-f` flag.
+
+
 ### CLI Output types
 #### Violations
 Terrascan's default output is a list of violations present in the scanned IaC.

docs/policies.md

@@ -2,7 +2,88 @@
 
 Terrascan policies are written using the [Rego policy language](https://www.openpolicyagent.org/docs/latest/policy-language/). With each rego policy a JSON "rule" file is included which defines metadata for the policy. Policies included within Terrascan are stored in the [pkg/policies/opa/rego](https://github.com/accurics/terrascan/tree/master/pkg/policies/opa/rego) directory.
 
-## Rule JSON file
+
+## Updating Terrascan with the latest policies
+
+The first time using Terrascan, if the `-p` flag is not specified, Terrascan will download the latest policies from the Terrascan repository. To update with the latest policies remove the `~/.terrascan` directory from your system and run `terrascan init`.
+
+## Ignoring Policies on a scan
+
+Terrascan keeps a copy of policies on your local filesystem on the `~/.terrascan/pkg/policies/opa/rego` directory. You can also specify a particular directory with rego policies to scan by using the `-p` flag. To ignore a particular policy from a scan, you can remove the rule `.json` file for the policy you would like to ignore from the scan. Note that this policy would be ignored until the `.json` file is added again to the directory.
+
+In a future enhancement, Terrascan will have a better way to ignore individual policies from scans without having to modify the policies stored in the file system [#367](https://github.com/accurics/terrascan/issues/367).
+
+## Adding policies
+
+For each policy there are 2 files required by Terrascan, a rule `.json` file with metada for the policy and a `.opa` [rego](https://www.openpolicyagent.org/docs/latest/policy-language/) file.
+
+### Writing an OPA rego policy file
+The input for the rego policies is the normalized input from the IaC provider. When writing policies you can obtain this as a normalized `.json` by using the `--config-only` flag of the scan command in combination with `-o json`. Let's use this Terraform HCL file for example:
+
+``` hcl
+resource "github_repository" "example" {
+  name        = "example"
+  description = "My awesome codebase"
+
+  private = false
+
+  template {
+    owner = "github"
+    repository = "terraform-module-template"
+  }
+}
+```
+
+Here's the output of the `--config-only` flag.
+
+``` json
+$ terrascan scan -i terraform --config-only -o json
+{
+  "github_repository": [
+    {
+      "id": "github_repository.example",
+      "name": "example",
+      "source": "main.tf",
+      "line": 1,
+      "type": "github_repository",
+      "config": {
+        "description": "My awesome codebase",
+        "name": "example",
+        "private": false,
+        "template": [
+          {
+            "owner": "github",
+            "repository": "terraform-module-template"
+          }
+        ]
+      }
+    }
+  ]
+}
+```
+
+You can use this `.json` output as the input in the (rego playgound)[https://play.openpolicyagent.org/]. The following policy can be used on the above Terraform to flag if the GitHub repository has been created with `private = false`.
+
+```
+package accurics
+
+privateRepoEnabled[api.id] {
+api := input.github_repository[_]
+not api.config.private == true
+}
+```
+
+A successful policy will trigger the following output:
+
+``` json
+{
+    "privateRepoEnabled": [
+        "github_repository.example"
+    ]
+}
+```
+
+### The Rule JSON file
 
 The rule files follow this naming convention: `<cloud-provider>.<resource-type>.<rule-category>.<severity>.<next-available-rule-number>.json`