file out writer implementation (#1229)

* file out writer implementation

* fixed golint error

* updated logs and doc URLs

* fixed broken section anchor

* fixed test cases

* review changes

* fixed test cases

* review changes

README.md

@@ -7,7 +7,7 @@
 [![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=accurics_terrascan&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=accurics_terrascan)
 [![AUR package](https://repology.org/badge/version-for-repo/aur/terrascan.svg)](https://repology.org/project/terrascan/versions)
 [![codecov](https://codecov.io/gh/accurics/terrascan/branch/master/graph/badge.svg)](https://codecov.io/gh/accurics/terrascan)
-[![Documentation Status](https://readthedocs.com/projects/accurics-terrascan/badge/?version=latest)](https://docs.accurics.com/projects/accurics-terrascan/en/latest/?badge=latest)
+[![Documentation Status](https://readthedocs.com/projects/accurics-terrascan/badge/?version=latest)](https://runterrascan.io/)
 [![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](code_of_conduct.md)
 
 ## Introduction

deploy/helm/README.md

@@ -6,7 +6,7 @@ Validating Webhook as well, that'll use the terrascan server as its backend.
 In server mode, terrascan will act both as an API server for
 performing remote scans of IAC, as well as a validating admission
 webhook for a Kubernetes cluster. Further details can be found in
-the [main documentation](https://docs.accurics.com/projects/accurics-terrascan/en/latest/).
+the [main documentation](https://runterrascan.io/).
 
 ## Usage
 ### Set up TLS certificates
@@ -25,7 +25,7 @@ will be named `terrascan` and hosted in `terrascan` namespace. You'll have to ma
 
 ### Terrascan configuration file
 This chart will look for a [terrascan configuration
-file](https://docs.accurics.com/projects/accurics-terrascan/en/latest/usage/#config-file)
+file](https://runterrascan.io/docs/usage/config_options/)
 at `data/config.toml`. If that file exists before running `helm
 install`, it's contents will be loaded into a configMap and provided
 to the terrascan server.

deploy/kustomize/README.md

@@ -1,26 +1,26 @@
-## INSTALLING TERRASCAN IN A KUBERNETES CLUSTER USING KUSTOMIZE
+## Installing terrascan in a Kubernetes cluster using Kustomize
 
 This guide will help you install terrascan server inside your kubernetes cluster.
 We have covered the following use cases in the sections below.
 
-  - #####[Deploying Terrascan Server](deploying-terrascan-server)
+  - [Deploying Terrascan Server](#deploying-terrascan-server)  
     Terrascan operating in http server mode.
 
-  - ####[Deploying Terrascan Server in TLS Mode](deploying-terrascan-server-in-tls-mode)
+  - [Deploying Terrascan Server in TLS Mode](#deploying-terrascan-server-in-tls-mode)  
     Terrascan operating in https server mode. This deployment is also a foundation for the terrascan webhook setup.
 
-  - ####[Deploying Terrascan Server for Remote Repository Scan](deploying-terrascan-server-for-remote-repository-scan)
+  - [Deploying Terrascan Server for Remote Repository Scan](#deploying-terrascan-server-for-private-remote-repository-scan)  
     Terrascan in https server mode installed with ssh capabilities, to scan ***private*** remote repositories. For remote
     scanning public repos, deploying `Terrascan Server in TLS Mode` is sufficient.
     This deployment can be handy for use-cases like an argocd pre-sync hook that sends remote repository scan requests to the server.
 
-  - ####[Setting Up Terrascan Webhook](setting-up-terrascan-webhook)
+  - [Setting Up Terrascan Webhook](#setting-up-terrascan-webhook)  
     A Kubernetes Validating Webhook, that safeguards your cluster by denying the creation of kubernetes resources that
     can cause potential security violations.
 
-  - ####[Clean Up](clean-up)
+  - [Clean Up](#clean-up)
 
-###PRE-REQUISITE
+### Pre-requisite
 1. Make sure you have required access on the kubernetes cluster to create and update the following resources:
 
   - Secrets
@@ -63,7 +63,7 @@ terrascan server. The string replacement will be required in the following files
   kubectl create namespace terrascan
   ```
 
-###Deploying Terrascan Server
+### Deploying Terrascan Server
 
 Deploy terrascan in server mode operating in plain HTTP mode.
 
@@ -78,7 +78,7 @@ Deploy terrascan in server mode operating in plain HTTP mode.
     kustomize build server/ | kubectl apply -f -
     ```
 
-###Deploying Terrascan Server in TLS Mode
+### Deploying Terrascan Server in TLS Mode
 
 Deploy terrascan in server mode operating in HTTPS mode.
 
@@ -126,7 +126,7 @@ Deploy terrascan in server mode operating in HTTPS mode.
     kustomize build server-tls/ | kubectl apply -f -
     ```
 
-###Deploying Terrascan Server For Private Remote Repository Scan
+### Deploying Terrascan Server For Private Remote Repository Scan
 
 For scanning ***Private*** remote IaC file repositories, Terrascan must be provided with the required SSH keys to connect and clone the
 repository locally to scan it. The following steps will help in setting up for that.
@@ -170,7 +170,7 @@ repository locally to scan it. The following steps will help in setting up for t
    kustomize build server-remote-repo-scan/ | kubectl apply -f -
     ```
 
-###Setting Up Terrascan Webhook
+### Setting Up Terrascan Webhook
 If you want to setup a Validating Webhook that scans your incoming kubernetes resources using terrascan,
 follow the steps below.
 
@@ -245,7 +245,7 @@ follow the steps below.
     kubectl apply -f webhook/validating-webhook.yaml
     ```
 
-###Clean Up
+### Clean Up
 
   Deleting the namespace that you used, will delete all the resources itself.
   ```bash

docs/integrations/argocd-integration.md

@@ -328,7 +328,7 @@ Host github.com
 
 After making changes to the webhook deployment file, apply this yaml in your cluster.
 
-You can also run terrascan admission controller server outside cluster, for more information and instructions on configuring terrascan as an admission controller webhook, see https://docs.accurics.com/projects/accurics-terrascan/en/latest/integrations/admission-controller-webhooks-usage.
+You can also run terrascan admission controller server outside cluster, for more information and instructions on configuring terrascan as an admission controller webhook, see https://runterrascan.io/docs/integrations/k8s/.
 
 #### Step 2: Create a Dockerfile
 

docs/usage/command_line_mode.md

@@ -21,7 +21,7 @@ $ terrascan
 Terrascan
 
 Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure.
-For more information, please visit https://docs.accurics.com
+For more information, please visit https://runterrascan.io/
 
 Usage:
   terrascan [command]
@@ -34,12 +34,13 @@ Available Commands:
   version     Shows the Terrascan version you are currently using.
 
 Flags:
-  -c, --config-path string   config file path
-  -h, --help                 help for terrascan
-  -l, --log-level string     log level (debug, info, warn, error, panic, fatal) (default "info")
-  -x, --log-type string      log output type (console, json) (default "console")
-  -o, --output string        output type (human, json, yaml, xml) (default "human")
-      --temp-dir string      temporary directory path to download remote repository,module and templates
+  -c, --config-path string      config file path
+  -h, --help                    help for terrascan
+  -l, --log-level string        log level (debug, info, warn, error, panic, fatal) (default "info")
+      --log-output-dir string   directory path to write the log and output files
+  -x, --log-type string         log output type (console, json) (default "console")
+  -o, --output string           output type (human, json, yaml, xml, junit-xml, sarif, github-sarif) (default "human")
+      --temp-dir string         temporary directory path to download remote repository,module and templates
 
 Use "terrascan [command] --help" for more information about a command.
 ```
@@ -290,6 +291,7 @@ aws_ecr_repository:
 | ----------- | ----------- |------------|
 | -c | Use this to specify config file settings | Format supported is `*.TOML` |
 | -l | Use this to specify what log settings | debug, **info**, warn, error, panic, fatal  |
+| --log-output-dir | Use this to specify the directory path for writing scan logs and output files along with console output. In case the directory could not be resolved, the scan logs and results will be printed on console only. |
 | -x | Use this to specify the log file format | **console**, json |
 | -o | Use this to specify the scan output type | **human**, json, yaml, xml, junit-xml, sarif, github-sarif |
 | --temp-dir | Use this to specify temporary directory path to download remote repository,module and templates |
@@ -335,9 +337,10 @@ Flags:
   -v, --verbose                             will show violations with details (applicable for default output)
 
 Global Flags:
-  -c, --config-path string   config file path
-  -l, --log-level string     log level (debug, info, warn, error, panic, fatal) (default "info")
-  -x, --log-type string      log output type (console, json) (default "console")
-  -o, --output string        output type (human, json, yaml, xml, junit-xml, sarif, github-sarif) (default "human")
-      --temp-dir string      temporary directory path to download remote repository,module and templates
+  -c, --config-path string      config file path
+  -l, --log-level string        log level (debug, info, warn, error, panic, fatal) (default "info")
+      --log-output-dir string   directory path to write the log and output files
+  -x, --log-type string         log output type (console, json) (default "console")
+  -o, --output string           output type (human, json, yaml, xml, junit-xml, sarif, github-sarif) (default "human")
+      --temp-dir string         temporary directory path to download remote repository,module and templates
 ```

go.mod

@@ -55,13 +55,13 @@ require (
 	github.com/zclconf/go-cty v1.9.1
 	go.uber.org/zap v1.16.0
 	golang.org/x/sys v0.0.0-20211205182925-97ca703d548d
-	golang.org/x/tools v0.1.8 // indirect
+	golang.org/x/tools v0.1.11-0.20220316014157-77aa08bb151a // indirect
 	google.golang.org/api v0.34.0
 	google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a
 	gopkg.in/src-d/go-git.v4 v4.13.1
 	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
 	helm.sh/helm/v3 v3.6.1
-	honnef.co/go/tools v0.2.2 // indirect
+	honnef.co/go/tools v0.3.1 // indirect
 	k8s.io/api v0.21.0
 	k8s.io/apimachinery v0.21.0
 	k8s.io/client-go v10.0.0+incompatible
@@ -175,9 +175,10 @@ require (
 	go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 // indirect
 	go.uber.org/atomic v1.6.0 // indirect
 	go.uber.org/multierr v1.5.0 // indirect
-	golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 // indirect
+	golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 // indirect
+	golang.org/x/exp/typeparams v0.0.0-20220218215828-6cf2b201936e // indirect
 	golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect
-	golang.org/x/mod v0.5.1 // indirect
+	golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 // indirect
 	golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f // indirect
 	golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43 // indirect
 	golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect

go.sum

@@ -1510,6 +1510,8 @@ golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWP
 golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
 golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 h1:It14KIkyBFYkHkwZ7k45minvA9aorojkyjGk9KJ5B/w=
 golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
+golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 h1:7I4JAnoQBe7ZtJcBaYHi5UtiO8tQHbUSXxL+pnGRANg=
+golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190125153040-c74c464bbbf2/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
@@ -1521,7 +1523,10 @@ golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u0
 golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
 golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
 golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
+golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6 h1:QE6XYQK6naiK1EPAe1g/ILLxN5RBoH5xkJk3CqlMI/Y=
 golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
+golang.org/x/exp/typeparams v0.0.0-20220218215828-6cf2b201936e h1:qyrTQ++p1afMkO4DPEeLGq/3oTsdlvdH4vqZUBWzUKM=
+golang.org/x/exp/typeparams v0.0.0-20220218215828-6cf2b201936e/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
 golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
 golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
 golang.org/x/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -1549,6 +1554,8 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.5.1 h1:OJxoQ/rynoF0dcCdI7cLPktw/hR2cueqYfjm43oqK38=
 golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
+golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 h1:kQgndtyPBW/JIYERgdxfwMYh3AVStj88WQTlNDi2a+o=
+golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY=
 golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180530234432-1e491301e022/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -1836,6 +1843,8 @@ golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4f
 golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
 golang.org/x/tools v0.1.8 h1:P1HhGGuLW4aAclzjtmJdf0mJOjVUZUzOTqkAkWL+l6w=
 golang.org/x/tools v0.1.8/go.mod h1:nABZi5QlRsZVlzPpHl034qft6wpY4eDcsTt5AaioBiU=
+golang.org/x/tools v0.1.11-0.20220316014157-77aa08bb151a h1:ofrrl6c6NG5/IOSx/R1cyiQxxjqlur0h/TvbUhkH0II=
+golang.org/x/tools v0.1.11-0.20220316014157-77aa08bb151a/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -2029,6 +2038,8 @@ honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9
 honnef.co/go/tools v0.0.1-2020.1.5/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
 honnef.co/go/tools v0.2.2 h1:MNh1AVMyVX23VUHE2O27jm6lNj3vjO5DexS4A1xvnzk=
 honnef.co/go/tools v0.2.2/go.mod h1:lPVVZ2BS5TfnjLyizF7o7hv7j9/L+8cZY2hLyjP9cGY=
+honnef.co/go/tools v0.3.1 h1:1kJlrWJLkaGXgcaeosRXViwviqjI7nkBvU2+sZW0AYc=
+honnef.co/go/tools v0.3.1/go.mod h1:vlRD9XErLMGT+mDuofSr0mMMquscM/1nQqtRSsh6m70=
 k8s.io/api v0.19.0 h1:XyrFIJqTYZJ2DU7FBE/bSPz7b1HvbVBuBf07oeo6eTc=
 k8s.io/api v0.19.0/go.mod h1:I1K45XlvTrDjmj5LoM5LuP/KYrhWbjUKT/SoPG0qTjw=
 k8s.io/apiextensions-apiserver v0.21.0 h1:Nd4uBuweg6ImzbxkC1W7xUNZcCV/8Vt10iTdTIVF3hw=

pkg/cli/output_writer.go

@@ -3,8 +3,10 @@ package cli
 import (
 	"io"
 	"os"
+	"path/filepath"
 
 	"github.com/accurics/terrascan/pkg/termcolor"
+	"go.uber.org/zap"
 )
 
 // NewOutputWriter gets a new io.Writer based on os.Stdout.
@@ -17,3 +19,36 @@ func NewOutputWriter(useColors bool) io.Writer {
 	}
 	return os.Stdout
 }
+
+// NewFileWriter gets a new io.Writer based on file output and closing function.
+// It returns `nil nil` if the value of dir is empty or if the file can't be opened
+func NewFileWriter(dir string, outputType string) (io.Writer, func() error) {
+
+	// if no directory resolved
+	if dir == "" {
+		return nil, nil
+	}
+
+	fileName := "scan-result.txt"
+
+	// decide the file extension/type
+	switch outputType {
+	case "json":
+		fileName = "scan-result.json"
+	case "yaml":
+		fileName = "scan-result.yaml"
+	case "xml", "junit-xml":
+		fileName = "scan-result.xml"
+	case "sarif", "github-sarif":
+		fileName = "scan-result.sarif"
+	}
+
+	filePath := filepath.Join(dir, fileName)
+	f, err := os.OpenFile(filePath, os.O_RDWR|os.O_CREATE, 0755)
+	if err != nil {
+		zap.S().Warn("unable to open file: %s, error:%w", filePath, err)
+		return nil, nil
+	}
+
+	return f, f.Close
+}

pkg/cli/register.go

@@ -42,14 +42,24 @@ func Execute() {
 	rootCmd.PersistentFlags().StringVarP(&OutputType, "output", "o", "human", "output type (human, json, yaml, xml, junit-xml, sarif, github-sarif)")
 	rootCmd.PersistentFlags().StringVarP(&ConfigFile, "config-path", "c", "", "config file path")
 	rootCmd.PersistentFlags().StringVarP(&CustomTempDir, "temp-dir", "", "", "temporary directory path to download remote repository,module and templates")
+	rootCmd.PersistentFlags().StringVarP(&LogOutputDir, "log-output-dir", "", "", "directory path to write the log and output files")
 
 	//Added init here in case flag parsing failed we should log which flag was incorrect.
-	logging.Init(LogType, LogLevel)
+	logging.Init(LogType, LogLevel, LogOutputDir)
 
 	// Function to execute before processing commands
 	cobra.OnInitialize(func() {
+		// making sure the LogOutputDir Exist
+		if LogOutputDir != "" {
+			err := os.MkdirAll(LogOutputDir, 0755)
+			if err != nil {
+				zap.S().Warnf("failed to resolve the log output directory: %s", LogOutputDir)
+				LogOutputDir = ""
+			}
+		}
+
 		// Set up the logger
-		logging.Init(LogType, LogLevel)
+		logging.Init(LogType, LogLevel, LogOutputDir)
 
 		if len(ConfigFile) == 0 {
 			ConfigFile = os.Getenv(config.ConfigEnvvarName)

pkg/cli/root.go

@@ -35,6 +35,9 @@ var (
 
 	// CustomTempDir Temporary directory path to download remote repository,module and templates
 	CustomTempDir string
+
+	// LogOutputDir Directory to write scan logs and result files
+	LogOutputDir string
 )
 
 var rootCmd = &cobra.Command{
@@ -43,6 +46,6 @@ var rootCmd = &cobra.Command{
 	Long: `Terrascan
 
 Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure.
-For more information, please visit https://docs.accurics.com
+For more information, please visit https://runterrascan.io/
 `,
 }