-
Notifications
You must be signed in to change notification settings - Fork 0
Template
The templates are the core part of gitAnalyzer.
A template is a yaml file, which defines, what operation will be performed
during a scan.
name: "New-Template"
description: "This is a description"
tags: [] #List of tags to filter/select this template
type: "Flat" #Flat only scans last commit, Full scans all branches and commits
requirements:
tools: [] #Needed cli tools e. g. curl, docker
pip: [] #Needed pip packages
npm: [] #Needed npm packages
regex:
- expression: (A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}
group: 0 #The captured group of the wanted value
description: "Example Regex"
references: [ "https", "https:" ] #Links to blog posts or regex101 etc.
false_positives: [ "wrongToken", "xxxxx" ] #Known false positive values, match via contain.
validation:
tests: #Specifies values for a unit test
- input: "aws_access_token: AKIAIOSFODNN7EXAMPLE"
want: [ "AKIAIOSFODNN7EXAMPLE" ]
output:
uniq: true #The output will be filtered, so only unique values will be shown.
match:
filename: ["package.json"] #Match only the files package.json
exclude: ["node_modules"] #Exclude all files inside the given folder/path.
script: #Will be executed for the matched files
language: "bash" #Language of the script. Bash for multiline bash scripts, cli for cli commands and python for python scripts
code: |+
#!/bin/bash
pre_script: #Will be executed once prior of the execution of all scripts
language: "bash"
code: "docker-compose up -d"
The name
section contains the name of the template as a string
.
It is a unique identifier and can be used to exclude a template from a scan
using the -e
or --exclude
flags for the run
command.
The description
contains a string
which describes, what the template is about.
Multiple strings
can be used inside the tags
section to describe key properties of the template.
While using the run
mode the tags
can be used to filter the loaded template to execute only a subset using
the --filter
or -f
flags.
The type
property can be used to set the level of detail the template
will search through commit objects in the git commit history.
Valid options are:
-
"Flat"
(default): The template will be executed only on the HEAD commit of the main branch.
This is the fastest type. -
"Deep"
: The template will be executed on all commits of the current main branch. -
"Full"
: This is the slowest possible type, as the template will be executed on all commits
on all branches of the repository. If some commits are part of multiple branches, the template
will be only executed once on these commits.
As a template can be used to execute other tools, the requirements
property can be used
define, on which tools or packages the template depends on:
-
tools
: CLI tools like curl or wget and be set as a dependency in this list. -
pip
: Necessary Python packages can be defined inside this list. -
npm
: Packages or tools from NPM can be set as dependencies too.
The regex
property can contain a list of regular expression. A single regular expression consist
of the following properties:
-
Expression (required): The
expression
property contains the regular expression which will be compiled and executed.
If the expression contains quotation marks"
they need to be escaped like\"
to not break the yaml syntax. -
Group (required): To get the correct value from the regular expression, the
group
property can be
used to set the correct capture group. Tools like regex101 can be used to fined the correct value for this property. -
Description (required): As the description for the template, the
description
property should describe what
the regular expression is about, e.g.: "Search for api key xy" -
References: This list of
strings
can be used to storereferences
to further information sources like blogpost or GitHub links. -
False positives: To reduce and minimize the amount of false positive results, the
false_positives
property can be used to store known false positive values. Results containing these values, will not be returned. -
Validation: This section/property contains the
tests
property, which can be used to provide multiple test cases
to validate, that the regular expression works as indent. A test case consist of aninput
as astring
and the listwant
of expected returned values of theinput
string.
This property defines settings regarding the output
of the template.
The following properties are available:
-
Uniq: The property
uniq
can be set in this property. If this property is set, a new result file will
be created, which contains only unique results of the template. Duplicates will be removed.
To filter where a command will be executed, the match
property can be used to filter
files and paths. Currently match
supports the following properties:
-
Filename: This property can be used to filter all files by a given set of
filename
. -
Exclude: If some files should not be includes e.g. a node_modules folder, the
exclude
property can be used for this purpose.
Apart from a regular expression, the script
properties is the other way to generate a result with a template.
The script
property defines a command which will be executed inside the repository.
To define a command, the following properties must be set:
-
Language: This property is used to determine, which interpreter will be used,
valid options for thelanguage
property are: CLI (default), bash and python -
Code (required): The command which will be executed, can be defined as a
string
inside the
code
property likecat package.json | wc -l
. If the command is a multiline command the command
can be appended after|+
like shown in log4shell.yaml. Multiple placeholders can be defined inside the command:-
{{Hash}}: To insert the commit hash of the current checked out commit, the
{{Hash}}
placeholder
can be used. -
{{File}}: As the
match
property can be used to execute a command on multiple files/path,
the{{File}}
placeholder can be used along it. The placeholder will be replaced with matching paths/files.
-
{{Hash}}: To insert the commit hash of the current checked out commit, the
The pre_script
property works the same way the script
does,
the only difference is, that the pre_script
is only executed once before the scan starts.
The template directory stores all templates as yaml files. The default path will be ./templates
.
If another directory with which contains template yaml files should be used, the path can be
set via the --templates
or -t
flags of the run
module.
Templates can be excluded from execution using the -e
or --exclude
flag followed by
the name of the template.