Skip to content

Template

Max v.A edited this page Jul 2, 2023 · 1 revision

Template

The templates are the core part of gitAnalyzer.
A template is a yaml file, which defines, what operation will be performed during a scan.

Base template.yaml

name: "New-Template"
description: "This is a description"
tags: [] #List of tags to filter/select this template
type: "Flat" #Flat only scans last commit, Full scans all branches and commits
requirements:
  tools: [] #Needed cli tools e. g. curl, docker
  pip: [] #Needed pip packages
  npm: [] #Needed npm packages
regex:
  - expression: (A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}
    group: 0 #The captured group of the wanted value
    description: "Example Regex"
    references: [ "https", "https:" ] #Links to blog posts or regex101 etc.
    false_positives: [ "wrongToken", "xxxxx" ] #Known false positive values, match via contain.
    validation:
      tests: #Specifies values for a unit test
        - input: "aws_access_token: AKIAIOSFODNN7EXAMPLE"
          want: [ "AKIAIOSFODNN7EXAMPLE" ]
output:
  uniq: true #The output will be filtered, so only unique values will be shown.
match:
  filename: ["package.json"] #Match only the files package.json
  exclude: ["node_modules"] #Exclude all files inside the given folder/path.
script: #Will be executed for the matched files
  language: "bash" #Language of the script. Bash for multiline bash scripts, cli for cli commands and python for python scripts
  code: |+
    #!/bin/bash
pre_script: #Will be executed once prior of the execution of all scripts
  language: "bash"
  code: "docker-compose up -d"

Properties

Name (required):

The name section contains the name of the template as a string.
It is a unique identifier and can be used to exclude a template from a scan
using the -e or --exclude flags for the run command.

Description (required):

The description contains a string which describes, what the template is about.

Tags (required):

Multiple strings can be used inside the tags section to describe key properties of the template.
While using the run mode the tags can be used to filter the loaded template to execute only a subset using
the --filter or -f flags.

Type (required):

The type property can be used to set the level of detail the template
will search through commit objects in the git commit history.
Valid options are:

  • "Flat" (default): The template will be executed only on the HEAD commit of the main branch.
    This is the fastest type.
  • "Deep": The template will be executed on all commits of the current main branch.
  • "Full": This is the slowest possible type, as the template will be executed on all commits
    on all branches of the repository. If some commits are part of multiple branches, the template
    will be only executed once on these commits.

Requirements:

As a template can be used to execute other tools, the requirements property can be used
define, on which tools or packages the template depends on:

  • tools: CLI tools like curl or wget and be set as a dependency in this list.
  • pip: Necessary Python packages can be defined inside this list.
  • npm: Packages or tools from NPM can be set as dependencies too.

Regex:

The regex property can contain a list of regular expression. A single regular expression consist
of the following properties:

  • Expression (required): The expression property contains the regular expression which will be compiled and executed.
    If the expression contains quotation marks " they need to be escaped like \" to not break the yaml syntax.
  • Group (required): To get the correct value from the regular expression, the group property can be
    used to set the correct capture group. Tools like regex101 can be used to fined the correct value for this property.
  • Description (required): As the description for the template, the description property should describe what
    the regular expression is about, e.g.: "Search for api key xy"
  • References: This list of strings can be used to store references to further information sources like blogpost or GitHub links.
  • False positives: To reduce and minimize the amount of false positive results, the false_positives
    property can be used to store known false positive values. Results containing these values, will not be returned.
  • Validation: This section/property contains the tests property, which can be used to provide multiple test cases
    to validate, that the regular expression works as indent. A test case consist of an input as a string
    and the list want of expected returned values of the input string.

Output:

This property defines settings regarding the output of the template.
The following properties are available:

  • Uniq: The property uniq can be set in this property. If this property is set, a new result file will
    be created, which contains only unique results of the template. Duplicates will be removed.

Match:

To filter where a command will be executed, the match property can be used to filter
files and paths. Currently match supports the following properties:

  • Filename: This property can be used to filter all files by a given set of filename.
  • Exclude: If some files should not be includes e.g. a node_modules folder, the exclude
    property can be used for this purpose.

Script:

Apart from a regular expression, the script properties is the other way to generate a result with a template.
The script property defines a command which will be executed inside the repository.
To define a command, the following properties must be set:

  • Language: This property is used to determine, which interpreter will be used,
    valid options for the language property are: CLI (default), bash and python
  • Code (required): The command which will be executed, can be defined as a string inside the
    code property like cat package.json | wc -l. If the command is a multiline command the command
    can be appended after |+ like shown in log4shell.yaml. Multiple placeholders can be defined inside the command:
    • {{Hash}}: To insert the commit hash of the current checked out commit, the {{Hash}} placeholder
      can be used.
    • {{File}}: As the match property can be used to execute a command on multiple files/path,
      the {{File}} placeholder can be used along it. The placeholder will be replaced with matching paths/files.

PreScript:

The pre_script property works the same way the script does,
the only difference is, that the pre_script is only executed once before the scan starts.

Template directory

The template directory stores all templates as yaml files. The default path will be ./templates.
If another directory with which contains template yaml files should be used, the path can be
set via the --templates or -t flags of the run module.
Templates can be excluded from execution using the -e or --exclude flag followed by
the name of the template.