diff --git a/docs/api/crds/parse-definition.md b/docs/api/crds/parse-definition.md index e41f5461..f99b2c1f 100644 --- a/docs/api/crds/parse-definition.md +++ b/docs/api/crds/parse-definition.md @@ -29,6 +29,14 @@ This uses the kubernetes default [imagePullSecrets structure](https://kubernetes `ttlSecondsAfterFinished` can be used to automatically delete the completed Kubernetes job used to run the parser. This sets the `ttlSecondsAfterFinished` field on the created job. This requires your cluster to have the [TTLAfterFinished](https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/) feature gate enabled in your cluster. +### ScopeLimiterAliases (Optional) + +`scopeLimiterAliases` can be used in combination with `scopeLimiter` to create aliases for fields in findings. +The goal of this field is to ensure that the `scopeSelector` can always select an alias, regardless of the underlying representation of the data in a finding. +This field supports Mustache templating and has access to the finding object. + +See the [Scope HowTo](/docs/how-tos/scope) for more information. + ### Affinity and Tolerations (optional) [`affinity`](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/) and [`tolerations`](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) can be used to control which nodes the parser is executed on. The values should be set via Helm values (during install) or by specifying `affinity` and/or `tolerations` in the `Scan` specification. @@ -45,6 +53,8 @@ spec: imagePullSecrets: - name: dockerhub-token ttlSecondsAfterFinished: 60 + scopeLimiterAliases: + domain: "{{attributes.host}}" ``` The Parse definition is different when integrating a new scanner. We use specific conventions when adding new ParseDefinitions to the secureCodeBox repository. -More information can be found on the [templates folder documentation page for integrating new scanners](/docs/contributing/integrating-a-scanner/templates-dir) \ No newline at end of file +More information can be found on the [templates folder documentation page for integrating new scanners](/docs/contributing/integrating-a-scanner/templates-dir) diff --git a/docs/api/crds/scan.md b/docs/api/crds/scan.md index 00b424b9..af05e48a 100644 --- a/docs/api/crds/scan.md +++ b/docs/api/crds/scan.md @@ -138,9 +138,161 @@ See [#789](https://github.com/secureCodeBox/secureCodeBox/issues/789) for more d To use cascades you'll need to have the [CascadingScan hook](https://docs.securecodebox.io/docs/hooks/cascading-scans) installed. - For an example on how they can be used see the [Scanning Networks HowTo](https://docs.securecodebox.io/docs/how-tos/scanning-networks) +#### ScopeLimiter (Optional) + +`scopeLimiter` allows you to define certain rules to which cascading scans must comply before they may cascade. +For example, you can define that you can only trigger a follow-up scan against a host if its IP address is within your predefined IP range. +You can use Mustache templating in order to select certain properties from findings. + +Under `scopeLimiter`, you may specify `anyOf`, `noneOf`, and `allOf` with a selector to limit your scope. +If you specify multiple fields, all the rules must pass. + +A selector looks similar to the [Kubernetes Label Selectors](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#labelselector-v1-meta). + +```yaml +anyOf: + - key: "scope.cascading.securecodebox.io/cidr" + operator: "InCIDR" + values: ["{{attributes.ip}}"] +``` + +The `key` references one of the annotations defined on your scan. +The annotation name _must_ start with `scope.cascading.securecodebox.io/`. +These annotations can only be added on the initial scan (i.e., they cannot be modified using the [`scanAnnotations`](/docs/api/crds/cascading-rule#scanlabels--scanannotations-optional) field of the cascading scan rules) and are inherited by default. + +`operator` is one of `In`, `NotIn`, `Contains`, `DoesNotContain`, `InCIDR`, `NotInCIDR`, `SubdomainOf`, `NotSubdomainOf`. + +`values` is a list of values for which the selector should pass. + +##### Selecting lists + +A custom rendering function has been provided to select attributes in findings that are in a list. An example finding: + +```json title="Finding" +{ + name: "Subdomains found", + category: "Subdomain" + attributes: { + domains: ["example.com", "subdomain.example.com"], + } +} +``` + +To select the domains data in this finding, use the `asList` notation as shown below. + +```yaml +annotations: + scope.cascading.securecodebox.io/domain: "example.com" +... +key: "scope.cascading.securecodebox.io/domain" +operator: "In" +values: ["{{#asList}}{{attributes.domains}}{{/asList}}"] +``` + +The values will render to: `["example.com", "subdomain.example.com"]`. + +Some findings have data in lists of objects, such as the following: + +```json title="Finding" +{ + name: "Subdomains found", + category: "Subdomain" + attributes: { + addresses: [ + { + domain: "example.com", + ip: "127.0.0.1", + }, + { + domain: "subdomain.example.com", + ip: "127.0.0.2", + } + ] + } +} +``` + +To select the domains data in this finding, use the `getValues` notation as shown below. + +```yaml +annotations: + scope.cascading.securecodebox.io/domain: "example.com" +... +key: "scope.cascading.securecodebox.io/domain" +operator: "In" +# Note that the parameter is *not* set inside curly braces! +values: ["{{#getValues}}attributes.addresses.domain{{/getValues}}"] +``` + +You can also manually split values from findings if your finding is like so: + +```json title="Finding" +{ + name: "Subdomains found", + category: "Subdomain" + attributes: { + domains: "example.com,subdomain.example.com", + } +} +``` + +To select the domains data in this finding, use the `split` notation as shown below. + +```yaml +annotations: + scope.cascading.securecodebox.io/domain: "example.com" +... +key: "scope.cascading.securecodebox.io/domain" +operator: "In" +values: ["{{#split}}{{attributes.domains}}{{/split}}"] +``` + +##### Operators + +`In` & `NotIn`: The scope annotation value exists in one of `values`. Matching example: +```yaml +annotations: + scope.cascading.securecodebox.io/domain: "example.com" +... +key: "scope.cascading.securecodebox.io/domain" +operator: "In" +values: ["example.com", "subdomain.example.com"] +``` + +`Contains` & `DoesNotContain`: The scope annotation value is considered a comma-seperated list and checks if every `values` is in that list. Matching example: +```yaml +annotations: + scope.cascading.securecodebox.io/domain: "example.com,subdomain.example.com,other.example.com" +... +key: "scope.cascading.securecodebox.io/domain" +operator: "Contains" +values: ["example.com", "subdomain.example.com"] +``` + +`InCIDR` & `NotInCIDR`: The scope annotation value is considered a [CIDR](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) and checks if every `values` is within the subnet of that CIDR. Supports both IPv4 and IPv6. If the scope is defined in IPv4, will only validate IPv4 IPs in the finding values. Vice-versa for IPv6 defined in scope and IPv4 found in values. Note that all IPs in finding values must be valid addresses, regardless of whether IPv4 or IPv6 was used in the scope definition. Matching example: +```yaml +annotations: + scope.cascading.securecodebox.io/cidr: "10.10.0.0/16" +... +key: "scope.cascading.securecodebox.io/cidr" +operator: "InCIDR" +values: ["10.10.1.2", "10.10.1.3", "2001:0:ce49:7601:e866:efff:62c3:fffe"] +``` + +`SubdomainOf` & `NotSubdomainOf`: Checks if every `values` is a subdomain of the scope annotation value (inclusive; i.e. example.com is a subdomain of example.com). Matching example: +```yaml +annotations: + scope.cascading.securecodebox.io/domain: "example.com" +... +key: "scope.cascading.securecodebox.io/domain" +operator: "SubdomainOf" +values: ["subdomain.example.com", "example.com"] +``` + +See the [Scope HowTo](/docs/how-tos/scope) for more information. + ### HookSelector (Optional) `hookSelector` allows you to select which hooks to run using [Kubernetes Label Selectors](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#labelselector-v1-meta). @@ -192,6 +344,9 @@ kind: Scan status: # Set during runtime. Do not edit via values.yaml etc. metadata: name: "nmap-scanme.nmap.org" + annotations: + scope.cascading.securecodebox.io/cidr: "10.10.0.0/16" + scope.cascading.securecodebox.io/domain: "example.com" spec: scanType: "nmap" parameters: @@ -215,4 +370,14 @@ spec: key: "securecodebox.io/invasive" operator: In values: [non-invasive, invasive] + scopeLimiter: + validOnMissingRender: true + allOf: + - key: "scope.cascading.securecodebox.io/cidr" + operator: "InCIDR" + values: ["{{attributes.ip}}"] + noneOf: + - key: "scope.cascading.securecodebox.io/domain" + operator: "SubdomainOf" + values: ["{{attributes.hostname}}"] ``` diff --git a/docs/contributing/integrating-a-scanner/values.yaml.md b/docs/contributing/integrating-a-scanner/values.yaml.md index c4e03559..44884414 100644 --- a/docs/contributing/integrating-a-scanner/values.yaml.md +++ b/docs/contributing/integrating-a-scanner/values.yaml.md @@ -28,6 +28,9 @@ parser: # parser.env -- Optional environment variables mapped into each parseJob (see: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/) env: [] + # parser.scopeLimiterAliases -- Optional finding aliases to be used in the scopeLimiter. + scopeLimiterAliases: {} + # Do the same for the scanner containers scanner: image: @@ -140,3 +143,7 @@ Optional additional Containers started with the job (see: [Init Containers | Kub ### securityContext Optional securityContext set on the container (see: [Configure a Security Context for a Pod or Container | Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)). + +### scopeLimiterAliases + +Optional scopeLimiterAliases set on the parse definition (see [ScopeLimiterAliases](/docs/api/crds/parse-definition#scopelimiteraliases-optional)) diff --git a/docs/how-tos/scope.md b/docs/how-tos/scope.md new file mode 100644 index 00000000..9a978c6e --- /dev/null +++ b/docs/how-tos/scope.md @@ -0,0 +1,186 @@ +--- +# SPDX-FileCopyrightText: 2021 iteratec GmbH +# +# SPDX-License-Identifier: Apache-2.0 + +title: "Enforcing Engagement Scope" +--- + +## Introduction + +In this step-by-step tutorial, we will go through all the required stages to set up engagement scope enforcement with secureCodeBox. +In this example, we are going to set up Amass and Nmap and run a scan with rules set-up so that we don't scan domains which are out-of-scope. + +## Setup + +For the sake of the tutorial, we assume that you have your Kubernetes cluster already up and running and that we can work in your default namespace. +If not, check out the [installation](/docs/getting-started/installation/) for more information. +We also assume that you have the latest version of cascading scans installed. + +We will start by installing Amass and Nmap: +```bash +helm upgrade --install amass secureCodeBox/amass +Release "amass" does not exist. Installing it now. +[...] +helm upgrade --install nmap secureCodeBox/nmap +Release "nmap" does not exist. Installing it now. +[...] +``` + +## Start Scan With Scope Limit + +Next, we can start creating our scan definition. +Let's assume that we would like amass to scan `nmap.org` for subdomains, but we would only like to run nmap on `scanme.nmap.org`. +In the example below, we only cascade if our `scope.cascading.securecodebox.io/domain` equals `{{attributes.name}}`. + +```yaml +apiVersion: "execution.securecodebox.io/v1" +kind: Scan +metadata: + name: "amass-scan" + annotations: + scope.cascading.securecodebox.io/domain: "scanme.nmap.org" +spec: + cascades: + scopeLimiter: + allOf: + - key: "scope.cascading.securecodebox.io/domain" + operator: "In" + values: ["{{attributes.name}}"] + scanType: "amass" + parameters: + - "-d" + - "nmap.org" +``` + +:::info +See the [CRD specification](/docs/api/crds/scan#scopelimiter-optional) for the specific syntax and allowed values. +::: + +Running this scan results in the following state: + +```shell +$ kubectl get scans +amass-scan amass Done 65 +nmap-scan-nmap-hostscan-86f2m nmap Done 6 +$ kubectl get pods +cascading-scans-amass-scan-8cssx--1-hh2w6 0/1 Completed 0 2m51s +cascading-scans-nmap-scan-nmap-hostscan-86f2m-4mljv--1-7zgm2 0/1 Completed 0 2m23s +parse-amass-scan-zfhxg--1-shwkj 0/1 Completed 0 2m54s +parse-nmap-scan-nmap-hostscan-86f2m-d6jh6--1-ztgls 0/1 Completed 0 2m26s +scan-amass-scan-79h5f--1-vk7v5 0/2 Completed 0 4m22s +scan-nmap-scan-nmap-hostscan-86f2m-54pkd--1-pfpp6 0/2 Completed 0 2m48s +``` + +As you can see, amass found 65 domain names, but only a single nmap scan was created. +In the cascading scans logs, you will see that lots of rules were not triggered as the domain was out of scope. + +```shell +$ kubectl logs cascading-scans-amass-scan-8cssx--1-hh2w6 +Starting hook for Scan "amass-scan" +Fetched 65 findings from the file storage +Fetching CascadingScans using LabelSelector: "" +Fetched 2 CascadingRules +Cascading Rule nmap-hostscan not triggered as scope limiter did not pass +[...] +``` + +## Handle Differences in Finding Formats + +In many cases, you are cascading to more than one scanner. +Unfortunately, not every scanner has the same finding format, making a scope as defined above more tricky. + +As an example, let's say you want to set up Nikto as a scanner. +This scanner cascades on Nmap's open port finding and uses `$.hostOrIp` to start the scan. +In some cases, nmap can return a hostname different to the original hostname, thus with scope rules we want to check for this. +Preferably, we would like to use the same rule as above. +Unfortunately, nmap gives its hostname in `attributes.hostname` instead of the defined `attributes.name` (Amass finding). +This results in the scope rule failing, and prevents Nikto from getting cascaded. + +To solve this situation, you have two options: + +### Option 1: Enable `validOnMissingRender` + +Enabling this option causes all defined rules which contain unresolved mustache templates to result in `true`. You can use this if you're sure that Nmap returns the valid hostname. + +Example: + +```yaml +apiVersion: "execution.securecodebox.io/v1" +kind: Scan +metadata: + name: "amass-scan" + annotations: + scope.cascading.securecodebox.io/domain: "scanme.nmap.org" +spec: + cascades: + // highlight-next-line + validOnMissingRender: true + scopeLimiter: + allOf: + - key: "scope.cascading.securecodebox.io/domain" + operator: "In" + values: ["{{attributes.name}}"] + scanType: "amass" + parameters: + - "-d" + - "nmap.org" +``` + +### Option 2: Create a Hostname Alias for All Deployed Scanners + +A more fool-proof solution is to ensure that the hostname field is available in the same place for each scanner. +When deploying your scanner, you can define `scopeLimiterAliases`. + +```shell +$ helm upgrade --install amass secureCodeBox/amass \ + --set="parser.scopeLimiterAliases.hostname=\{\{attributes.name\}\}" +Release "amass" has been upgraded. Happy Helming! +[...] +$ helm upgrade --install nmap secureCodeBox/nmap \ + --set="parser.scopeLimiterAliases.hostname=\{\{attributes.hostname\}\}" +Release "nmap" has been upgraded. Happy Helming! +[...] +``` + +The aliases are added to the scanner's parse definition: + +```yaml +apiVersion: execution.securecodebox.io/v1 +kind: ParseDefinition +metadata: + name: amass-jsonl + namespace: default +spec: + env: [] + image: docker.io/securecodebox/parser-amass:3.5.0 + imagePullPolicy: IfNotPresent + scopeLimiterAliases: + hostname: "{{attributes.name}}" + ttlSecondsAfterFinished: null +``` + +To use your custom alias, you reference it with `$.hostname` in `scopeLimiter` in your scan definition. + +```yaml +scopeLimiter: + allOf: + - key: "scope.cascading.securecodebox.io/domain" + operator: "In" + values: ["{{$.hostname}}"] +``` + +Running this scan inside the cluster runs Amass, Nmap, and Nikto as expected. + +```shell +$ kubectl get scans +NAME TYPE STATE FINDINGS +amass-scan amass Done 65 +nikto-scan-nmap-hostscan-rhhqz-nikto-http-ps8cl nikto Done 6 +nmap-scan-nmap-hostscan-rhhqz nmap Done 6 +``` + +:::caution +The scope limiter only applies to cascading scans. +If your initial scan is out-of-scope, it will still run. +::: diff --git a/scripts/utils/config.js b/scripts/utils/config.js index 8fd17574..f7c3a4d9 100644 --- a/scripts/utils/config.js +++ b/scripts/utils/config.js @@ -52,6 +52,7 @@ const docsConfig = { "how-tos/scanning-networks", "how-tos/scanning-web-applications", "how-tos/hooks", + "how-tos/scope", ], }, sidebarEnd: {