diff --git a/docs/community/contribute/checks/overview.md b/docs/community/contribute/checks/overview.md new file mode 100644 index 00000000000..5706c5a02a9 --- /dev/null +++ b/docs/community/contribute/checks/overview.md @@ -0,0 +1,131 @@ +# Contribute Rego Checks + +The following guide provides an overview of contributing checks to the default checks in Trivy. + +All of the checks in Trivy can be found in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository on GitHub. Before you begin writing a check, ensure: + +1. The check does not already exist as part of the default checks in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository. +2. The pull requests in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/pulls) repository to see whether someone else is already contributing the check that you wanted to add. +3. The [issues in Trivy](https://github.com/aquasecurity/trivy/issues) to see whether any specific checks are missing in Trivy that you can contribute. + +If anything is unclear, please [start a discussion](https://github.com/aquasecurity/trivy/discussions/new) and we'll do our best to help. + +## Check structure + +Checks are written in Rego and follow a particular structure in Trivy. Below is an example check for AWS: + +```rego +# METADATA +# title: "RDS IAM Database Authentication Disabled" +# description: "Ensure IAM Database Authentication is enabled for RDS database instances to manage database access" +# scope: package +# schemas: +# - input: schema["aws"] +# related_resources: +# - https://docs.aws.amazon.com/neptune/latest/userguide/iam-auth.html +# custom: +# id: AVD-AWS-0176 +# avd_id: AVD-AWS-0176 +# provider: aws +# service: rds +# severity: MEDIUM +# short_code: enable-iam-auth +# recommended_action: "Modify the PostgreSQL and MySQL type RDS instances to enable IAM database authentication." +# input: +# selector: +# - type: cloud +# subtypes: +# - service: rds +# provider: aws + +package builtin.aws.rds.aws0176 + +deny[res] { + instance := input.aws.rds.instances[_] + instance.engine.value == ["postgres", "mysql"][_] + not instance.iamauthenabled.value + res := result.new("Instance does not have IAM Authentication enabled", instance.iamauthenabled) +} +``` + +## Verify the provider and service exists + +Every check for a cloud service references a cloud provider. The list of providers are found in the [Trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. + +Before writing a new check for a cloud provider, you need to verify if the cloud provider or resource type that your check targets is supported by Trivy. If it's not, you'll need to add support for it. Additionally, if the provider that you want to target exists, you need to check whether the service your policy will target is supported. As a reference you can take a look at the AWS provider [here](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/aws.go). + +???+ note + New Kubernetes and Dockerfile checks do not require any additional provider definitions. You can find an example of a Dockerfile check [here](https://github.com/aquasecurity/trivy-policies/blob/main/checks/docker/add_instead_of_copy.rego) and a Kubernetes check [here](https://github.com/aquasecurity/trivy-policies/blob/main/checks/kubernetes/general/CPU_not_limited.rego). + +Before writing a new check, you need to verify if the provider your check targets is supported by Trivy. If the provider that you want to target exists, you need to check whether the service your policy will target is supported. If the service is not support, please refer to the documentation on adding support for a new service in an existing provider. + +### Add Support for a New Service in an existing Provider + +[Please reference the documentation on adding Support for a New Service](./service-support.md). + +This guide also showcases how to add new properties for an existing Service. + +## Create a new .rego file + +The following directory in the trivy-policies repository contains all of our custom checks. Depending on what type of check you want to create, you will need to nest a new `.rego` file in either of the [subdirectories](https://github.com/aquasecurity/trivy-policies/tree/main/checks): + +* cloud: All checks related to cloud providers and their services +* docker: Docker specific checks +* kubernetes: Kubernetes specific checks + +## Check Package name + +Have a look at the existing package names in the [built in checks](https://github.com/aquasecurity/trivy-policies/tree/main/checks). + +The package name should be in the format `builtin.PROVIDER.SERVICE.ID`, e.g. `builtin.aws.rds.aws0176`. + +## Generating an ID + +Every check has a custom ID that is referenced throughout the metadata of the check to uniquely identify the check. If you plan to contribue your check back into the [trivy-policies](https://github.com/aquasecurity/trivy-policies) repository, it will require a valid ID. + +Running `make id` in the root of the trivy-policies repository will provide you with the next available _ID_ for your rule. + +## Check Schemas + +Rego Checks for Trivy can utilise Schemas to map the input to specific objects. The schemas available are listed [here.](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). + +More information on using the builtin schemas is provided in the [main documentation.](../../../docs/scanner/misconfiguration/custom/schema.md) + +## Check Metadata + +The metadata is the top section that starts with `# METADATA`, and has to be placed on top of the check. You can copy and paste from another check as a starting point. This format is effectively _yaml_ within a Rego comment, and is [defined as part of Rego itself](https://www.openpolicyagent.org/docs/latest/policy-language/#metadata). + +For detailed information on each component of the Check Metadata, please refer to the [main documentation.](../../../docs/scanner/misconfiguration/custom/index.md) + +Note that while the Metadata is optional in your own custom checks for Trivy, if you are contributing your check to the Trivy builtin checks, the Metadata section will be required. + + +## Writing Rego Rules + +Rules are defined using _OPA Rego_. You can find a number of examples in the `checks` directory ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks)). The [OPA documentation](https://www.openpolicyagent.org/docs/latest/policy-language/) is a great place to start learning Rego. You can also check out the [Rego Playground](https://play.openpolicyagent.org/) to experiment with Rego, and [join the OPA Slack](https://slack.openpolicyagent.org/). + + +```rego +deny[res] { + instance := input.aws.rds.instances[_] + instance.engine.value == ["postgres", "mysql"][_] + not instance.iamauthenabled.value + res := result.new("Instance does not have IAM Authentication enabled", instance.iamauthenabled) +} +``` + +The rule should return a result, which can be created using `result.new`. This function does not need to be imported, it is defined internally and provided at runtime. The first argument is the message to display and the second argument is the resource that the issue was detected on. + +It is possible to pass any rego variable that references a field of the input document. + +## Generate docs + +Finally, you'll want to generate documentation for your newly added rule. Please run `make docs` in the [trivy-policies](https://github.com/aquasecurity/trivy-policies) directory to generate the documentation for your new policy and submit a PR for us to take a look at. + +## Adding Tests + +All Rego checks need to have tests. There are many examples of these in the `checks` directory for each check ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks)). More information on how to write tests for Rego checks is provided in the [custom misconfiguration](../../../docs/scanner/misconfiguration/custom/testing.md) section of the docs. + +## Example PR + +You can see a full example PR for a new rule being added here: [https://github.com/aquasecurity/defsec/pull/1000](https://github.com/aquasecurity/defsec/pull/1000). diff --git a/docs/community/contribute/checks/service-support.md b/docs/community/contribute/checks/service-support.md new file mode 100644 index 00000000000..91bcefe14da --- /dev/null +++ b/docs/community/contribute/checks/service-support.md @@ -0,0 +1,69 @@ +# Add Service Support + +A service refers to a service by a cloud provider. This section details how to add a new service to an existing provider. All contributions need to be made to the [trivy repository](https://github.com/aquasecurity/trivy/). + +## Prerequisites + +Before you begin, verify that the [provider](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) does not already have the service that you plan to add. + +## Adding a new service to an existing provider + +Adding a new service involves two steps. The service will need a data structure to store information about the required resources that will be scanned. Additionally, the service will require one or more adapters to convert the scan targetes as input(s) into the aforementioned data structure. + +### Create a new file in the provider directory + +In this example, we are adding the CodeBuild service to the AWS provider. + +First, create a new directory and file for your new service under the provider directory: e.g. [aws/codebuild/codebuild.go](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/codebuild/codebuild.go) + +The CodeBuild service will require a structure `struct` to hold the information on the input that is scanned. The input is the CodeBuild resource that a user configured and wants to scan for misconfiguration. + +``` +type CodeBuild struct { + Projects []Project +} +``` + +The CodeBuild service manages `Project` resources. The `Project` struct has been added to hold information about each Project resources; `Project` Resources in turn manage `ArtifactSettings`: + +``` +type Project struct { + Metadata iacTypes.Metadata + ArtifactSettings ArtifactSettings + SecondaryArtifactSettings []ArtifactSettings +} + +type ArtifactSettings struct { + Metadata iacTypes.Metadata + EncryptionEnabled iacTypes.BoolValue +} +``` + +The `iacTypes.Metadata` struct is embedded in all of the Trivy types and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined and the name of the resource. + +A resource in this example `Project` can have a name and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the trivy types `iacTypes.Metadata` and `iacTypes.BoolValue`. These types wrap the raw values and provide additional metadata about the value. For instance, whether it was set by the user and the file and line number where the resource was defined. + +Have a look at the other providers and services in the [`iac/providers`](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) directory in Trivy. + +Next you'll need to add a reference to your new service struct in the [provider struct](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/aws.go) at `pkg/iac/providers/aws/aws.go`: + +``` +type AWS struct { + ... + CodeBuild codebuild.CodeBuild + ... +} +``` + +### Update Adapters + +Now you'll need to update all of the [adapters](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/adapters) which populate the struct of the provider that you have been using. Following the example above, if you want to add support for CodeBuild in Terraform, you'll need to update the Terraform AWS adatper as shown here: [`trivy/pkg/iac/adapters/terraform/aws/codebuild/adapt.go`](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/adapters/terraform/aws/codebuild/adapt.go). + +Another example for updating the adapters is provided in the [following PR.](https://github.com/aquasecurity/defsec/pull/1000/files) Additionally, please refer to the respective Terraform documentation on the provider to which you are adding the service. For instance, the Terraform documentation for AWS CodeBuild is provided [here.](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project) + + +## Create a new Schema for your provider + +Once the new service has been added to the provider, you need to create the schema for the service as part of the provider schema. + +This process has been automated with mage commands. In the Trivy root directory run `mage schema:generate` to generate the schema for your new service and `mage schema:verify`. diff --git a/docs/docs/scanner/misconfiguration/custom/contribute-checks.md b/docs/docs/scanner/misconfiguration/custom/contribute-checks.md new file mode 100644 index 00000000000..037973fe4ad --- /dev/null +++ b/docs/docs/scanner/misconfiguration/custom/contribute-checks.md @@ -0,0 +1,9 @@ +## Contribute Rego Checks + +The contributing section provides detailed information on + +* how to contribute custom checks to the [trivy-policies repository](../../../../community/contribute/checks/overview.md/) +* add [new services](../../../../community/contribute/checks/service-support.md) to existing cloud providers +* add [new fields](../../../../community/contribute/checks/service-support.md) to an existing Service + +This way, they become accessible as default [policies.](https://github.com/aquasecurity/trivy-policies) \ No newline at end of file diff --git a/docs/docs/scanner/misconfiguration/custom/index.md b/docs/docs/scanner/misconfiguration/custom/index.md index 8b08c5e4129..0e75b1d76cb 100644 --- a/docs/docs/scanner/misconfiguration/custom/index.md +++ b/docs/docs/scanner/misconfiguration/custom/index.md @@ -61,9 +61,9 @@ If you add a new custom policy, it must be defined under a new package like `use ### Policy structure -`# METADATA` (optional) +`# METADATA` (optional unless the check will be contributed into Trivy) : - SHOULD be defined for clarity since these values will be displayed in the scan results - - `custom.input` SHOULD be set to indicate the input type the policy should be applied to. See [list of available types](https://github.com/aquasecurity/defsec/blob/418759b4dc97af25f30f32e0bd365be7984003a1/pkg/types/sources.go) + - `custom.input` SHOULD be set to indicate the input type the policy should be applied to. See [list of available types][source-types] `package` (required) : - MUST follow the Rego's [specification][package] @@ -80,7 +80,6 @@ If you add a new custom policy, it must be defined under a new package like `use - A `string` denoting the detected issue - Although `object` with `msg` field is accepted, other fields are dropped and `string` is recommended if `result.new()` is not utilised. - e.g. `{"msg": "deny message", "details": "something"}` - ### Package A package name must be unique per policy. @@ -119,21 +118,54 @@ Trivy supports extra fields in the `custom` section as described below. # - type: kubernetes ``` -All fields are optional. The `schemas` field should be used to enable policy validation using a built-in schema. The +If you are creating checks for your Trivy misconfiguration scans, some fields are optional as referenced in the table below. The `schemas` field should be used to enable policy validation using a built-in schema. The schema that will be used is based on the input document type. It is recommended to use this to ensure your policies are correct and do not reference incorrect properties/values. -| Field name | Allowed values | Default value | In table | In JSON | -|----------------------------|-------------------------------------------------------------------|:----------------------------:|:----------------:|:----------------:| -| title | Any characters | N/A | :material-check: | :material-check: | -| description | Any characters | | :material-close: | :material-check: | -| schemas.input | `schema["kubernetes"]`, `schema["dockerfile"]`, `schema["cloud"]` | (applied to all input types) | :material-close: | :material-close: | -| custom.id | Any characters | N/A | :material-check: | :material-check: | -| custom.severity | `LOW`, `MEDIUM`, `HIGH`, `CRITICAL` | UNKNOWN | :material-check: | :material-check: | -| custom.recommended_actions | Any characters | | :material-close: | :material-check: | -| custom.input.selector.type | Any item(s) in [this list][source-types] | | :material-close: | :material-check: | -| url | Any characters | | :material-close: | :material-check: | +| Field name | Allowed values | Description | Required in Trivy Check | Default value | In table | In JSON | +|----------------------------|-------------------------------------------------------------------|:-------------------------------------------------------:|:------------------------------:|:-----------------------------:|:----------------:|:----------------:| +| title | Any characters | Name of the policy | :material-check: | N/A | :material-check: | :material-check: | +| description | Any characters | Description of the problem | :material-check: | N/A | :material-close: | :material-check: | +| schemas.input | `schema["One of the available schemas"]` | Used to validate the policy for syntax errors | :material-close: | (applied to all input types) | :material-close: | :material-close: | +| scope | `package` | Defines the policy scope | :material-check: | (applied to all input types) | :material-close: | :material-check: | +| custom.id | Any characters | More information provided below | :material-check: | N/A | :material-close: | :material-check: | +| custom.avd_id | Any characters | More information provided below | :material-check: | N/A | :material-close: | :material-check: | +| custom.severity | `LOW`, `MEDIUM`, `HIGH`, `CRITICAL` | The severity of the misconfiguration | :material-check: | `MEDIUM` | :material-close: | :material-check: | +| custom.provider | Any provider available in Trivy | The resource provider this check relates to | :material-close: | N/A | :material-close: | :material-check: | +| custom.service | The service name of the provider | One of the services available in the provider | :material-close: | N/A | :material-close: | :material-check: | +| custom.short_code | Any characters | Descriptive name for the check | :material-check: | N/A | :material-close: | :material-check: | +| custom.subtypes | Cloud Resource Subtypes | Refer to the section on subtypes below | :material-close: | N/A | :material-close: | :material-check: | +| custom.recommended_actions | Any characters | Describing what the user should do to resolve the issue | :material-check: | N/A | :material-close: | :material-check: | +| custom.input.selector.type | Any item(s) in [this list][source-types] | More information provided below | :material-check: | N/A | :material-close: | :material-check: | +| related_resource | Any characters | URL to related resources | :material-check: | N/A | :material-close: | :material-check: | + +#### schemas.input + +The input schema is required for validating the checks for functional correctness (syntax). Any of the [schemas](./schema.md) can be referenced as the input. + +#### custom.avd_id and custom.id + +The AVD_ID can be used to link the check to the Aqua Vulnerability Database (AVD) entry. For example, the `avd_id` `AVD-AWS-0176` is the ID of the check in the [AWS Vulnerability Database](https://avd.aquasec.com/). If you are [contributing your check to trivy-policies](../../../../community/contribute/checks/overview.md), you need to generate an ID using `make id` in the [trivy-policies](https://github.com/aquasecurity/trivy-policies) repository. The output of the command will provide you the next free IDs for the different providers in Trivy. + +The ID is based on the AVD_ID. For instance if the `avd_id` is `AVD-AWS-0176`, the ID is `ID0176`. + +#### custom.provider + +The `provider` field references the [provider](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) available in Trivy. This should be the same as the provider name in the `pkg/iac/providers` directory, e.g. `aws`. + +#### custom.service + +Services are defined within a provider. For instance, RDS is a service and AWS is a provider. This should be the same as the service name in one of the provider directories. ([Link](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers)), e.g. `aws/rds`. + +#### custom.input + +The `input` tells Trivy what inputs this check should be applied to. Cloud provider checks should always use the `selector` input, and should always use the `type` selector with `cloud`. Check targeting Kubernetes yaml can use `kubenetes`, RBAC can use `rbac`, and so on. + +#### Subtypes in the custom data + +Subtypes currently only need to be defined for cloud providers [as detailed in the documentation.](./selectors.md/#enabling-selectors-and-subtypes) +#### Scan Result Some fields are displayed in scan results. @@ -181,7 +213,7 @@ You can specify input format via the `custom.input` annotation. - `dockerfile` (Dockerfile) - `kubernetes` (Kubernetes YAML/JSON) - `rbac` (Kubernetes RBAC YAML/JSON) - - `cloud` (Cloud format, as defined by defsec - this is used for Terraform, CloudFormation, and Cloud/AWS scanning) + - `cloud` (Cloud format, as defined by Trivy - this is used for Terraform, CloudFormation, and Cloud/AWS scanning) - `yaml` (Generic YAML) - `json` (Generic JSON) - `toml` (Generic TOML) @@ -200,4 +232,4 @@ See [here](schema.md) for the detail. [rego]: https://www.openpolicyagent.org/docs/latest/policy-language/ [package]: https://www.openpolicyagent.org/docs/latest/policy-language/#packages -[source-types]: https://github.com/aquasecurity/defsec/blob/418759b4dc97af25f30f32e0bd365be7984003a1/pkg/types/sources.go +[source-types]: https://github.com/aquasecurity/trivy/blob/9361cdb7e28fd304d6fd2a1091feac64a6786672/pkg/iac/types/sources.go#L4 diff --git a/docs/docs/scanner/misconfiguration/custom/schema.md b/docs/docs/scanner/misconfiguration/custom/schema.md index ea5efeb7b3e..2af36af2887 100644 --- a/docs/docs/scanner/misconfiguration/custom/schema.md +++ b/docs/docs/scanner/misconfiguration/custom/schema.md @@ -4,7 +4,8 @@ Policies can be defined with custom schemas that allow inputs to be verified against them. Adding a policy schema enables Trivy to show more detailed error messages when an invalid input is encountered. -In Trivy we have been able to define a schema for a [Dockerfile](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/rego/schemas/dockerfile.json). Without input schemas, a policy would be as follows: +In Trivy we have been able to define a schema for a [Dockerfile](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas) +Without input schemas, a policy would be as follows: !!! example ``` diff --git a/docs/tutorials/misconfiguration/custom-checks.md b/docs/tutorials/misconfiguration/custom-checks.md index 97ab67a7f64..87201477b81 100644 --- a/docs/tutorials/misconfiguration/custom-checks.md +++ b/docs/tutorials/misconfiguration/custom-checks.md @@ -9,7 +9,8 @@ When you are writing a check, it's important to understand the input to the chec Since Rego is primarily tailored to query JSON objects, all incoming configuration files needs to be first converted to structured objects, which is available to the Rego code as the input variable. This is nothing that users have to do manually in Trivy. Instead, Rego makes it possible to pass in custom Schemas that detail how files are converted. Once Rego has access to a custom Schema, it will know in which format to access configuration files such as a Dockerfile. [Here you can find the schemas](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas) that define how different configuration files are converted to JSON by Trivy. -This tutorial will make use of the [dockerfile.json schema](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/rego/schemas/dockerfile.json). The schema will need to be parsed into your custom check. + +This tutorial will make use of the [dockerfile.json schema](https://github.com/aquasecurity/defsec/tree/master/pkg/rego/schemas). The schema will need to be parsed into your custom check. Users can also use the [Schema Explorer](https://aquasecurity.github.io/trivy-schemas/) to view the structure of the data provided to Rego. diff --git a/mkdocs.yml b/mkdocs.yml index c3437fecf41..8d96af75035 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -65,6 +65,7 @@ nav: - Schemas: docs/scanner/misconfiguration/custom/schema.md - Testing: docs/scanner/misconfiguration/custom/testing.md - Debugging Policies: docs/scanner/misconfiguration/custom/debug.md + - Contribute Checks: docs/scanner/misconfiguration/custom/contribute-checks.md - Secret: docs/scanner/secret.md - License: docs/scanner/license.md - Coverage: @@ -185,6 +186,9 @@ nav: - Issues: community/contribute/issue.md - Discussions: community/contribute/discussion.md - Pull Requests: community/contribute/pr.md + - Contribute Rego Checks: + - Overview: community/contribute/checks/overview.md + - Add Service Supprot: community/contribute/checks/service-support.md - Maintainer: - Help Wanted: community/maintainer/help-wanted.md - Triage: community/maintainer/triage.md