From 5110d8701107279abda5b5156370c8408097a935 Mon Sep 17 00:00:00 2001 From: Steven Smith Date: Thu, 2 Mar 2023 13:32:59 -0500 Subject: [PATCH] Updates clair book to add configuration fields --- clair/master.adoc | 14 ++- modules/config-fields-clair-auth.adoc | 18 ++++ modules/config-fields-clair-indexer.adoc | 35 +++++++ modules/config-fields-clair-matcher.adoc | 44 +++++++++ modules/config-fields-clair-matchers.adoc | 32 ++++++ modules/config-fields-clair-metrics.adoc | 18 ++++ modules/config-fields-clair-notifiers.adoc | 107 +++++++++++++++++++++ modules/config-fields-clair-trace.adoc | 36 +++++++ modules/config-fields-clair-updaters.adoc | 31 ++++++ modules/config-fields-general-clair.adoc | 21 ++++ modules/config-fields-overview.adoc | 97 +++++++++++++++++++ 11 files changed, 452 insertions(+), 1 deletion(-) create mode 100644 modules/config-fields-clair-auth.adoc create mode 100644 modules/config-fields-clair-indexer.adoc create mode 100644 modules/config-fields-clair-matcher.adoc create mode 100644 modules/config-fields-clair-matchers.adoc create mode 100644 modules/config-fields-clair-metrics.adoc create mode 100644 modules/config-fields-clair-notifiers.adoc create mode 100644 modules/config-fields-clair-trace.adoc create mode 100644 modules/config-fields-clair-updaters.adoc create mode 100644 modules/config-fields-general-clair.adoc create mode 100644 modules/config-fields-overview.adoc diff --git a/clair/master.adoc b/clair/master.adoc index c303b7bd2..d3751de3f 100644 --- a/clair/master.adoc +++ b/clair/master.adoc @@ -67,4 +67,16 @@ include::modules/clair-openshift-airgap-database-standalone.adoc[leveloffset=+3] include::modules/clair-openshift-airgap-import-bundle-standalone.adoc[leveloffset=+3] include::modules/clair-crda-configuration.adoc[leveloffset=+2] -include::modules/mapping-repositories-to-cpe-information.adoc[leveloffset=+2] \ No newline at end of file +include::modules/mapping-repositories-to-cpe-information.adoc[leveloffset=+2] + + +include::modules/config-fields-overview.adoc[leveloffset=+1] +include::modules/config-fields-general-clair.adoc[leveloffset=+2] +include::modules/config-fields-clair-indexer.adoc[leveloffset=+2] +include::modules/config-fields-clair-matcher.adoc[leveloffset=+2] +include::modules/config-fields-clair-matchers.adoc[leveloffset=+2] +include::modules/config-fields-clair-updaters.adoc[leveloffset=+2] +include::modules/config-fields-clair-notifiers.adoc[leveloffset=+2] +include::modules/config-fields-clair-auth.adoc[leveloffset=+2] +include::modules/config-fields-clair-trace.adoc[leveloffset=+2] +include::modules/config-fields-clair-metrics.adoc[leveloffset=+2] \ No newline at end of file diff --git a/modules/config-fields-clair-auth.adoc b/modules/config-fields-clair-auth.adoc new file mode 100644 index 000000000..131092576 --- /dev/null +++ b/modules/config-fields-clair-auth.adoc @@ -0,0 +1,18 @@ +:_content-type: CONCEPT +[id="config-fields-clair-auth"] += Clair authorization configuration fields + +The following authorization configuration fields are available for Clair. + +[cols="3a,1a,2a",options="header"] + +|=== +| Field | Type | Description +| **auth** | Object | Defines Clair's external and intra-service JWT based authentication. If multiple `auth` mechanisms are defined, Clair picks one. Currently, multiple mechanisms are unsupported. + +| **.psk** | String | Defines pre-shared key authentication. + +| **.psk.key** | String | A shared base64 encoded key distributed between all parties signing and verifying JWTs. + +| **.psk.iss** | String | A list of JWT issuers to verify. An empty list accepts any issuer in a JWT claim. +|=== diff --git a/modules/config-fields-clair-indexer.adoc b/modules/config-fields-clair-indexer.adoc new file mode 100644 index 000000000..08f2f05b1 --- /dev/null +++ b/modules/config-fields-clair-indexer.adoc @@ -0,0 +1,35 @@ +:_content-type: CONCEPT +[id="config-fields-clair-indexer"] += Clair indexer configuration fields + +The following indexer configuration fields are available for Clair. + +[cols="3a,1a,2a",options="header"] +|=== +| Field | Type | Description +| **indexer** | Object | Provides Clair indexer node configuration. + +| **.airgap** | Boolean | Disables HTTP access to the internet for indexers and fetchers. Private IPv4 and IPv6 addresses are allowed. Database connections are unaffected. + +| **.connstring** | String | A Postgres connection string. Accepts format as a URL or libpq connection string. + +| **.index_report_request_concurrency** | Integer | Rate limits the number of index report creation requests. Setting this to `0` attemps to auto-size this value. Setting a negative value means unlimited. The auto-sizing is a multiple of the number of available cores. + +The API returns a `429` status code if concurrency is exceeded. + +| **.scanlock_retry** | Integer | A positive integer representing seconds. Concurrent indexers lock on manifest scans to avoid clobbering. This value tunes how often a waiting indexer polls for the lock. + +| **.layer_scan_concurrency** | Integer | Positive integer limiting the number of concurrent layer scans. Indexers will match a manifest's layer concurrently. This value tunes the number of layers an indexer scans in parallel. + +| **.migrations** | Boolean | Whether indexer nodes handle migrations to their database. + +| **.scanner** | String | Indexer configuration. + +Scanner allows for passing configuration options to layer scanners. The scanner will have this configuration pass to it on construction if designed to do so. + +| **.scanner.dist** | String | A map with the name of a particular scanner and arbitrary YAML as a value. + +| **.scanner.package** | String | A map with the name of a particular scanner and arbitrary YAML as a value. + +| **.scanner.repo** | String | A map with the name of a particular scanner and arbitrary YAML as a value. +|=== \ No newline at end of file diff --git a/modules/config-fields-clair-matcher.adoc b/modules/config-fields-clair-matcher.adoc new file mode 100644 index 000000000..fd8668d27 --- /dev/null +++ b/modules/config-fields-clair-matcher.adoc @@ -0,0 +1,44 @@ +:_content-type: CONCEPT +[id="config-fields-clair-matcher"] += Clair matcher configuration fields + +The following matcher configuration fields are available for Clair. + +[NOTE] +==== +Differs from `matchers` configuration fields. +==== + +[cols="3a,1a,2a",options="header"] +|=== +| Field | Type | Description +| **matcher** | Object | Provides Clair matcher node configuration. + +| **.cache_age** | String | Controls how long users should be hinted to cache responses for. + +| **.connstring** | String | A Postgres connection string. Accepts format as a URL or libpq connection string. + +| **.max_conn_pool** | Integer | Limits the database connection pool size. + +Clair allows for a custom connection pool size. This number directly sets how many active database connections are allowed concurrently. + +This parameter will be ignored in a future version. Users should configure this through the connection string. + +| **.indexer_addr** | String | A matcher contacts an indexer to create a `VulnerabilityReport`. The location of this indexer is required. + +Defaults to `30m`. + +| **.migrations** | Boolean | Whether matcher nodes handle migrations to their databases. + +| **.period** | String | Determines how often updates for new security advisories take place. + +Defaults to `30m`. + +| **.disable_updaters** | Boolean | Whether to run background updates or not. + +| **.update_retention** | Integer | Sets the number of update operations to retain between garbage collection cycles. This should be set to a safe MAX value based on database size constraints. + +Defaults to `10m`. + +If a value of less than `0` is provided, garbage collection is disabled. `2` is the minimum value to ensure updates can be compared to notifications. +|=== \ No newline at end of file diff --git a/modules/config-fields-clair-matchers.adoc b/modules/config-fields-clair-matchers.adoc new file mode 100644 index 000000000..ab614504b --- /dev/null +++ b/modules/config-fields-clair-matchers.adoc @@ -0,0 +1,32 @@ +:_content-type: CONCEPT +[id="config-fields-clair-matchers"] += Clair matchers configuration fields + +The following matchers configuration fields are available for Clair. + +[NOTE] +==== +Differs from `matcher` configuration fields. +==== + +[cols="3a,1a,2a",options="header"] +|=== +| Field | Type | Description +| **matchers** | Array of strings | Provides configuration for the in-tree `matchers` and `remotematchers`. + +| **.names** | String | A list of string values informing the matcher factory about enabled matchers. If value is set to `null`, the default list of matchers run: +*alpine*, *aws*, *debian*, *oracle*, *photon*, *python*, *python*, *rhel*, *suse*, *ubuntu*, *crda* + +| **.config** | String | Provides configuration to a specific matcher. + +A map keyed by the name of the matcher containing a sub-object which will be provided to the matchers factory constructor. For example: + +[source,yaml] +---- +config: + python: + ignore_vulns: + - CVE-XYZ + - CVE-ABC +---- +|=== \ No newline at end of file diff --git a/modules/config-fields-clair-metrics.adoc b/modules/config-fields-clair-metrics.adoc new file mode 100644 index 000000000..983fd4cef --- /dev/null +++ b/modules/config-fields-clair-metrics.adoc @@ -0,0 +1,18 @@ +:_content-type: CONCEPT +[id="config-fields-clair-metrics"] += Clair metrics configuration fields + +The following metrics configuration fields are available for Clair. + +[cols="3a,1a,2a",options="header"] + +|=== +| Field | Type | Description +| **metrics** | Object | Defines distributed tracing configuration based on OpenTelemetry. + +| **.name** | String | The name of the metrics in use. + +| **.prometheus** | String | Configuration for a Prometheus metrics exporter. + +| **.prometheus.endpoint** | String | Defines the path where metrics are served. +|=== \ No newline at end of file diff --git a/modules/config-fields-clair-notifiers.adoc b/modules/config-fields-clair-notifiers.adoc new file mode 100644 index 000000000..eccfd2e6c --- /dev/null +++ b/modules/config-fields-clair-notifiers.adoc @@ -0,0 +1,107 @@ +:_content-type: CONCEPT +[id="config-fields-clair-notifiers"] += Clair notifier configuration fields + +The following notifier configuration fields are available for Clair. + +[cols="3a,1a,2a",options="header"] + +|=== +| Field | Type | Description +| **notifier** | Object | Provides Clair notifier node configuration. + +| **.connstring** | String | Postgres connection string. Accepts format as URL, or libpq connection string. + +| **.migrations** | Boolean | Whether notifier nodes handle migrations to their database. + +| **.indexer_addr** | String | A notifier contacts an indexer to create or obtain manifests affected by vulnerabilities. The location of this indexer is required. + +| **.matcher_addr** | String | A notifier contacts a matcher to list update operations and acquire diffs. The location of this matcher is required. + +| **.poll_interval** | String | The frequency at which the notifier will query a matcher for update operations. + +| **.delivery_interval** | String | The frequency at which the notifier attempts delivery of created, or previously failed, notifications. + +| **.disable_summary** | Boolean | Controls whether notifications should be summarized to one per manifest. + +| **.webhook** | Object | Configures the notifier for webhook delivery. + +| **.webhook.target** | String | URL where the webhook will be delivered. + +| **.webhook.callback** | String | The callback URL where notifications can be retrieved. The notification ID will be appended to this URL. + +This will typically be where the Clair notifier is hosted. + +| **.webhook.headers** | String | A map associating a header name to a list of values. + +| **.amqp** | Object | Configures the notifier for AMQP delivery. + +[NOTE] +==== +Clair does not declare any AMQP components on its own. All attempts to use an exchange or queue are passive only and will fail. Broker administrators should setup exchanges and queues ahead of time. +==== + +| **.amqp.direct** | Boolean | If `true`, the notifier will deliver individual notifications (not a callback) to the configured AMQP broker. + +| **.amqp.rollup** | Integer | When `amqp.direct` is set to `true`, this value informs the notifier of how many notifications to send in a direct delivery. For example, if `direct` is set to `true`, and `amqp.rollup` is set to `5`, the notifier delivers no more than 5 notifications in a single JSON payload to the broker. Setting the value to `0` effectively sets it to `1`. + +| **.amqp.exchange** | Object | The AMQP exchange to connect to. + +| **.amqp.exchange.name** | String | The name of the exchange to connect to. + +| **.amqp.exchange.type** | String | The type of the exchange. Typically one of the following: *direct*, *fanout*, *topic*, *headers*. + +| **.amqp.exchange.durability** | Boolean | Whether the configured queue is durable. + +| **.amqp.exchange.auto_delete** | Boolean | Whether the configured queue uses an `auto_delete_policy`. + +| **.amqp.routing_key** | String | The name of the routing key each notification is sent with. + +| **.amqp.callback** | String | If `amqp.direct` is set to `false`, this URL is provided in the notification callback sent to the broker. This URL should point to Clair's notification API endpoint. + +| **.amqp.uris** | String | A list of one or more AMQP brokers to connect to, in priority order. + +| **.amqp.tls** | Object | Configures TLS/SSL connection to an AMQP broker. + +| **.amqp.tls.root_ca** | String | The filesystem path where a root CA can be read. + +| **.amqp.tls.cert** | String | The filesystem path where a TLS/SSL certificate can be read. + +[NOTE] +==== +Clair also allows `SSL_CERT_DIR`, as documented for the Go `crypto/x509` package. +==== + +| **.amqp.tls.key** | String | The filesystem path where a TLS/SSL private key can be read. + +| **.stomp** | Object | Configures the notifier for STOMP delivery. + +| **.stomp.direct** | Boolean | If `true`, the notifier delivers individual notifications (not a callback) to the configured STOMP broker. + +| **.stomp.rollup** | Integer | If `stomp.direct` is set to `true`, this value limits the number of notifications sent in a single direct delivery. For example, if `direct` is set to `true`, and `rollup` is set to `5`, the notifier delivers no more than 5 notifications in a single JSON payload to the broker. Setting the value to `0` effectively sets it to `1`. + +| **.stomp.callback** | String | If `stomp.callback` is set to `false`, the provided URL in the notification callback is sent to the broker. This URL should point to Clair's notification API endpoint. + +| **.stomp.destination** | String | The STOMP destination to deliver notifications to. + +| **.stomp.uris** | String | A list of one or more STOMP brokers to connect to in priority order. + +| **.stomp.tls** | Object | Configured TLS/SSL connection to STOMP broker. + +| **.stomp.tls.root_ca** | String | The filesystem path where a root CA can be read. + +[NOTE] +==== +Clair also respects `SSL_CERT_DIR`, as documented for the Go `crypto/x509` package. +==== + +| **.stomp.tls.cert** | String | The filesystem path where a TLS/SSL certificate can be read. + +| **.stomp.tls.key** | String | The filesystem path where a TLS/SSL private key can be read. + +| **.stomp.user** | String | Configures login details for the STOMP broker. + +| **.stomp.user.login** | String | The STOMP login to connect with. + +| **.stomp.user.passcode** | String | The STOMP passcode to connect with. +|=== \ No newline at end of file diff --git a/modules/config-fields-clair-trace.adoc b/modules/config-fields-clair-trace.adoc new file mode 100644 index 000000000..499eb7e74 --- /dev/null +++ b/modules/config-fields-clair-trace.adoc @@ -0,0 +1,36 @@ +:_content-type: CONCEPT +[id="config-fields-clair-trace"] += Clair trace configuration fields + +The following trace configuration fields are available for Clair. + +[cols="3a,1a,2a",options="header"] + +|=== +| Field | Type | Description +| **trace** | Object | Defines distributed tracing configuration based on OpenTelemetry. + +| **.name** | String | The name of the application traces will belong to. + +| **.probability** | Integer | The probability a trace will occur. + +| **.jaeger** | Object | Defines values for Jaeger tracing. + +| **.jaeger.agent** | Object | Defines values for configuring delivery to a Jaeger agent. + +| **.jaeger.agent.endpoint** | String | An address in the `:` syntax where traces can be submitted. + +| **.jaeger.collector** | Object | Defines values for configuring delivery to a Jaeger collector. + +| **.jaeger.collector.endpoint** | String | An address in the `:` syntax where traces can be submitted. + +| **.jaeger.collector.username** | String | A Jaeger username. + +| **.jaeger.collector.password** | String | A Jaeger password. + +| **.jaeger.service_name** | String | The service name registered in Jaeger. + +| **.jaeger.tags** | String | Key-value pairs to provide additional metadata. + +| **.jaeger.buffer_max** | Integer | The maximum number of spans that can be buffered in memory before they are sent to the Jaeger backend for storage and analysis. +|=== \ No newline at end of file diff --git a/modules/config-fields-clair-updaters.adoc b/modules/config-fields-clair-updaters.adoc new file mode 100644 index 000000000..fcd0e6d8c --- /dev/null +++ b/modules/config-fields-clair-updaters.adoc @@ -0,0 +1,31 @@ +:_content-type: CONCEPT +[id="config-fields-clair-updaters"] += Clair updaters configuration fields + +The following updaters configuration fields are available for Clair. + +[cols="3a,1a,2a",options="header"] + +|=== +| Field | Type | Description +| **updaters** | Object | Provides configuration for the matcher's update manager. + +| **.sets** | String | A list of values informing the update manager which updaters to run. + +If value is set to `null`, the default set of updaters runs the following: *alpine*, *aws*, *debian*, *oracle*, *photon*, *pyupio*, *rhel*, *suse*, *ubuntu* + +If left blank, zero updaters run. + +| **.config** | String | Provides configuration to specific updater sets. + +A map keyed by the name of the updater set containing a sub-object which will be provided to the updater set's constructor. For example: + +[source,yaml] +---- +config: + ubuntu: + security_tracker_url: http://security.url + ignore_distributions: + - cosmic +---- +|=== \ No newline at end of file diff --git a/modules/config-fields-general-clair.adoc b/modules/config-fields-general-clair.adoc new file mode 100644 index 000000000..24e2ccafd --- /dev/null +++ b/modules/config-fields-general-clair.adoc @@ -0,0 +1,21 @@ +:_content-type: CONCEPT +[id="config-fields-required-clair"] += Clair general fields + +The following section describes the general configuration fields available for a Clair deployment: + +[cols="3a,1a,2a",options="header"] +|=== +| Field | Typhttp_listen_ae | Description +| **http_listen_addr** | String | Configures where the HTTP API is exposed. + +Default: `:6060` + +| **introspection_addr** | String | Configures where Clair's metrics and health endpoints are exposed. + +| **log_level** | String | Sets the logging level. Requires one of the following strings: *debug-color*, *debug*, *info*, *warn*, *error*, *fatal*, *panic* + +| **tls** | String | A map containing the configuration for serving the HTTP API of TLS/SSL and HTTP/2. + +| **.cert** | String | The TLS certificate to be used. Must be a full-chain certificate. +|=== diff --git a/modules/config-fields-overview.adoc b/modules/config-fields-overview.adoc new file mode 100644 index 000000000..12fb2947a --- /dev/null +++ b/modules/config-fields-overview.adoc @@ -0,0 +1,97 @@ +:_content-type: CONCEPT +[id="config-fields-overview"] += Clair configuration overview + +Clair is configured by a structured YAML file. Each Clair node needs to specify what mode it will run in and a path to a configuration file through CLI flags or environment variables. For example: + +[source,terminal] +---- +$ clair -conf ./path/to/config.yaml -mode indexer +---- + +or + +[source,terminal] +---- +$ clair -conf ./path/to/config.yaml -mode matcher +---- + +The aforementioned commands each start two Clair nodes using the same configuration file. One runs the indexing facilities, while other runs the matching facilities. + +Environment variables respected by the Go standard library can be specified if needed, for example: + +* `HTTP_PROXY` +* `HTTPS_PROXY` +* `SSL_CERT_DIR` + +If you are running Clair in `combo` mode, you must supply the indexer, matcher, and notifier configuration blocks in the configuration. + +[id="config-fields-clair-reference"] +== Clair configuration reference + +The following YAML shows an example Clair configuration: + +[source,yaml] +---- +http_listen_addr: "" +introspection_addr: "" +log_level: "" +tls: {} +indexer: + connstring: "" + scanlock_retry: 0 + layer_scan_concurrency: 0 + migrations: false + scanner: {} + airgap: false +matcher: + connstring: "" + indexer_addr: "" + migrations: false + period: "" + disable_updaters: false + update_retention: 2 +matchers: + names: nil + config: nil +updaters: + sets: nil + config: nil +notifier: + connstring: "" + migrations: false + indexer_addr: "" + matcher_addr: "" + poll_interval: "" + delivery_interval: "" + disable_summary: false + webhook: null + amqp: null + stomp: null +auth: + psk: nil +trace: + name: "" + probability: null + jaeger: + agent: + endpoint: "" + collector: + endpoint: "" + username: null + password: null + service_name: "" + tags: nil + buffer_max: 0 +metrics: + name: "" + prometheus: + endpoint: null + dogstatsd: + url: "" +---- + +[NOTE] +==== +The above YAML file lists every key for completeness. Using this configuration file as-is will result in some options not having their defaults set normally. +==== \ No newline at end of file