Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(metrics-operator): support Cortex metrics provider #3361

Merged
merged 5 commits into from
Apr 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/actions/spelling/expect.txt
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ controlleroptions
controllerutil
coredns
CORS
cortex
coverpkg
coverprofile
cowsay
Expand Down
4 changes: 2 additions & 2 deletions .github/scripts/.helm-tests/default/result.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13614,8 +13614,8 @@ spec:
type: string
type:
description: Type represents the provider type. This can be one of
prometheus, thanos, dynatrace, datadog, dql.
pattern: prometheus|thanos|dynatrace|datadog|dql
cortex, datadog, dql, dynatrace, prometheus or thanos.
pattern: cortex|datadog|dql|dynatrace|prometheus|thanos
type: string
required:
- targetServer
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13750,8 +13750,8 @@ spec:
type: string
type:
description: Type represents the provider type. This can be one of
prometheus, thanos, dynatrace, datadog, dql.
pattern: prometheus|thanos|dynatrace|datadog|dql
cortex, datadog, dql, dynatrace, prometheus or thanos.
pattern: cortex|datadog|dql|dynatrace|prometheus|thanos
type: string
required:
- targetServer
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2589,8 +2589,8 @@ spec:
type: string
type:
description: Type represents the provider type. This can be one of
prometheus, thanos, dynatrace, datadog, dql.
pattern: prometheus|thanos|dynatrace|datadog|dql
cortex, datadog, dql, dynatrace, prometheus or thanos.
pattern: cortex|datadog|dql|dynatrace|prometheus|thanos
type: string
required:
- targetServer
Expand Down
4 changes: 2 additions & 2 deletions .github/scripts/.helm-tests/metrics-only/result.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2589,8 +2589,8 @@ spec:
type: string
type:
description: Type represents the provider type. This can be one of
prometheus, thanos, dynatrace, datadog, dql.
pattern: prometheus|thanos|dynatrace|datadog|dql
cortex, datadog, dql, dynatrace, prometheus or thanos.
pattern: cortex|datadog|dql|dynatrace|prometheus|thanos
type: string
required:
- targetServer
Expand Down
4 changes: 2 additions & 2 deletions .github/scripts/.helm-tests/metrics-with-certs/result.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2604,8 +2604,8 @@ spec:
type: string
type:
description: Type represents the provider type. This can be one of
prometheus, thanos, dynatrace, datadog, dql.
pattern: prometheus|thanos|dynatrace|datadog|dql
cortex, datadog, dql, dynatrace, prometheus or thanos.
pattern: cortex|datadog|dql|dynatrace|prometheus|thanos
type: string
required:
- targetServer
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/assets/crd/examples/yaml-synopsis.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ metadata:
name: <data-source-instance-name>
namespace: <namespace>
spec:
type: prometheus | thanos | dynatrace | dql | datadog
type: cortex | datadog | dql | dynatrace | prometheus | thanos
targetServer: "<data-source-url>"
secretKeyRef:
name: <secret-name>
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/components/metrics-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ of the application and infrastructure.

While Kubernetes has ways to extend its metrics APIs, there are limitations,
especially that they allow you to use only a single observability platform
such as Prometheus, Thanos, Dynatrace or Datadog.
such as Prometheus, Thanos, Cortex, Dynatrace or Datadog.
The Keptn Metrics Operator solves this problem
by providing a single entry point for
all your metrics data, regardless of its source.
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/contribute/software/add-new-metric-provider.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ The steps to create your own metrics provider are:
[line](https://github.com/keptn/lifecycle-toolkit/blob/main/metrics-operator/api/v1/keptnmetricsprovider_types.go#L29)
to look like this

`// +kubebuilder:validation:Pattern:=prometheus|thanos|dynatrace|datadog|dql|placeholder`.
`// +kubebuilder:validation:Pattern:=cortex|datadog|dql|dynatrace|prometheus|thanos|placeholder`.

In the metric-operator directory run `make generate manifests` to update the metrics-operator crd config
Then modify the helm chart and the helm chart crd validation to match the update in the metrics-operator crd config
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/core-concepts/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ The Keptn metrics feature extends the functionality of

* Handles observability data from multiple instances
of multiple observability solutions
– Prometheus, Thanos, Dynatrace, Datadog and others –
– Prometheus, Thanos, Cortex, Dynatrace, Datadog and others –
as well as data that comes directly from your cloud provider
such as AWS, Google, or Azure.

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/getting-started/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ such as whether a rollout is good, or whether to scale up or down.

Your observability data may come
from multiple observability solutions --
Prometheus, Thanos, Dynatrace, Datadog and others --
Prometheus, Thanos, Cortex, Dynatrace, Datadog and others --
or may be data that comes directly
from your cloud provider such as AWS, Google, or Azure.
The Keptn Metrics Server unifies and standardizes access to all this data.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/installation/k8s.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ Your cluster should include the following:
* At least one observability data provider such as
[Prometheus](https://prometheus.io/),
[Thanos](https://thanos.io/),
[Cortex](https://cortexmetrics.io/),
[Dynatrace](https://www.dynatrace.com/),
or [Datadog](https://www.datadoghq.com/);
you can use multiple instances of different data providers.
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/migrate/keptn/strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ Keptn v1
[SLIs](https://v1.keptn.sh/docs/1.0.x/reference/files/sli/)
(Service Level Indicators)
represent queries from the data provider
such as Prometheus, Thanos, Dynatrace, or Datadog,
such as Prometheus, Thanos, Cortex, Dynatrace, or Datadog,
which is configured as a Keptn integration.

When migrating to Keptn, you need to define a
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/reference/api-reference/metrics/v1/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -373,7 +373,7 @@ _Appears in:_

| Field | Description | Default | Optional |Validation |
| --- | --- | --- | --- | --- |
| `type` _string_ | Type represents the provider type. This can be one of prometheus, thanos, dynatrace, datadog, dql. || x | Optional: {} <br />Pattern: `prometheus|thanos|dynatrace|datadog|dql` <br /> |
| `type` _string_ | Type represents the provider type. This can be one of cortex, datadog, dql, dynatrace, prometheus or thanos. || x | Optional: {} <br />Pattern: `cortex|datadog|dql|dynatrace|prometheus|thanos` <br /> |
| `targetServer` _string_ | TargetServer defines URL (including port and protocol) at which the metrics provider is reachable. || x | |
| `secretKeyRef` _[SecretKeySelector](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#secretkeyselector-v1-core)_ | SecretKeyRef defines an optional secret for access credentials to the metrics provider. || ✓ | Optional: {} <br /> |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ metadata:
namespace: <namespace-where-this-resource-resides>
spec:
provider:
name: prometheus | thanos | dynatrace | dql | datadog
name: cortex | datadog | dql | dynatrace | prometheus | thanos
query: <query>
```

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/reference/crd-reference/evaluationdefinition.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,12 +116,12 @@ that are now taken from the specified [KeptnMetric](metric.md) CRD.
The synopsis was:

```yaml
apiVersion: lifecycle.keptn.sh/v1
apiVersion: lifecycle.keptn.sh/v1alpha2
kind: KeptnEvaluationDefinition
metadata:
name: <evaluation-name>
spec:
source: prometheus | dynatrace | datadog | thanos
source: datadog | dynatrace | prometheus
objectives:
odubajDT marked this conversation as resolved.
Show resolved Hide resolved
- name: query-1
query: "xxxx"
Expand Down
8 changes: 4 additions & 4 deletions docs/docs/reference/crd-reference/metricsprovider.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ comments: true
# KeptnMetricsProvider

A `KeptnMetricsProvider` resource defines an instance of a data provider
(such as Prometheus, Thanos, Dynatrace, or Datadog)
(such as Prometheus, Thanos, Cortex, Dynatrace, or Datadog)
that is used by one or more [KeptnMetric](metric.md) resources.

One Keptn application can perform
Expand Down Expand Up @@ -90,9 +90,9 @@ For detailed information please look at the [Examples section](#examples).

<!-- markdownlint-disable MD046 -->

=== "Prometheus and Thanos"
=== "Prometheus, Cortex and Thanos"

An example of Prometheus as a metrics provider with a Secret holding
An example of Prometheus, Thanos or Cortex as a metrics provider with a Secret holding
the authentication data looks like the following:

```yaml
Expand Down Expand Up @@ -129,7 +129,7 @@ For detailed information please look at the [Examples section](#examples).
> **Note**
When using Dynatrace as metrics provider you can
define the key name of your DT token stored in a secret,
which is not possible for Datadog, Prometheus or Thanos.
which is not possible for Datadog, Prometheus, Cortex or Thanos.
For this example `myCustomTokenKey` was used.

<!-- markdownlint-enable MD046 -->
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/use-cases/non-k8s.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,13 +179,13 @@ similar to what the metrics evaluations of the
Keptn v1 quality gates feature provided.
The data used can come from multiple instances
of multiple data providers
(such as Prometheus, Thanos, Dynatrace, and DataDog).
(such as Prometheus, Thanos, Cortex, Dynatrace, and DataDog).

A Keptn analysis can be run for any application running anywhere
as long Keptn can access a monitoring provider endpoint
that serves metrics for the application.
You can point to multiple instances of the supported monitoring providers
(Prometheus, Thanos, Dynatrace, Datadog, and dql)
(Prometheus, Thanos, Cortex, Dynatrace, Datadog, and dql)
so the application itself can run anywhere.

To implement a Keptn analysis for your deployment:
Expand Down
4 changes: 2 additions & 2 deletions metrics-operator/api/v1/keptnmetricsprovider_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ import (
// KeptnMetricsProviderSpec defines the desired state of KeptnMetricsProvider
type KeptnMetricsProviderSpec struct {
// +kubebuilder:validation:Optional
// +kubebuilder:validation:Pattern:=prometheus|thanos|dynatrace|datadog|dql
// Type represents the provider type. This can be one of prometheus, thanos, dynatrace, datadog, dql.
// +kubebuilder:validation:Pattern:=cortex|datadog|dql|dynatrace|prometheus|thanos
// Type represents the provider type. This can be one of cortex, datadog, dql, dynatrace, prometheus or thanos.
Type string `json:"type"`
// TargetServer defines URL (including port and protocol) at which the metrics provider is reachable.
TargetServer string `json:"targetServer"`
Expand Down
13 changes: 13 additions & 0 deletions metrics-operator/api/v1/keptnmetricsprovider_types_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,19 @@ func TestKeptnMetricsProvider_GetType(t *testing.T) {
},
want: "thanos",
},
{
name: "cortex provider type set",
fields: fields{
ObjectMeta: metav1.ObjectMeta{
Name: "provider1",
},
Spec: KeptnMetricsProviderSpec{
Type: "cortex",
TargetServer: "",
},
},
want: "cortex",
},
{
name: "provider type not set, should return name",
fields: fields{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,8 @@ spec:
type: string
type:
description: Type represents the provider type. This can be one of
prometheus, thanos, dynatrace, datadog, dql.
pattern: prometheus|thanos|dynatrace|datadog|dql
cortex, datadog, dql, dynatrace, prometheus or thanos.
pattern: cortex|datadog|dql|dynatrace|prometheus|thanos
type: string
required:
- targetServer
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,8 @@ spec:
type: string
type:
description: Type represents the provider type. This can be one of
prometheus, thanos, dynatrace, datadog, dql.
pattern: prometheus|thanos|dynatrace|datadog|dql
cortex, datadog, dql, dynatrace, prometheus or thanos.
pattern: cortex|datadog|dql|dynatrace|prometheus|thanos
type: string
required:
- targetServer
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ func TestProvidersPool(t *testing.T) {

func TestProvidersPool_StartProviders(t *testing.T) {

numJobs := 4
numJobs := 6
ctx, cancel := context.WithCancel(context.Background())
resChan := make(chan metricsapi.ProviderResult)
// Create a mock IObjectivesEvaluator, Client, and Logger for testing
Expand All @@ -273,7 +273,7 @@ func TestProvidersPool_StartProviders(t *testing.T) {
time.Sleep(time.Millisecond * 100)

// Assert the expected number of workers (goroutines) were started
require.Equal(t, 4, len(pool.providers))
require.Equal(t, 6, len(pool.providers))
require.Equal(t, numJobs, cap(pool.providers["prometheus"]))
// Stop the providers after testing
pool.StopProviders()
Expand Down
3 changes: 3 additions & 0 deletions metrics-operator/controllers/common/providers/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,14 @@ const DynatraceProviderType = "dynatrace"
const DynatraceDQLProviderType = "dql"
const PrometheusProviderType = "prometheus"
const ThanosProviderType = "thanos"
const CortexProviderType = "cortex"
const DataDogProviderType = "datadog"

var SupportedProviders = []string{
odubajDT marked this conversation as resolved.
Show resolved Hide resolved
DynatraceProviderType,
DynatraceDQLProviderType,
PrometheusProviderType,
DataDogProviderType,
CortexProviderType,
ThanosProviderType,
}
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ type ProviderFactory func(providerType string, log logr.Logger, k8sClient client
func NewProvider(providerType string, log logr.Logger, k8sClient client.Client) (KeptnSLIProvider, error) {

switch strings.ToLower(providerType) {
case PrometheusProviderType, ThanosProviderType:
case PrometheusProviderType, ThanosProviderType, CortexProviderType:
return prometheus.NewPrometheusProvider(log, k8sClient), nil
case DynatraceProviderType:
return &dynatrace.KeptnDynatraceProvider{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ func TestFactory(t *testing.T) {
provider: &prometheus.KeptnPrometheusProvider{},
err: false,
},
{
providerType: CortexProviderType,
provider: &prometheus.KeptnPrometheusProvider{},
err: false,
},
{
providerType: DynatraceProviderType,
provider: &dynatrace.KeptnDynatraceProvider{},
Expand Down
Loading