NETOBSERV-844 Unable to have a working statusUrl in FlowCollector with Loki Operator 5.6 #307

jpinsonneau · 2023-03-24T14:47:52Z

Added statusTls option in CRD for status URL

Check example yaml:

    # Uncomment lines below for typical installation with loki-operator (5.6+ needed)
    # url: 'https://loki-gateway-http.netobserv.svc:8080/api/logs/v1/network/'
    # statusUrl: 'https://loki-query-frontend-http.netobserv.svc:3100/'
    # authToken: FORWARD
    # statusTls:
    #   enable: true
    #   caCert:
    #     certFile: service-ca.crt
    #     name: loki-ca-bundle
    #     type: configmap
    #   userCert:
    #     certFile: tls.crt
    #     certKey: tls.key
    #     name: loki-query-frontend-http
    #     type: secret

codecov · 2023-03-24T14:53:33Z

Codecov Report

Merging #307 (9fb3657) into main (3130154) will increase coverage by 0.73%.
The diff coverage is 56.66%.

@@            Coverage Diff             @@
##             main     #307      +/-   ##
==========================================
+ Coverage   49.67%   50.41%   +0.73%     
==========================================
  Files          43       43              
  Lines        5075     5096      +21     
==========================================
+ Hits         2521     2569      +48     
+ Misses       2343     2318      -25     
+ Partials      211      209       -2

Flag	Coverage Δ
unittests	`50.41% <56.66%> (+0.73%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
api/v1alpha1/flowcollector_webhook.go	`0.00% <0.00%> (ø)`
api/v1alpha1/zz_generated.conversion.go	`0.27% <0.00%> (+<0.01%)`	⬆️
api/v1beta1/flowcollector_types.go	`100.00% <ø> (ø)`
...trollers/consoleplugin/consoleplugin_reconciler.go	`62.17% <0.00%> (+1.55%)`	⬆️
controllers/consoleplugin/consoleplugin_objects.go	`92.76% <78.57%> (+0.30%)`	⬆️
api/v1beta1/zz_generated.deepcopy.go	`42.50% <100.00%> (+0.16%)`	⬆️
pkg/helper/flowcollector.go	`65.38% <100.00%> (+3.68%)`	⬆️

... and 1 file with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

jpinsonneau · 2023-03-27T09:31:41Z

config/samples/flows_v1beta1_flowcollector.yaml

@@ -68,7 +68,18 @@ spec:
    # Uncomment lines below for typical installation with loki-operator (5.6+ needed)
    # url: 'https://loki-gateway-http.netobserv.svc:8080/api/logs/v1/network/'
    # statusUrl: 'https://loki-query-frontend-http.netobserv.svc:3100/'
-    # authToken: HOST
+    # authToken: FORWARD


FORWARD is recommended in the doc; we should also recommend it in the sample

jpinsonneau · 2023-03-27T09:32:38Z

config/samples/flows_v1beta1_flowcollector.yaml

+    # statusTls:
+    #   enable: true
+    #   caCert:
+    #     certFile: service-ca.crt
+    #     name: loki-ca-bundle
+    #     type: configmap
+    #   userCert:
+    #     certFile: tls.crt
+    #     certKey: tls.key
+    #     name: loki-query-frontend-http
+    #     type: secret


statusTls needs both loki-ca-bundle and loki-query-frontend-http crt and key files for mTLS

msherif1234 · 2023-03-27T12:02:31Z

controllers/consoleplugin/consoleplugin_objects.go

+			args = append(args, "--loki-status-user-key-path", helper.GetUserKeyPath(&statusTLS, lokiStatusCerts))
+		}
+	}
+


can we extend unit-test to cover this newly added field ?

https://github.com/netobserv/network-observability-operator/pull/307/files#diff-22b7d1b615362ec58a371291942fa2bfe7290375729e4fb6b0be03f3af4ca993

msherif1234 · 2023-03-27T12:03:54Z

controllers/consoleplugin/consoleplugin_objects.go

+	statusTLS := helper.LokiStatusTLS(&desired.Loki)
+	if statusTLS.Enable {
+		if statusTLS.InsecureSkipVerify {
+			args = append(args, "-loki-status-skip-tls")


is this a typo it should be --loki-status-skip-tls ?

I think we should harmonize everything with a single minus character as it's already the case for most of these args. I will also fix the existing --loki-ca-path

Thanks for pointing that !

msherif1234 · 2023-03-27T12:07:42Z

pkg/helper/flowcollector.go

@@ -42,6 +42,13 @@ func LokiForwardUserToken(spec *flowslatest.FlowCollectorLoki) bool {
 	return spec.AuthToken == flowslatest.LokiAuthForwardUserToken
 }

+func LokiStatusTLS(spec *flowslatest.FlowCollectorLoki) flowslatest.ClientTLS {


nit: GetLokiStatusTLS ?

Amoghrd · 2023-03-27T13:37:49Z

/ok-to-test

github-actions · 2023-03-27T13:43:46Z

New images:

quay.io/netobserv/network-observability-operator:1b8fdb7
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-1b8fdb7
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-1b8fdb7

They will expire after two weeks.

Catalog source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-1b8fdb7
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

jotak · 2023-03-27T14:32:11Z

pkg/helper/flowcollector.go

+	if spec.StatusTLS != nil {
+		return *spec.StatusTLS
+	}
+	return spec.TLS


Rather than having a pointer and allowing nil to switch to using global TLS , wouldn't it be better to rely on whether the status URL is set, to look at StatusTLS or just TLS ?

If we use main URL as the status URL, then we use also main TLS as the TLS config - StatusTLS is unused in that case

If statusURL is set, look at StatusTLS (which can still be disabled if not wanted)

IMO it makes things more explicit so less surprising (?)

I'm fine with that 👍

msherif1234 · 2023-03-28T12:26:28Z

/lgtm

Amoghrd · 2023-03-28T12:58:22Z

/ok-to-test

github-actions · 2023-03-28T13:03:02Z

New images:

quay.io/netobserv/network-observability-operator:f80106b
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-f80106b
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-f80106b

They will expire after two weeks.

Catalog source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-f80106b
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

jpinsonneau · 2023-03-28T15:42:22Z

@Amoghrd I've restored the default values as we discussed together 467289f

              "statusTls": {
                "caCert": {
                  "certFile": "service-ca.crt",
                  "name": "loki-ca-bundle",
                  "type": "configmap"
                },
                "userCert": {
                  "certFile": "tls.crt",
                  "certKey": "tls.key",
                  "name": "loki-query-frontend-http",
                  "type": "secret"
                },
                "enable": false,
                "insecureSkipVerify": false
              },

Amoghrd · 2023-03-28T15:45:24Z

/ok-to-test

Amoghrd · 2023-03-28T16:01:25Z

/rm ok-to-test

jpinsonneau · 2023-03-28T16:07:00Z

/ok-to-test

github-actions · 2023-03-28T16:14:15Z

New images:

quay.io/netobserv/network-observability-operator:56e777e
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-56e777e
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-56e777e

They will expire after two weeks.

Catalog source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-56e777e
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

jpinsonneau · 2023-03-28T16:17:56Z

/ok-to-test

github-actions · 2023-03-28T16:23:24Z

New images:

quay.io/netobserv/network-observability-operator:9f45e7d
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-9f45e7d
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-9f45e7d

They will expire after two weeks.

Catalog source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-9f45e7d
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

Amoghrd · 2023-03-28T17:23:45Z

/label qe-approved

jotak · 2023-03-30T07:31:33Z

/lgtm

jpinsonneau · 2023-03-30T08:26:17Z

/approve

openshift-ci · 2023-03-30T08:26:23Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jpinsonneau

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jpinsonneau]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot added the do-not-merge/work-in-progress label Mar 24, 2023

jpinsonneau force-pushed the 844 branch from 5803074 to d11f232 Compare March 27, 2023 09:24

openshift-merge-robot added the needs-rebase label Mar 27, 2023

loki statusTls CRD option

f1c47ff

jpinsonneau force-pushed the 844 branch from d11f232 to f1c47ff Compare March 27, 2023 09:30

openshift-merge-robot removed the needs-rebase label Mar 27, 2023

jpinsonneau commented Mar 27, 2023

View reviewed changes

jpinsonneau requested a review from stleerh March 27, 2023 09:34

jpinsonneau marked this pull request as ready for review March 27, 2023 09:34

openshift-ci bot removed the do-not-merge/work-in-progress label Mar 27, 2023

jpinsonneau mentioned this pull request Mar 27, 2023

[Backport] NETOBSERV-844 Unable to have a working statusUrl in FlowCollector with Loki Operator 5.6 #309

Merged

msherif1234 reviewed Mar 27, 2023

View reviewed changes

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 27, 2023

jotak reviewed Mar 27, 2023

View reviewed changes

addressed feedback

7d64e7c

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 28, 2023

rely on status URL for status TLS

5af9b70

openshift-ci bot assigned msherif1234 Mar 28, 2023

openshift-ci bot added the lgtm label Mar 28, 2023

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 28, 2023

openshift-ci bot removed the lgtm label Mar 28, 2023

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 28, 2023

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 28, 2023

jpinsonneau removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 28, 2023

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 28, 2023

status tls default values

9fb3657

jpinsonneau force-pushed the 844 branch from 467289f to 9fb3657 Compare March 28, 2023 16:16

jpinsonneau removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 28, 2023

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 28, 2023

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 28, 2023

openshift-ci bot added the qe-approved QE has approved this pull request label Mar 28, 2023

openshift-ci bot assigned jotak Mar 30, 2023

openshift-ci bot added the lgtm label Mar 30, 2023

openshift-ci bot added the approved label Mar 30, 2023

openshift-merge-robot merged commit 55f6bd7 into netobserv:main Mar 30, 2023
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NETOBSERV-844 Unable to have a working statusUrl in FlowCollector with Loki Operator 5.6 #307

NETOBSERV-844 Unable to have a working statusUrl in FlowCollector with Loki Operator 5.6 #307

jpinsonneau commented Mar 24, 2023 •

edited

codecov bot commented Mar 24, 2023 •

edited

jpinsonneau Mar 27, 2023

jpinsonneau Mar 27, 2023

msherif1234 Mar 27, 2023

jpinsonneau Mar 28, 2023

msherif1234 Mar 27, 2023

jpinsonneau Mar 27, 2023

msherif1234 Mar 27, 2023

Amoghrd commented Mar 27, 2023

github-actions bot commented Mar 27, 2023

jotak Mar 27, 2023 •

edited

jpinsonneau Mar 28, 2023

jpinsonneau Mar 28, 2023

msherif1234 commented Mar 28, 2023

Amoghrd commented Mar 28, 2023

github-actions bot commented Mar 28, 2023

jpinsonneau commented Mar 28, 2023

Amoghrd commented Mar 28, 2023

Amoghrd commented Mar 28, 2023

jpinsonneau commented Mar 28, 2023

github-actions bot commented Mar 28, 2023

jpinsonneau commented Mar 28, 2023

github-actions bot commented Mar 28, 2023

Amoghrd commented Mar 28, 2023

jotak commented Mar 30, 2023

jpinsonneau commented Mar 30, 2023

openshift-ci bot commented Mar 30, 2023

NETOBSERV-844 Unable to have a working statusUrl in FlowCollector with Loki Operator 5.6 #307

NETOBSERV-844 Unable to have a working statusUrl in FlowCollector with Loki Operator 5.6 #307

Conversation

jpinsonneau commented Mar 24, 2023 • edited

codecov bot commented Mar 24, 2023 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Amoghrd commented Mar 27, 2023

github-actions bot commented Mar 27, 2023

jotak Mar 27, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msherif1234 commented Mar 28, 2023

Amoghrd commented Mar 28, 2023

github-actions bot commented Mar 28, 2023

jpinsonneau commented Mar 28, 2023

Amoghrd commented Mar 28, 2023

Amoghrd commented Mar 28, 2023

jpinsonneau commented Mar 28, 2023

github-actions bot commented Mar 28, 2023

jpinsonneau commented Mar 28, 2023

github-actions bot commented Mar 28, 2023

Amoghrd commented Mar 28, 2023

jotak commented Mar 30, 2023

jpinsonneau commented Mar 30, 2023

openshift-ci bot commented Mar 30, 2023

jpinsonneau commented Mar 24, 2023 •

edited

codecov bot commented Mar 24, 2023 •

edited

jotak Mar 27, 2023 •

edited