Bug 1777137: add observation of idp config and validation of its cm/secrets #222

stlaz · 2019-11-28T15:46:48Z

currently probably quite fragile, many FIXMEs and TODOs

The previous resource sync for the resources from the oauth config was replaces by syncing it from the configobserver and it seems to work, just as well as does the slightly reworked logic about how mounts are placed in the pods

add observation for whatever we need from the consoles
add observation for whatever we need from the infrastructures object
make the deployment use the observed config
?
profit

stlaz · 2019-12-03T07:32:34Z

/retest

stlaz · 2019-12-04T13:04:43Z

/retest

stlaz · 2019-12-06T08:52:21Z

did some very wild rebasing to make this reviewable

marun

Not too serious a review as befits my lack of experience with the operator and the WIP nature of the PR. I'm surprised so many jobs passed, I guess that's an indication of a lack of test coverage?

Given the amount of repetition between the observers, maybe there's room for generic helpers that can simplify reading and writing unstructured data? I'm thinking most observers could be driven by data rather than code:

// Accepts lister and returns a mapping of canonical field names and their current values
func currentConfigFunc(configobservation.Listers) (map[string]interface{}, error)

// Example result
currentConfig := map[string]interface{}{
  // . separated field names
  "my.value": 0,
}

Given a function that can marshal data from a static type from the API to unstructured, maybe it's possible for generic helpers to be used to read/write from unstructured?

pkg/operator2/configobservation/apiserver/observe_cors.go

pkg/operator2/configobservation/oauth/observe_tokenconfig.go

marun · 2019-12-21T01:21:12Z

pkg/operator2/configobservation/oauth/observe_tokenconfig.go

+	"github.com/openshift/cluster-authentication-operator/pkg/operator2/configobservation"
+)
+
+const defaultAccessTokenMaxAgeSeconds = float64(86400) // a day


Given that the golant struct fields are int32 and the value in seconds is fixed point, is it desirable to use a floating point type?

glad you asked, this was such a PITA to get right.

The config is really a json that gets deserialized into golang nested interfaces.

When we make this int32 and store as such, the information about it being int is obviously stored.

Eventually, the configObserver, which lives in library-go, compares the config it got originally with the one that it got as observed config.

This is where all the fun happens. because the json parser actually likes to think of numbers as of floats prior to them being integers. That means that if we make this integer, same jsons will actually be considered different in their go interpretation even if our comparison here tells you the numbers are equal. That results in hotloop trying to update the observedConfig field on same values.

I tried many things, played with the json parser flags, use json.Number as the default representation for numbers found in the jsons we have here but nothing except this really worked.

I did a terrible job with code comments about this, I see.

pkg/operator2/configobservation/oauth/observe_tokenconfig.go

pkg/operator2/configobservation/oauth/template_conversions.go

pkg/operator2/configobservation/routersecret/observe_router_secret.go

stlaz

First bunch of answers, for some reason github defaults to comments using their review system. Meh.

stlaz · 2020-06-16T12:44:06Z

addressed the comments, during which I noticed default authz token max age, which is not actually being dynamically set, was missing, fixed that

stlaz · 2020-06-16T13:36:19Z

/retest

marun · 2020-06-17T00:36:58Z

pkg/controllers/configobservation/oauth/observe_tokenconfig_test.go

+			expected: map[string]interface{}{
+				"oauthConfig": map[string]interface{}{
+					"tokenConfig": map[string]interface{}{
+						"accessTokenMaxAgeSeconds":    float64(86400),


(No action required) Maybe this is just my ignorance of accepted practice for cluster operators, but I find it strange to see magic numbers rather than constants.

marun · 2020-06-17T00:38:01Z

pkg/controllers/configobservation/oauth/observe_tokenconfig_test.go

+			errors: []error{},
+		},
+		{
+			name: "inactivity 0 means disabled",


I'm confused - how is this different from "max age 0 still means default max age"?

marun · 2020-06-17T00:43:01Z

@stlaz Thank you for cleanup! Other than the seemingly duplicated test, lgtm. Please squash for merge.

openshift-ci-robot · 2020-06-18T15:26:40Z

@stlaz: This pull request references Bugzilla bug 1777137, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.6.0) matches configured target release for branch (4.6.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1777137: add observation of idp config and validation of its cm/secrets

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-06-18T15:26:43Z

@stlaz: This pull request references Bugzilla bug 1777137, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.6.0) matches configured target release for branch (4.6.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1777137: add observation of idp config and validation of its cm/secrets

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

marun · 2020-06-18T17:33:11Z

/lgtm
/retest

openshift-ci-robot · 2020-06-18T17:33:30Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: marun, stlaz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [stlaz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

stlaz · 2020-06-19T09:08:38Z

/retest

stlaz · 2020-06-19T10:40:39Z

/retest

stlaz · 2020-06-19T12:32:24Z

/retest
unrelated failures

openshift-ci-robot · 2020-06-19T14:30:26Z

@stlaz: All pull requests linked via external trackers have merged: openshift/cluster-authentication-operator#222. Bugzilla bug 1777137 has been moved to the MODIFIED state.

In response to this:

Bug 1777137: add observation of idp config and validation of its cm/secrets

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot requested review from deads2k and sttts November 28, 2019 15:47

stlaz force-pushed the observe_config branch 6 times, most recently from 0313a34 to 3eb0494 Compare December 2, 2019 15:17

stlaz force-pushed the observe_config branch 3 times, most recently from 6e43291 to dbe3514 Compare December 6, 2019 08:50

stlaz changed the title ~~[WIP] [ugly commits] add config observation~~ [WIP] add config observation Dec 6, 2019

stlaz mentioned this pull request Dec 10, 2019

things david doesn't like #141

Closed

marun suggested changes Dec 21, 2019

View reviewed changes

stlaz commented Jan 14, 2020

View reviewed changes

stlaz force-pushed the observe_config branch from dbe3514 to 24f5d40 Compare January 14, 2020 13:58

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 14, 2020

stlaz force-pushed the observe_config branch from 24f5d40 to f5778ac Compare January 14, 2020 14:05

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 14, 2020

stlaz force-pushed the observe_config branch from f5778ac to 7ca3966 Compare January 14, 2020 15:44

stlaz mentioned this pull request Jan 15, 2020

Add only a basic config observer #242

Merged

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 24, 2020

stlaz force-pushed the observe_config branch from 7ca3966 to 08abc5f Compare April 21, 2020 10:57

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 21, 2020

stlaz force-pushed the observe_config branch 3 times, most recently from 21e87ef to 7e12e9a Compare June 16, 2020 12:42

marun reviewed Jun 17, 2020

View reviewed changes

stlaz added 8 commits June 18, 2020 17:02

add observer for identity providers

9d5cec4

move transport helpers to their own directory

7b85b0c

simplify syncdata code and add unit tests

47e3db0

Add oauth templates observer

c8dc086

add observers for oauth tokenconfig

d23add5

add validation of secrets and configmaps from config

44daac0

go mod tidy && go mod vendor

9b2047a

move controllers to their own separate directory

94e4e71

stlaz force-pushed the observe_config branch from 7e12e9a to 94e4e71 Compare June 18, 2020 15:14

openshift-ci-robot assigned marun Jun 18, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 18, 2020

openshift-merge-robot merged commit 852cd73 into openshift:master Jun 19, 2020

stlaz deleted the observe_config branch July 2, 2020 09:03

openshift-ci-robot mentioned this pull request Aug 6, 2020

Bug 1777137: don't post new observed config on resources validation #312

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug 1777137: add observation of idp config and validation of its cm/secrets #222

Bug 1777137: add observation of idp config and validation of its cm/secrets #222

stlaz commented Nov 28, 2019 •

edited

stlaz commented Dec 3, 2019

stlaz commented Dec 4, 2019

stlaz commented Dec 6, 2019 •

edited

marun left a comment

marun Dec 21, 2019

stlaz Jan 14, 2020

stlaz left a comment

stlaz commented Jun 16, 2020

stlaz commented Jun 16, 2020

marun Jun 17, 2020

marun Jun 17, 2020

marun commented Jun 17, 2020

openshift-ci-robot commented Jun 18, 2020

openshift-ci-robot commented Jun 18, 2020

marun commented Jun 18, 2020

openshift-ci-robot commented Jun 18, 2020

stlaz commented Jun 19, 2020

stlaz commented Jun 19, 2020

stlaz commented Jun 19, 2020

openshift-ci-robot commented Jun 19, 2020

Bug 1777137: add observation of idp config and validation of its cm/secrets #222

Bug 1777137: add observation of idp config and validation of its cm/secrets #222

Conversation

stlaz commented Nov 28, 2019 • edited

stlaz commented Dec 3, 2019

stlaz commented Dec 4, 2019

stlaz commented Dec 6, 2019 • edited

marun left a comment

Choose a reason for hiding this comment

marun Dec 21, 2019

Choose a reason for hiding this comment

stlaz Jan 14, 2020

Choose a reason for hiding this comment

stlaz left a comment

Choose a reason for hiding this comment

stlaz commented Jun 16, 2020

stlaz commented Jun 16, 2020

marun Jun 17, 2020

Choose a reason for hiding this comment

marun Jun 17, 2020

Choose a reason for hiding this comment

marun commented Jun 17, 2020

openshift-ci-robot commented Jun 18, 2020

openshift-ci-robot commented Jun 18, 2020

marun commented Jun 18, 2020

openshift-ci-robot commented Jun 18, 2020

stlaz commented Jun 19, 2020

stlaz commented Jun 19, 2020

stlaz commented Jun 19, 2020

openshift-ci-robot commented Jun 19, 2020

stlaz commented Nov 28, 2019 •

edited

stlaz commented Dec 6, 2019 •

edited