get picks for aggregator status #14513

deads2k · 2017-06-07T19:03:54Z

Aggregator status usage didn't come until long after the API was added, so this pulls in the most important picks that are precursors to it so that the conflicts aren't crazy. The picks came in relatively clean. I've kept the fitting commits separate for ease.

@liggitt

deads2k · 2017-06-07T19:04:03Z

[test]

deads2k · 2017-06-08T17:18:51Z

Alright, the last commit is a change that is needed to wire openshift into a more "normal" aggregated API server shape so that our admission chain works after the aggregator picks we needed from upstream. It is mostly a shuffle, but it is fairly large. Luckily, our integration test flows cover this. More refactoring is necessary, but this is about as small as I could get and have it work.

@soltysh @mfojtik @sttts ptal.
@liggitt @eparis this is a structural change that has almost no shot of making it through the origin queue in a reasonable time due to likely conflict/rebase cycles that push me to the back. I'm requesting a green button on lgtm and green tests.

deads2k · 2017-06-08T17:19:59Z

@liggitt @eparis this is a structural change that has almost no shot of making it through the origin queue in a reasonable time due to likely conflict/rebase cycles that push me to the back. I'm requesting a green button on lgtm and green tests.

for perspective, the queue is currently 26 deep and working on a pull that was tagged for merge on Monday.

sttts · 2017-06-09T13:45:15Z

pkg/cmd/server/origin/apiserver.go

@@ -0,0 +1,336 @@
+package origin


openshift-apiserver.go ?

or originapi-apiserver.go

sttts · 2017-06-09T13:47:21Z

pkg/cmd/server/origin/nonapiserver.go

+	TemplateNamespaces []string
+}
+
+// OpenshiftNonAPIServer is only responsible for serving the APIs for Openshift


outdated comment?

sttts · 2017-06-09T13:52:47Z

pkg/cmd/server/origin/master.go

+	// this is installing the openshift APIs into the kubeapiserver
+	installAPIs(openshiftAPIServerConfig, kubeAPIServer.GenericAPIServer)
+	// this sets up the openapi endpoints
+	preparedKubeAPIServer := kubeAPIServer.GenericAPIServer.PrepareRun()


don't like that we call PrepareRun() only here and not for the others. Is it the same in kube-apiserver?

don't like that we call PrepareRun() only here and not for the others. Is it the same in kube-apiserver?

Yeah. openapi is the source of many sins

sttts · 2017-06-09T14:00:05Z

pkg/cmd/server/origin/apiserver.go

+
+			// TODO all of our groups currently have one version, by the time we get more than one, these should be split up
+			// into their own api servers
+			apiGroupInfo.GroupMeta.GroupVersion = gv


Why not keep isPreferredGroupVersion?

sttts · 2017-06-09T14:00:43Z

pkg/cmd/server/origin/apiserver.go

+			// into their own api servers
+			apiGroupInfo.GroupMeta.GroupVersion = gv
+
+			glog.Infof("Starting Origin API at %%s%s/%s/%s", api.GroupPrefix, gv.Group, gv.Version)


note the %%s. This is substituted somewhere else in the old code.

and all the other places with %%s

sttts · 2017-06-09T14:04:05Z

pkg/cmd/server/origin/nonapiserver.go

+}
+
+func (c completedOpenshiftNonAPIConfig) New(delegationTarget genericapiserver.DelegationTarget, stopCh <-chan struct{}) (*OpenshiftNonAPIServer, error) {
+	genericServer, err := c.OpenshiftNonAPIConfig.GenericConfig.SkipComplete().New("openshift-nonapiserver", delegationTarget) // completion is done in Complete, no need for a second time


nonapiserver is strange somehow. openshift-non-api-routes?

nonapiserver is strange somehow. openshift-non-api-routes?

Not quite as much fun, but sure.

sttts · 2017-06-09T14:06:20Z

pkg/cmd/server/origin/storage.go

+	//TODO/REBASE use something other than c.KubeClientsetInternal
+	nodeConnectionInfoGetter, err := kubeletclient.NewNodeConnectionInfoGetter(c.KubeClientExternal.CoreV1().Nodes(), *c.KubeletClientConfig)
+	if err != nil {
+		return nil, fmt.Errorf("Unable to configure the node connection info getter: %v", err)


nit: lower case

everywhere below as well

sttts · 2017-06-09T14:08:11Z

pkg/cmd/server/origin/storage.go

+	}
+
+	buildStorage, buildDetailsStorage, err := buildetcd.NewREST(c.GenericConfig.RESTOptionsGetter)
+	if err != nil {


would write those if-clause in one line, if gofmt accepts that. Func gets very long with 50% error handling.

would write those if-clause in one line, if gofmt accepts that. Func gets very long with 50% error handling.

I miss java too :)

This function is on the chopping block. I wouldn't get overly caught up in it.

sttts · 2017-06-09T14:12:34Z

pkg/cmd/server/origin/master.go

-}
-
-// initMetricsRoute initializes an HTTP endpoint for metrics.
-func initMetricsRoute(apiContainer *restful.Container, path string) {


this was dead code?

sttts · 2017-06-09T14:13:55Z

pkg/cmd/server/origin/master.go

-
-	storage := make(map[string]rest.Storage)
-	for k, v := range all {
-		if excludedV1Types.Has(k) {


this was empty I guess.

sttts · 2017-06-09T14:15:18Z

pkg/cmd/server/origin/nonapiserver.go

+
+	// c.Options.OAuthConfig.MasterPublicURL
+	MasterPublicURL string
+	// c.Options.OAuthConfig != nil


Better comment? These code comments will get outdated.

they were breadcrumbs for me. removing

sttts · 2017-06-09T14:16:45Z

pkg/cmd/server/origin/storage.go

+	_ "github.com/openshift/origin/pkg/api/install"
+)
+
+// TODO this function needs to be broken apart with each API group owning their own storage, probably with two method


yes, please. 300 lines.

yes, please. 300 lines.

follow up

fine with me.

deads2k · 2017-06-09T15:45:00Z

Comments addressed and some more tests fixed. So glad we have tests.

sttts · 2017-06-09T16:27:37Z

Looks good from my side. Having some more eyes on this wouldn't harm though. Quite a bit of code moved and split apart.

deads2k · 2017-06-09T17:46:09Z

Appears to be clean of conflicts with the next two in the queue. Plan to merge on green.

deads2k · 2017-06-09T18:44:36Z

@liggitt @sttts Hit a snag with admission. See the last two commits for a "fix" that works until we aggregate in 3.7.

deads2k · 2017-06-09T20:35:18Z

This breaks the back of the problem, but then we have at least one TODO that needs doing and the generated code problem. I'd like to see this merged to get those a lot smaller.

deads2k · 2017-06-09T21:59:10Z

Re[test]

deads2k · 2017-06-10T01:51:32Z

@sttts if you could make peace with the last two commits on Monday :)

sttts · 2017-06-12T09:43:42Z

pkg/cmd/server/origin/master.go

+	// we're going through a "normal" API installation in the wrong server, we need to switch the admission chain
+	// *only while we're installing these APIs*.  There are tests that make sure this works and doesn't drop
+	// plugins and we'll remove it once we're aggregating
+	kubeAPIServer.GenericAPIServer.SetAdmission(openshiftAPIServerConfig.GenericConfig.AdmissionControl)


ugly, but ok.

Wondering which of these temporary hacks we can move into the if-clause below for the non-aggregating run mode?

Wondering which of these temporary hacks we can move into the if-clause below for the non-aggregating run mode?

Can't. These are always needed to get the admission chain right until we're always aggregating.
:(

sttts · 2017-06-12T09:44:47Z

@deads2k wouldn't call it peace, but I am ok with those two (temporary) commits.

…erver

…handling concerns

deads2k · 2017-06-12T11:50:50Z

[merge]

openshift-bot · 2017-06-12T11:54:15Z

continuous-integration/openshift-jenkins/merge Waiting: You are in the build queue at position: 5

openshift-bot · 2017-06-12T11:54:15Z

Evaluated for origin merge up to 78f422e

openshift-bot · 2017-06-12T11:54:16Z

Evaluated for origin test up to 78f422e

deads2k · 2017-06-12T13:26:09Z

flaked on #14573

openshift-bot · 2017-06-12T13:45:42Z

continuous-integration/openshift-jenkins/test FAILURE (https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin/2130/) (Base Commit: c09c601)

deads2k · 2017-06-12T14:35:52Z

Green except for the flake. Blocks aggregator usage. Merging as planned. Doesn't conflict with top two pulls.

deads2k force-pushed the agg-05-status branch from cfa24a6 to 8d7c009 Compare June 9, 2017 13:27

sttts reviewed Jun 9, 2017

View reviewed changes

deads2k force-pushed the agg-05-status branch from 8d7c009 to aef49c6 Compare June 9, 2017 15:44

deads2k force-pushed the agg-05-status branch from aef49c6 to 04d7fb0 Compare June 9, 2017 18:44

sttts reviewed Jun 12, 2017

View reviewed changes

deads2k added 3 commits June 12, 2017 07:41

UPSTREAM: 44399: add deregistration for paths

a1bdcb0

UPSTREAM: 44466: use our own serve mux that directs how we want

b0038b1

UPSTREAM: 42896: require codecfactory

8fa1c2d

deads2k added 15 commits June 12, 2017 07:41

react to requiring a codec

e5e13b1

UPSTREAM: 42835: remove legacy insecure port options from genericapis…

ec81dbb

…erver

react to moving insecure options

e24c7d5

UPSTREAM: 43003: separate discovery from the apiserver

635742b

UPSTREAM: 00000: disambiguate operation names for legacy discovery

3433472

react to updating discovery wiring

ae090a9

UPSTREAM: 45370: refactor names for the apiserver handling chain

18b5331

react to refactor names for the apiserver handling chain

13476eb

UPSTREAM: 45432: use apiservice.status to break apart controller and …

e86c44f

…handling concerns

UPSTREAM: 44408: aggregator controller changes only

d4d1d6e

UPSTREAM: 46440: fix api server handler routing (move CRD behind TPR)

17a3b95

react to fixed http handlers

d662b41

wire up openshift to be slight more 'normal'

12f76d4

UPSTREAM: <drop>: drop post 3.7 rebase. allows disabled aggregator

7fc9d38

reset admission plugins for the openshift APIs

78f422e

deads2k force-pushed the agg-05-status branch from 04d7fb0 to 78f422e Compare June 12, 2017 11:50

deads2k merged commit b57c2fd into openshift:master Jun 12, 2017

deads2k deleted the agg-05-status branch August 3, 2017 19:26

get picks for aggregator status #14513

get picks for aggregator status #14513

Conversation

deads2k commented Jun 7, 2017

deads2k commented Jun 7, 2017

deads2k commented Jun 8, 2017

deads2k commented Jun 8, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deads2k commented Jun 9, 2017

sttts commented Jun 9, 2017

deads2k commented Jun 9, 2017

deads2k commented Jun 9, 2017

deads2k commented Jun 9, 2017

deads2k commented Jun 9, 2017

deads2k commented Jun 10, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sttts commented Jun 12, 2017

deads2k commented Jun 12, 2017

openshift-bot commented Jun 12, 2017 • edited Loading

openshift-bot commented Jun 12, 2017

openshift-bot commented Jun 12, 2017

deads2k commented Jun 12, 2017

openshift-bot commented Jun 12, 2017

deads2k commented Jun 12, 2017 • edited Loading

openshift-bot commented Jun 12, 2017 •

edited

Loading

deads2k commented Jun 12, 2017 •

edited

Loading