Shift productivity infrastructure to a community managed environment

[bsnchan]

Current

All the infrastructure used by the Productivity WG is currently inaccessible to people outside of Google

Proposed

TBD.

Notes

• We've previously discussed sharing the cost of infrastructure at the steering level. Let's revisit this conversation to see how we might want to approach this.
• I believe folks outside of Google also cannot submit performance metrics to mako.dev

[mattmoor]

Looks like the Mako folks are giving up: google/mako#2 (comment)

[rgregg]

Any community alternatives to Mako?

[mattmoor]

@bbrowning What do the Openshift perf folks use for dashboarding?

[bbrowning]

@mattmoor I'm not sure we have any tools which are exposed publicly, but I'll ask around to see what they use internally to see if that's something we could use in the community.

[duglin]

@bsnchan does this issue cover just the performance stuff or do you see it covering all tooling used by knative? e.g. github management (e.g prow), release building, PR testing, etc...

It would be great if we could expand it. Start by enumerating all of the bits that (today) require a specific cloud provider, and then work on 1) tweaking the code/scripts to support multiple providers, and 2) look for volunteers to spread the cost and hosting responsibilities.

[mattmoor]

It covers everything, I was just noting that Mako was particularly closed.

[thisisnotapril]

Mako is now going in to maintenance mode; so if there are other alternatives people know of and like; might be a good time to review those? Mako will still be supported but no new features; and PII remains an issue for getting a Mako version that can be accessed by all.

[vaikas]

Since things have changed a little bit since the last time this was looked at (github actions for one), I'm going to start the list of tools that I'm aware of so that we can evaluate options. @duglin @mattmoor @n3wscott what others are missing from here? Also @bbrowning were you able to find anything out?

• testgrid
• mako
• prow
• release jobs

[n3wscott]

more than just prow, there are several jobs that run in prow that do work for us. So finding an alternative for those jobs will be needed and likely require some dev work.

[vaikas]

@n3wscott where should I look for that list of jobs? Thanks :)

[n3wscott]

I think here @vaikas https://github.com/GoogleCloudPlatform/oss-test-infra/tree/master/prow/knative

[evankanderson]

Looking through that list and the generated config, I see:

• Test log storage and viewing. (https://github.com/kubernetes/test-infra/tree/master/prow/plank, https://github.com/kubernetes/test-infra/tree/master/prow/spyglass + Google Cloud Storage)
• Slack reporting
• Automatic merging with testing. (https://github.com/kubernetes/test-infra/tree/master/prow/tide)
• Label management for non-writers (https://github.com/GoogleCloudPlatform/oss-test-infra/blob/master/prow/knative/plugins.yaml#L99 -- /area et al)
• Test/retest for non-writers
• Cherrypicker (https://github.com/kubernetes/test-infra/tree/master/prow/external-plugins/cherrypicker)

[vaikas]

I'm wary of moving more things to GH actions given our ongoing issues with queuing issues, and the lack of visibility.
Prow has also seen tons of improvements as of late, so things are getting better on that front.

[vaikas]

@evankanderson suggested using the Github actions with runners into prow cluster.

[evankanderson]

See also https://docs.google.com/document/d/1LwkzXVDJYLmiVElCEiJ1ITSr4TGkHfipe8ABK_7IxCQ/edit?hl=en#heading=h.ky4f33l2ky29

[bsnchan]

Steering doesn't consider this to be critical for 1.0, our productivity group may be spread too thin at the moment so we may not pick this up until post 1.0 (unless someone raises their hand to help with this in the meantime).

[dprotaso]

cc @chizhg just presented a roadmap document during the ToC

https://docs.google.com/document/d/1U6wybYjCslEhsKOiennrwhoKZmSVnd1NGbFI8gnnnOg/edit#

[dprotaso]

related/dupe
#786

[evankanderson]

/close

Dup of #786

[knative-prow-robot]

@evankanderson: Closing this issue.

In response to this:

/close

Dup of #786

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.