Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go workspaces for k/k and k/staging/* #123529

Merged
merged 137 commits into from Mar 1, 2024

Conversation

thockin
Copy link
Member

@thockin thockin commented Feb 27, 2024

KEP: kubernetes/enhancements#4402

This is a new PR based on #122624, to clear the lengthy history.

What is it?

The main Kubernetes git repo (github.com/kubernetes/kubernetes, colloquially called "k/k") is the embodiment of evolution. When it was created, the only way to build Go applications was GOPATH. Over time we built tooling which understood GOPATH, and it leaked into many parts of our build, test, and release systems. Then Go added modules, in part because GOPATH was unpleasant and had a tendency to leak into tools. We adopted modules, but we also added the idea of "staging" repositories - a way to eat our cake and have, too. We get the benefits of a monorepo (atomic commits, fast iteration across repos) and then publish those to standalone repos for downstream consumption. To make this work with modules, we abused GOPATH even harder and wrote even more tools.

The Go project saw what we (and others) were doing and did not like it. So they created workspaces. They basically created a solution that is purpose-built for us.

Let's use it.

On reviewing: Review each commit individually. This PR was crafted in lock-step with a series of gengo commits, but since those are merged and gengo is in a different repo, we can't really see those any more. This means that a lot of things don't work at each commit in this PR. This is the only way this could be a sane PR to review though (if you call this sane).

I fully expect that once this merges we will find a long-tail of out-of-tree impacts (tests which need to use go 1.22, etc).

One major aspect of this is code-generation. It's FAR simpler now, but how does it perform? After factoring out build time (by doing a full build first) we can see it has little impact:

before:

$ time ./hack/update-codegen.sh protobuf deepcopy prerelease defaults conversions openapi applyconfigs clients listers informers
go version go1.21.5 linux/amd64
+++ [0107 16:21:01] Generating protobufs for 67 targets
+++ [0107 16:21:01] protoc 23.4 not found (can install with hack/install-protoc.sh); generating containerized...
+++ [0107 16:21:01] Verifying Prerequisites....
+++ [0107 16:21:02] Building Docker image kube-build:build-d19b652805-5-v1.29.0-go1.21.5-bullseye.0
+++ [0107 16:21:04] Syncing sources to container
+++ [0107 16:21:06] Output from this container will be rsynced out upon completion. Set KUBE_RUN_COPY_OUTPUT=n to disable.
+++ [0107 16:21:06] Running build command...
+++ [0107 16:21:25] Syncing out of container
+++ [0107 16:21:27] Generating deepcopy code for 250 targets
+++ [0107 16:21:32] Generating prerelease-lifecycle code for 30 targets
+++ [0107 16:21:34] Generating defaulter code for 100 targets
+++ [0107 16:21:42] Generating conversion code for 139 targets
+++ [0107 16:22:19] Generating openapi code
+++ [0107 16:22:29] Generating apply-config code for 56 targets
+++ [0107 16:22:35] Generating client code for 51 targets
+++ [0107 16:22:40] Generating lister code for 55 targets
+++ [0107 16:22:42] Generating informer code for 55 targets

real	1m43.608s
user	1m29.445s
sys	0m23.746s

after:

$ time ./hack/update-codegen.sh protobuf deepcopy prerelease defaults conversions openapi applyconfigs clients listers informers
+++ [0107 16:15:22] Generating protobufs for 67 targets
+++ [0107 16:15:23] protoc 23.4 not found (can install with hack/install-protoc.sh); generating containerized...
+++ [0107 16:15:23] Verifying Prerequisites....
+++ [0107 16:15:23] Building Docker image kube-build:build-08a5207a41-5-v1.29.0-go1.21.5-bullseye.0
+++ [0107 16:15:25] Syncing sources to container
+++ [0107 16:15:27] Output from this container will be rsynced out upon completion. Set KUBE_RUN_COPY_OUTPUT=n to disable.
+++ [0107 16:15:27] Running build command...
+++ [0107 16:15:49] Syncing out of container
+++ [0107 16:15:51] Generating deepcopy code for 250 targets
+++ [0107 16:15:57] Generating prerelease-lifecycle code for 30 targets
+++ [0107 16:15:59] Generating defaulter code for 100 targets
+++ [0107 16:16:04] Generating conversion code for 139 targets
+++ [0107 16:16:15] Generating openapi code
+++ [0107 16:16:26] Generating apply-config code for 56 targets
+++ [0107 16:16:33] Generating client code for 51 targets
+++ [0107 16:16:39] Generating lister code for 55 targets
+++ [0107 16:16:42] Generating informer code for 55 targets

real	1m23.480s
user	1m49.018s
sys	0m21.897s

/kind bug
/kind cleanup
/kind feature
/kind api-change

The kubernetes repo now uses Go workspaces.  This should not impact end users at all, but does have impact for developers of downstream projects.  Switching to workspaces caused some breaking changes in the flags to the various k8s.io/code-generator tools.  Downstream consumers should look at staging/src/k8s.io/code-generator/kube_codegen.sh to see the changes.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. kind/feature Categorizes issue or PR as related to a new feature. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 27, 2024
@k8s-ci-robot k8s-ci-robot added area/apiserver area/cloudprovider area/code-generation area/conformance Issues or PRs related to kubernetes conformance tests area/dependency Issues or PRs related to dependency changes labels Feb 27, 2024
@k8s-ci-robot k8s-ci-robot added area/kube-proxy area/kubectl area/kubelet area/release-eng Issues or PRs related to the Release Engineering subproject area/test sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. labels Feb 27, 2024
Needed because defaulter-gen tests end up depending on apimachinery.
Because of how the previous 100+ commits were done, so changes snuck
thru that properly belong in earlier commits but it's not really
possible to do that without a lot of effort.

We agreed it was OK to "spackle" these cracks with a final commit.
@thockin
Copy link
Member Author

thockin commented Mar 1, 2024

Reverted the BUILDNAME stuff.

Removed FIXME.

I think it's done. All comments resolved. CI passed last time, so unless I horked something it should pass now. @liggitt if you are happy me, slap me with some LGTM. But please actually BE happy.

@thockin
Copy link
Member Author

thockin commented Mar 1, 2024

/retest

@dims
Copy link
Member

dims commented Mar 1, 2024

/approve
/lgtm

will wait for @liggitt to remove hold

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 1, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: bad7702dd466d0e53c4b69d82a6e0d3536e192c4

Copy link
Member

@liggitt liggitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

One non-blocking nit, unhold at will. May the rebase conflict odds be ever in your favor.

Comment on lines +49 to +50
_KUBE_OUTPUT_SUBPATH="${KUBE_OUTPUT_SUBPATH:-_output/local}"
export KUBE_OUTPUT="${KUBE_ROOT}/${_KUBE_OUTPUT_SUBPATH}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-blocking nit, could be a follow-up cleanup, but this would get you one less ambient bash var:

Suggested change
_KUBE_OUTPUT_SUBPATH="${KUBE_OUTPUT_SUBPATH:-_output/local}"
export KUBE_OUTPUT="${KUBE_ROOT}/${_KUBE_OUTPUT_SUBPATH}"
export KUBE_OUTPUT="${KUBE_ROOT}/${KUBE_OUTPUT_SUBPATH:-_output/local}"

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims, liggitt, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@thockin
Copy link
Member Author

thockin commented Mar 1, 2024

/unhold

Fire in the hole!

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 1, 2024
@dims
Copy link
Member

dims commented Mar 1, 2024

Live picture of @thockin !!

henry-cavill-as-dc-superhero-superman-flying-6hf1nd1s7h49c0ps

@k8s-ci-robot k8s-ci-robot merged commit df36610 into kubernetes:master Mar 1, 2024
28 checks passed
SIG Node CI/Test Board automation moved this from PRs Waiting on Author to Done Mar 1, 2024
SIG Node PR Triage automation moved this from Triage to Done Mar 1, 2024
@k8s-ci-robot k8s-ci-robot added this to the v1.30 milestone Mar 1, 2024
@liggitt
Copy link
Member

liggitt commented Mar 1, 2024

first-try

@thockin
Copy link
Member Author

thockin commented Mar 1, 2024

What's the over/under on time to first explosion?

@dims
Copy link
Member

dims commented Mar 1, 2024

@thockin - as soon as ci-kubernetes-build and ci-kubernetes-build-fast finish running, rest of the periodic jobs pick things up. probably 6-12 hours we'll see some. but more tomorrow. (at least patterns will show up better)

@aojea
Copy link
Member

aojea commented Mar 1, 2024

/honk

@k8s-ci-robot
Copy link
Contributor

@aojea:
goose image

In response to this:

/honk

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/apiserver area/cloudprovider area/code-generation area/conformance Issues or PRs related to kubernetes conformance tests area/dependency Issues or PRs related to dependency changes area/kube-proxy area/kubectl area/kubelet area/release-eng Issues or PRs related to the Release Engineering subproject area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/bug Categorizes issue or PR as related to a bug. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/release Categorizes an issue or PR as relevant to SIG Release. sig/storage Categorizes an issue or PR as relevant to SIG Storage. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
Archived in project
Archived in project
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet