Group Makefile targets #3192

Skarlso · 2022-02-10T20:08:48Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

A nicer Makefile help output:

➜  cluster-api-provider-aws git:(group_makefile) ✗ make

Usage:
  make <target>

Testing
  test                                      Run tests
  generate-test-flavors                     Generate test template flavors
  test-conformance                          Run clusterctl based conformance test on workload cluster (requires Docker).
  test-cover                                Run tests with code coverage and code generate  reports

Binaries
  binaries                                  Builds and installs all binaries
  manager-aws-infrastructure                Build manager binary.
  clusterawsadm                             Build clusterawsadm binary.

Linting
  lint                                      Lint codebase
  lint-fix                                  Lint the codebase and run auto-fixers if supported by the linter

Generate
  modules                                   Runs go mod to ensure proper vendoring.
  generate                                  Generate code
  generate-go-apis                          Runs Go related generate targets

Docker
  docker-build                              Build the docker image for controller-manager
  docker-push                               Push the docker image
  docker-push-core-manifest                 Push the fat manifest docker image.

Release
  list-staging-releases                     List staging images for image promotion
  release                                   Builds and push container images using the latest git tag for the commit.
  release-changelog                         Builds the changelog for a release
  release-binaries                          Builds the binaries to publish with a release
  release-staging                           Builds and push container images and manifests to the staging bucket.
  release-staging-nightly                   Tags and push container images to the staging bucket.
  upload-staging-artifacts                  Upload release artifacts to the staging bucket
  create-gh-release                         Create release on Github
  upload-gh-artifacts                       Upload artifacts to Github release
  release-templates                         Generate release templates

Cleanup / Verification
  clean                                     Remove all generated files
  clean-bin                                 Remove all generated binaries
  clean-temporary                           Remove all temporary files and folders
  serve-book                                Run a server with the documentation book
  clean-release                             Remove the release folder
  verify-conversions                        Verifies expected API conversion are in place
  verify-shellcheck                         Verify shell files
  clean-artifacts                           Remove the _artifacts directory

Help
  help                                      Display this help

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #3188

Special notes for your reviewer:

Checklist:

squashed commits
includes documentation
adds unit tests
adds or updates e2e tests

Release note:

Adding a nicer Makefile help output so it's easy to locate a certain target.

k8s-ci-robot · 2022-02-10T20:08:55Z

@Skarlso: This issue is currently awaiting triage.

If CAPA/CAPI contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · 2022-02-10T20:08:57Z

Hi @Skarlso. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Skarlso · 2022-02-10T20:10:32Z

@shivi28 👋 :)

These are the categories in the Makefile itself. I wanted to keep that as it is but I'm happy to reorder as needed. :)

sedefsavas · 2022-02-10T20:11:48Z

/ok-to-test

sedefsavas · 2022-02-10T20:21:10Z

It would be nice to make the categories look more like cluster-api, feel free to reorder as needed.

Usage:
  make <target>

Targets:

generate:
  generate                                       Run all generate-manifests-*, generate-go-deepcopy-* and generate-go-conversions-* targets
  generate-manifests                             Run all generate-manifests-* targets
  generate-manifests-core                        Generate manifests e.g. CRD, RBAC etc. for core
  generate-manifests-kubeadm-bootstrap           Generate manifests e.g. CRD, RBAC etc. for kubeadm bootstrap
  generate-manifests-kubeadm-control-plane       Generate manifests e.g. CRD, RBAC etc. for kubeadm control plane
  generate-go-deepcopy                           Run all generate-go-deepcopy-* targets
  generate-go-deepcopy-core                      Generate deepcopy go code for core
  generate-go-deepcopy-kubeadm-bootstrap         Generate deepcopy go code for kubeadm bootstrap
  generate-go-deepcopy-kubeadm-control-plane     Generate deepcopy go code for kubeadm control plane
  generate-go-conversions                        Run all generate-go-conversions-* targets
  generate-go-conversions-core                   Generate conversions go code for core
  generate-go-conversions-kubeadm-bootstrap      Generate conversions go code for kubeadm bootstrap
  generate-go-conversions-kubeadm-control-plane  Generate conversions go code for kubeadm control plane
  diagrams                                       Generate diagrams for *.plantuml files

lint and verify:
  modules                                        Run go mod tidy to ensure modules are up to date
  lint                                           Lint the codebase
  lint-fix                                       Lint the codebase and run auto-fixers if supported by the linter
  apidiff                                        Check for API differences
  format-tiltfile                                Format the Tiltfile
  verify                                         Run all verify-* targets
  verify-modules                                 Verify go modules are up to date
  verify-gen                                     Verfiy go generated files are up to date
  verify-conversions                             Verifies expected API conversion are in place
  verify-boilerplate                             Verify boilerplate text exists in each file
  verify-shellcheck                              Verify shell files
  verify-tiltfile                                Verify Tiltfile format
  verify-book-links                              Verify book links

build:
  clusterctl                                     Build the clusterctl binary
  managers                                       Run all manager-* targets
  manager-core                                   Build the core manager binary into the ./bin folder
  manager-kubeadm-bootstrap                      Build the kubeadm bootstrap manager binary into the ./bin folder
  manager-kubeadm-control-plane                  Build the kubeadm control plane manager binary into the ./bin folder
  docker-build                                   Run all docker-build-* targets
  docker-build-core                              Build the docker image for core controller manager
  docker-build-kubeadm-bootstrap                 Build the docker image for kubeadm bootstrap controller manager
  docker-build-kubeadm-control-plane             Build the docker image for kubeadm control plane controller manager
  e2e-framework                                  Builds the CAPI e2e framework

test:
  test                                           Run unit and integration tests
  test-e2e                                       Run the e2e tests
  test-junit                                     Run tests with verbose setting and generate a junit report
  test-cover                                     Run tests with code coverage and code generate reports
  test-verbose                                   Run tests with verbose settings
  serve-book                                     Build and serve the book with live-reloading enabled
  kind-cluster                                   Create a new kind cluster designed for testing with Tilt
  docker-build-e2e                               Rebuild all Cluster API provider images to be used in the e2e tests

release:
  release                                        Build and push container images using the latest git tag for the commit
  release-manifests                              Build the manifests to publish with a release
  release-manifests-dev                          Build the development manifests and copies them in the release folder
  release-binaries                               Build the binaries to publish with a release
  release-staging                                Build and push container images to the staging bucket
  release-staging-nightly                        Tag and push container images to the staging bucket. Example image tag: cluster-api-controller:nightly_main_20210121
  release-alias-tag                              Add the release alias tag to the last build tag
  docker-build-all                               Build all the architecture docker images
  docker-push                                    Push the docker images
  docker-push-all                                Push all the architecture docker images
  docker-push-manifest-core                      Push the multiarch manifest for the core docker images
  docker-push-manifest-kubeadm-bootstrap         Push the multiarch manifest for the the kubeadm bootstrap docker images
  docker-push-manifest-kubeadm-control-plane     Push the multiarch manifest for the the kubeadm control plane docker images

clean:
  clean                                          Remove all generated files
  clean-bin                                      Remove all generated binaries
  clean-book                                     Remove all generated GitBook files
  clean-release                                  Remove the release folder
  clean-release-git                              Restores the git files usually modified during a release
  clean-generated-conversions                    Remove files generated by conversion-gen from the mentioned dirs. Example SRC_DIRS="./api/v1alpha4"

hack/tools:
  controller-gen                                 Build a local copy of controller-gen
  conversion-gen                                 Build a local copy of conversion-gen
  conversion-verifier                            Build a local copy of conversion-verifier
  gotestsum                                      Build a local copy of gotestsum
  go-apidiff                                     Build a local copy of apidiff
  golangci-lint                                  Build a local copy of golangci-lint
  envsubst                                       Build a local copy of envsubst
  kustomize                                      Build a local copy of kustomize
  setup-envtest                                  Build a local copy of setup-envtest
  tilt-prepare                                   Build a local copy of tilt-prepare
  kpromo                                         Build a local copy of kpromo

Same category names, like instead of binaries --> build

Skarlso · 2022-02-10T20:46:29Z

@sedefsavas how about this:

➜  cluster-api-provider-aws git:(group_makefile) make

Usage:
  make <target>

generate:
  generate                                  Generate code
  generate-go-apis                          Runs Go related generate targets

list and verify:
  verify-conversions                        Verifies expected API conversion are in place
  verify-shellcheck                         Verify shell files

build:
  binaries                                  Builds and installs all binaries
  manager-aws-infrastructure                Build manager binary.
  clusterawsadm                             Build clusterawsadm binary.

lint:
  lint                                      Lint codebase
  lint-fix                                  Lint the codebase and run auto-fixers if supported by the linter

test:
  test                                      Run tests
  generate-test-flavors                     Generate test template flavors
  test-conformance                          Run clusterctl based conformance test on workload cluster (requires Docker).
  test-cover                                Run tests with code coverage and code generate  reports
  serve-book                                Run a server with the documentation book

release:
  list-staging-releases                     List staging images for image promotion
  release                                   Builds and push container images using the latest git tag for the commit.
  release-changelog                         Builds the changelog for a release
  release-binaries                          Builds the binaries to publish with a release
  release-staging                           Builds and push container images and manifests to the staging bucket.
  release-staging-nightly                   Tags and push container images to the staging bucket.
  upload-staging-artifacts                  Upload release artifacts to the staging bucket
  create-gh-release                         Create release on Github
  upload-gh-artifacts                       Upload artifacts to Github release
  release-templates                         Generate release templates

clean:
  clean                                     Remove all generated files
  clean-bin                                 Remove all generated binaries
  clean-temporary                           Remove all temporary files and folders
  clean-release                             Remove the release folder
  clean-artifacts                           Remove the _artifacts directory

tools:
  docker-build                              Build the docker image for controller-manager
  docker-push                               Push the docker image
  docker-push-core-manifest                 Push the fat manifest docker image.
  modules                                   Runs go mod to ensure proper vendoring.

help:
  help                                      Display this help

sedefsavas · 2022-02-10T21:13:03Z

Makefile

-		echo "go module files are out of date"; exit 1; \
-	fi
+.PHONY: docker-build
+docker-build: docker-pull-prerequisites ## Build the docker image for controller-manager


This can go under build category?

Sure. I was on the fence about it since it's docker build. 😂

sedefsavas · 2022-02-10T21:13:21Z

Makefile

-	echo verification of api conversions initiated
-	$(CONVERSION_VERIFIER)
+.PHONY: docker-push
+docker-push: ## Push the docker image


This can go under release category.

sedefsavas · 2022-02-10T21:14:50Z

Makefile

-.PHONY: docker-build
-docker-build: docker-pull-prerequisites ## Build the docker image for controller-manager
-	docker build --build-arg ARCH=$(ARCH) --build-arg LDFLAGS="$(LDFLAGS)" . -t $(CORE_CONTROLLER_IMG)-$(ARCH):$(TAG)
+##@ list and verify:


Could combine with lint and add modules under it.

sedefsavas · 2022-02-10T21:15:55Z

Makefile

-verify-boilerplate:
-	-rm ./hack/tools/bin/*.sh
-	./hack/verify-boilerplate.sh
+##@ tools:


Don't need this IMO, everything under it has a suitable category.

Skarlso · 2022-02-11T08:47:25Z

I added some descriptions which where missing, therefore, they weren't in the help at all. That might have been maybe intentional? I can remove them if desired. Further, .build and .* targets aren't rendered by the help. I can fix that if desired, if not then ¯\_(ツ)_/¯

➜  cluster-api-provider-aws git:(group_makefile) ✗ make

Usage:
  make <target>

generate:
  generate                                  Generate code
  generate-go-apis                          Alias for .build/generate-go-apis
  defaulters                                Generate all Go types

lint and verify:
  modules                                   Runs go mod to ensure proper vendoring.
  lint                                      Lint codebase
  lint-fix                                  Lint the codebase and run auto-fixers if supported by the linter
  verify-boilerplate                        Verify boilerplate
  verify-modules                            Verify go modules integrity
  verify-conversions                        Verifies expected API conversion are in place
  verify-shellcheck                         Verify shell files
  verify-gen                                Verify generated files

build:
  binaries                                  Builds and installs all binaries
  managers                                  Alias for manager-aws-infrastructure
  manager-aws-infrastructure                Build manager binary
  clusterawsadm                             Build clusterawsadm binary
  docker-pull-prerequisites                 Build prerequisites using docker
  docker-build                              Build the docker image for controller-manager

test:
  test                                      Run tests
  generate-test-flavors                     Generate test template flavors
  test-conformance                          Run clusterctl based conformance test on workload cluster (requires Docker).
  test-cover                                Run tests with code coverage and code generate  reports
  serve-book                                Run a server with the documentation book

release:
  build-toolchain                           Build the toolchain
  check-github-token                        Check if the github token is set
  check-previous-release-tag                Check if the previous release tag is set
  check-release-tag                         Check if the release tag is set
  create-gh-release                         Create release on Github
  compiled-manifest                         Compile the manifest files
  docker-push                               Push the docker image
  docker-push-core-manifest                 Push the fat manifest docker image.
  docker-push-manifest                      Push the manifest image
  image-patch-source-manifest               Patch the source manifest
  image-patch-kustomization                 Alias for image-patch-kustomization-without-webhook
  image-patch-kustomization-without-webhook  Patch the image in the kustomization file
  image-patch-pull-policy                   Patch the pull policy
  list-staging-releases                     List staging images for image promotion
  list-image                                List images for RELEASE_TAG
  release                                   Builds and push container images using the latest git tag for the commit.
  release-policies                          Release policies
  release-manifests                         Release manifest files
  release-changelog                         Builds the changelog for a release
  release-binaries                          Builds the binaries to publish with a release
  release-binary                            Release binary
  release-staging                           Builds and push container images and manifests to the staging bucket.
  release-staging-nightly                   Tags and push container images to the staging bucket.
  release-templates                         Generate release templates
  upload-staging-artifacts                  Upload release artifacts to the staging bucket
  upload-gh-artifacts                       Upload artifacts to Github release
  staging-manifests                         Staging manifests

clean:
  clean                                     Remove all generated files
  clean-bin                                 Remove all generated binaries
  clean-temporary                           Remove all temporary files and folders
  clean-release                             Remove the release folder
  clean-artifacts                           Remove the _artifacts directory

help:
  help                                      Display this help

Skarlso · 2022-02-12T13:37:56Z

/retest

Skarlso · 2022-02-12T13:38:55Z

I just changed the makefile ordering, I don't think I messed with the tests. Unless, I accidentally deleted a make target maybe?

Skarlso · 2022-02-12T14:07:47Z

Yaaay, seems like the tests passed.

shivi28 · 2022-02-12T17:21:46Z

Hey @Skarlso Thanks for the PR.
Yes tests were not failing because of your changes, there was some issue with test pipeline, it's been fixed now.

I have one suggestion, can we sort the categories command alphabetically. Wdyt?
eg

test:
  test                                      Run tests
  generate-test-flavors                     Generate test template flavors
  test-conformance                          Run clusterctl based conformance test on workload cluster (requires Docker).
  test-cover                                Run tests with code coverage and code generate  reports
  serve-book                                Run a server with the documentation book

can be re-written as

test:
  generate-test-flavors                     Generate test template flavors
  test                                      Run tests
  test-conformance                          Run clusterctl based conformance test on workload cluster (requires Docker).
  test-cover                                Run tests with code coverage and code generate  reports
  serve-book                                Run a server with the documentation book

Same we can do with others also.

shivi28 · 2022-02-12T17:24:06Z

And once all changes are done, we can squash all the commits to single comment

Skarlso · 2022-02-12T17:36:50Z

@shivi28 sure, no problem.

shivi28 · 2022-02-12T18:47:07Z

/lgtm

sedefsavas · 2022-02-14T08:13:20Z

/approve

k8s-ci-robot · 2022-02-14T08:13:33Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sedefsavas

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sedefsavas]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-priority labels Feb 10, 2022

k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Feb 10, 2022

k8s-ci-robot requested review from dlipovetsky and sedefsavas February 10, 2022 20:08

k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Feb 10, 2022

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Feb 10, 2022

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Feb 10, 2022

Skarlso force-pushed the group_makefile branch from 61f8cd9 to 6655036 Compare February 10, 2022 20:49

sedefsavas reviewed Feb 10, 2022

View reviewed changes

Skarlso force-pushed the group_makefile branch from 2142ba9 to 91a305b Compare February 12, 2022 13:26

Group Makefile targets

ce2ba25

Skarlso force-pushed the group_makefile branch from 309ac74 to ce2ba25 Compare February 12, 2022 17:43

k8s-ci-robot assigned shivi28 Feb 12, 2022

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 12, 2022

sedefsavas added this to the v1.4.0 milestone Feb 14, 2022

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 14, 2022

k8s-ci-robot merged commit c30df60 into kubernetes-sigs:main Feb 14, 2022

k8s-ci-robot modified the milestones: v1.4.0, v1.x Feb 14, 2022

Skarlso deleted the group_makefile branch February 14, 2022 08:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Group Makefile targets #3192

Group Makefile targets #3192

Skarlso commented Feb 10, 2022

k8s-ci-robot commented Feb 10, 2022

k8s-ci-robot commented Feb 10, 2022

Skarlso commented Feb 10, 2022

sedefsavas commented Feb 10, 2022

sedefsavas commented Feb 10, 2022 •

edited

Skarlso commented Feb 10, 2022 •

edited

sedefsavas Feb 10, 2022

Skarlso Feb 11, 2022

sedefsavas Feb 10, 2022

sedefsavas Feb 10, 2022

sedefsavas Feb 10, 2022

Skarlso commented Feb 11, 2022

Skarlso commented Feb 12, 2022

Skarlso commented Feb 12, 2022

Skarlso commented Feb 12, 2022

shivi28 commented Feb 12, 2022

shivi28 commented Feb 12, 2022

Skarlso commented Feb 12, 2022

shivi28 commented Feb 12, 2022

sedefsavas commented Feb 14, 2022

k8s-ci-robot commented Feb 14, 2022

Group Makefile targets #3192

Group Makefile targets #3192

Conversation

Skarlso commented Feb 10, 2022

k8s-ci-robot commented Feb 10, 2022

k8s-ci-robot commented Feb 10, 2022

Skarlso commented Feb 10, 2022

sedefsavas commented Feb 10, 2022

sedefsavas commented Feb 10, 2022 • edited

Skarlso commented Feb 10, 2022 • edited

sedefsavas Feb 10, 2022

Choose a reason for hiding this comment

Skarlso Feb 11, 2022

Choose a reason for hiding this comment

sedefsavas Feb 10, 2022

Choose a reason for hiding this comment

sedefsavas Feb 10, 2022

Choose a reason for hiding this comment

sedefsavas Feb 10, 2022

Choose a reason for hiding this comment

Skarlso commented Feb 11, 2022

Skarlso commented Feb 12, 2022

Skarlso commented Feb 12, 2022

Skarlso commented Feb 12, 2022

shivi28 commented Feb 12, 2022

shivi28 commented Feb 12, 2022

Skarlso commented Feb 12, 2022

shivi28 commented Feb 12, 2022

sedefsavas commented Feb 14, 2022

k8s-ci-robot commented Feb 14, 2022

sedefsavas commented Feb 10, 2022 •

edited

Skarlso commented Feb 10, 2022 •

edited