Speed up code generation ("make codegen") (#6465)

Running "make codegen" only takes 5 minutes in CI, but it takes upwards of 20
minutes locally on macOS. This is because the code generation is disk I/O heavy,
and the bind mount of the Antrea source code into the Docker container is very
slow on non-Linux hosts (using Docker Desktop or Colima).

To speed up code generation, we implement the following 2 changes:

1) Before generating any code, we make a copy of the source tree from the bind
   mount to a local directory on the container's writable layer. After that, all
   reads and writes happen in the copy, which is much faster than the bind
   mount. When we are done with code generation, we copy generated files from
   the copy back to the original repository as needed.
2) We mount gopkgmod and the gocache as volumes into the container, which
   persist across runs. This speeds up the code generation process at the cost
   of a few GBs of storage on the local host.

Change 1) uses "git clone" to make a copy of the source tree, which is the most
efficient and simple solution. However, it requires that developers first
commit all their changes before running "make codegen". Otherwise, uncommitted
changes will be ignored, which can impact code generation. However, I have found
that it is always a good idea to commit changes beforehand anyway: it makes it
easier to check the generated code changes with "git diff" and it makes it
easier to rollback changes if needed. Running "make codegen" with committing
local changes will result in an error from now on.

We also simplify code vendoring (used to include Protobuf dependencies) in
hack/update-codegen-dockerized.sh.

Before this change, running "make codegen" would take 22 minutes on my macOS
laptop. After this change, the first run of "make codegen" takes 9-10 minutes,
and subsequent runs take 3-4 minutes.

Signed-off-by: Antonin Bas <antonin.bas@broadcom.com>

ci/check-codegen.sh

@@ -25,9 +25,6 @@ function echoerr {
 THIS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
 pushd $THIS_DIR/.. > /dev/null
 
-git grep --files-with-matches -e 'Code generated by .* DO NOT EDIT' pkg | xargs rm
-git grep --files-with-matches -e 'This file was autogenerated by go-to-protobuf' pkg | xargs rm
-
 make codegen
 cd multicluster;make codegen;cd ..
 diff="$(git status --porcelain pkg multicluster)"

docs/contributors/code-generation.md

@@ -6,7 +6,8 @@ Antrea uses [protoc](https://github.com/protocolbuffers/protobuf), [protoc-gen-g
 and [protoc-gen-go-grpc](https://github.com/grpc/grpc-go) to generate CNI gRPC service code.
 
 If you make any change to [cni.proto](../../pkg/apis/cni/v1beta1/cni.proto), you can
-re-generate the code by invoking `make codegen`.
+re-generate the code by invoking `make codegen`. Make sure that you commit your changes before
+running the script, and that you commit the generated code after running it.
 
 ## Extension API Resources and Custom Resource Definitions
 
@@ -18,7 +19,9 @@ generated codes are located in the conventional paths: `pkg/apis/<resource group
 types and `pkg/apis/<resource group>/<version>` for versioned types and `pkg/client/clientset` for
 clients.
 
-If you make any change to any `types.go`, you can re-generate the code by invoking `make codegen`.
+If you make any change to any `types.go`, you can re-generate the code by invoking `make
+codegen`. Make sure that you commit your changes before running the script, and that you commit the
+generated code after running it.
 
 ## Mocks
 
@@ -31,6 +34,8 @@ code for the interface `Baz` defined in the package `pkg/foo/bar` will be genera
 `pkg/foo/bar/testing/mock_bar.go`, and you can import it via `pkg/foo/bar/testing`.
 
 Same as above, you can re-generate the mock source code (with `mockgen`) by invoking `make codegen`.
+Make sure that you commit your changes before running the script, and that you commit the
+generated code after running it.
 
 ## Generated Documentation
 

hack/update-codegen-common.sh

@@ -49,6 +49,7 @@ function generate_mocks {
 }
 
 function reset_year_change {
+  oldstate=$(set +o)
   set +x
   echo "=== Start resetting changes introduced by YEAR ==="
   # The call to 'tac' ensures that we cannot have concurrent git processes, by
@@ -60,4 +61,6 @@ function reset_year_change {
       echo "=== ${file} is reset ==="
     fi
   done
+  # restore options
+  eval "$oldstate"
 }

hack/update-codegen-dockerized.sh

@@ -36,6 +36,18 @@ ANTREA_PKG="antrea.io/antrea"
 # https://github.com/kubernetes-sigs/gateway-api/issues/11
 ANTREA_PROTO_PKG="antrea_io.antrea"
 
+# We make a temporary working copy of the source repository using git clone. The
+# copy is in the container's writable layer, which is much faster than a bind
+# mount on non-Linux hosts (Docker Desktop, Colima).
+ANTREA_SRC_PATH=$(pwd)
+ANTREA_CODEGEN_PATH=/go/src/antrea.io/antrea
+git clone ${ANTREA_SRC_PATH} ${ANTREA_CODEGEN_PATH}
+pushd ${ANTREA_CODEGEN_PATH}
+
+# Remove all auto-generated files
+git grep --files-with-matches -e 'Code generated by .* DO NOT EDIT' pkg | xargs rm
+git grep --files-with-matches -e 'This file was autogenerated by go-to-protobuf' pkg | xargs rm
+
 MOCKGEN_TARGETS=(
   "pkg/agent/cniserver SriovNet testing"
   "pkg/agent/cniserver/ipam IPAMDriver testing"
@@ -160,21 +172,31 @@ generate_mocks
 
 # Download vendored modules to the vendor directory so it's easier to
 # specify the search path of required protobuf files.
+# Explicitly set GOFLAGS to ignore vendor, since GOFLAGS=-mod=vendor breaks
+# dependency resolution while rebuilding vendor.
+export GOFLAGS="-mod=mod"
 go mod vendor
-# In Go 1.14, vendoring changed (see release notes at
-# https://golang.org/doc/go1.14), and the presence of a go.mod file specifying
-# go 1.14 or higher causes the go command to default to -mod=vendor when a
-# top-level vendor directory is present in the module. This causes the
-# go-to-protobuf command below to complain about missing packages under vendor/,
-# which were not downloaded by "go mod vendor". We can workaround this easily by
-# renaming the vendor directory.
-mv vendor /tmp/includes
 PACKAGES="${ANTREA_PKG}/pkg/apis/stats/v1alpha1=${ANTREA_PROTO_PKG}.pkg.apis.stats.v1alpha1,\
 ${ANTREA_PKG}/pkg/apis/controlplane/v1beta2=${ANTREA_PROTO_PKG}.pkg.apis.controlplane.v1beta2"
 $GOPATH/bin/go-to-protobuf \
-  --proto-import /tmp/includes \
+  --proto-import vendor \
   --packages "${PACKAGES}" \
   --go-header-file hack/boilerplate/license_header.go.txt
-rm -rf /tmp/includes
+
+rm -rf vendor
 
 reset_year_change
+
+# Copy generated code back into the source repository, which was cloned at the
+# beginning of this script to make a temporary working copy.
+git ls-files --modified --others --exclude-standard | while read file; do
+    if [ -e $file ]; then
+        cp $file ${ANTREA_SRC_PATH}/$file
+    else
+        rm ${ANTREA_SRC_PATH}/$file
+    fi
+done
+
+popd
+
+rm -rf ${ANTREA_CODEGEN_PATH}

hack/update-codegen.sh

@@ -17,31 +17,49 @@
 set -o errexit
 set -o pipefail
 
+function echoerr {
+    >&2 echo "$@"
+}
+
 ANTREA_ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )/../" && pwd )"
 IMAGE_NAME="antrea/codegen:kubernetes-1.29.2-build.1"
 
-# Recent versions of Git will not access .git directories which are owned by
-# another user (as a security measure), unless the directories are explicitly
-# added to a "safe" list in the Git config. When we run the Docker container,
-# the Antrea source directory may be owned (depends on the Docker platform)
-# by a user which is different from the container user (as the source directory
-# is mounted from the host). If this is the case, the Git program inside the
-# container will refuse to run. This is why we explicitly add the Antrea source
-# directory to the list of "safe" directories. We are still looking into the
-# possibility of running the Docker container as the "current host user".
+# We will use git clone to make a working copy of the repository into a
+# temporary directory. This requires that all changes have been committed
+# otherwise the generated code may not be up-to-date. It is anyway good practice
+# to commit changes before auto-generating code, in case there is an issue with
+# the script.
+# We run these checks here instead of inside the Docker container for 2 reasons:
+# speed (bind mounts are slow) and to avoid having to add the source repository
+# to the "safe" list in the Git config when starting the container (required
+# because of user mismatch).
+if git_status=$(git status --porcelain --untracked=no 2>/dev/null) && [[ -n "${git_status}" ]]; then
+  echoerr "!!! Dirty tree. Clean up and try again."
+  exit 1
+fi
+# It is very common to have untracked files in a repository, so we only give a
+# warning if we find untracked Golang source files.
+untracked=$(git ls-files --others --exclude-standard '**.go')
+if [ -n "$untracked" ]; then
+    echoerr "The following Golang files are untracked, and will be ignored by code generation:"
+    echoerr "$untracked"
+fi
+
 function docker_run() {
   # Silence CLI suggestions.
   export DOCKER_CLI_HINTS=false
   docker pull ${IMAGE_NAME}
   set -x
-  ANTREA_PATH="/go/src/antrea.io/antrea"
+  ANTREA_SRC_PATH="/mnt/antrea"
   docker run --rm \
 		-e GOPROXY=${GOPROXY} \
 		-e HTTP_PROXY=${HTTP_PROXY} \
 		-e HTTPS_PROXY=${HTTPS_PROXY} \
-		-w ${ANTREA_PATH} \
-		-v ${ANTREA_ROOT}:${ANTREA_PATH} \
-		"${IMAGE_NAME}" bash -c "git config --global --add safe.directory ${ANTREA_PATH} && $@"
+		-w ${ANTREA_SRC_PATH} \
+                --mount type=bind,source=${ANTREA_ROOT},target=${ANTREA_SRC_PATH} \
+                --mount type=volume,source=antrea-codegen-gopkgmod,target=/go/pkg/mod \
+                --mount type=volume,source=antrea-codegen-gocache,target=/root/.cache/go-build \
+		"${IMAGE_NAME}" bash -c "$@"
 }
 
 # Combine hack/update-codegen-dockerized.sh and the arguments to the script as a