Use a private tag in commit messages

* Use --EKS-PRIVATE-- in commit messages
  to determine which commits are intended
  to be private.
* When the prepare_patches.sh script tries
  to determine the number of private patches,
  it looks at the EKSDataPlaneKubernetes
  repository, so the number of private and public
  patches is based off the patches you have
  in EKSDataPlaneKubernetes.
* Use 0-public and 1-private for ordering sanity
* Remove individual files instead of entire directory.  Structure
  remove in a loop with a string concatenation so the target path
  is never empty.
* Add max results patch for kubernetes#106280
* See the upstream PR
  (kubernetes#106280)
* DROP in 1.24

cr: https://code.amazon.com/reviews/CR-64014162

README.md

@@ -2,10 +2,29 @@
 
 ## Summary
 
-This package is used to track patches that EKS applies on top of upstream [kubernetes](https://github.com/kubernetes/kubernetes).
-EKSDataPlaneCDK clones this repo and applies the patches on top of upstream based on the GIT_TAG.
+This package is used to track patches that EKS applies on top of upstream
+[kubernetes](https://github.com/kubernetes/kubernetes).  EKSDataPlaneCDK clones
+this repo and applies the patches on top of upstream based on the GIT_TAG.
 
-## Development
+## Patch Development
+
+### Note About EKS Patches
+
+Patches are cherry-picks or custom commits that are applied to the upstream
+Kubernetes codebase before we build binaries used in EKS.  Every patch that is
+not present in upstream should have the marker --EKS-PATCH-- at the beginning
+of the first line of the commit message.
+
+Additionally, any patch that is not going to be published to
+[eks-distro](https://github.com/aws/eks-distro), should have the
+--EKS-PRIVATE-- marker somewhere in the commit message (for readability, at the
+beginning of the second or last line).
+
+The public patches (those present in eks-distro) are applied first to upstream
+code, followed by our private patches.  This allows the public patch files to
+be used by eks-distro without modification.
+
+### Setup
 
 Clone this repo and the gitfarm kubernetes repository.
 ```
@@ -17,7 +36,21 @@ $ brazil ws use -p EKSKubernetesPatches
 $ cd src/EKSKubernetesPatches/
 ```
 
-Apply the patches. Make sure the EKSDataPlaneKubernetes repository is clean because the script will modify it.
+Note: kubernetes is a large repository.  If you are on a slow internet
+connection, and already have EKSDataPlaneKubernetes cloned, you can link soft
+link it to the desired location:
+
+```
+$ ln -s ~/workplace/EKSDataPlaneKubernetes/src/EKSDataPlaneKubernetes ~/workplace/EKSKubernetesPatches/src/EKSDataPlaneKubernetes
+```
+
+### Modify Existing Patches in EKSKubernetesPatches
+
+In order to modify existing patches in EKSKubernetesPatches, first they must be
+applied to the appropriate git tag in the EKSDataPlaneKubernetes repository.
+Make sure the EKSDataPlaneKubernetes repository is clean because the script
+will modify it.
+
 ```
 $ pushd ~/workplace/EKSKubernetesPatches/src/EKSKubernetesPatches/
 $ cat patches/1.22/GIT_TAG
@@ -26,113 +59,60 @@ $ ./hack/apply_patches.sh patches/1.22 ~/workplace/EKSKubernetesPatches/src/EKSD
 $ popd
 ```
 
-Add, edit, drop, or reorder patches with `git rebase -i`, `git cherry-pick`, etc.
+Now that they are applied to the appropriate tag, you can add, edit, drop, or
+reorder patches with `git rebase -i`, `git cherry-pick`, etc.
+
 ```
 $ pushd ~/workplace/EKSKubernetesPatches/src/EKSDataPlaneKubernetes/
 $ git cherry-pick PATCH-1234
 $ git checkout -b PATCH-1234
 $ popd
 ```
 
-Prepare the new patches. Make sure the EKSKubernetesPatches repository is clean because the script will modify it.
+Next, you must create new patch files from the commits you modified. Make sure
+the EKSKubernetesPatches repository is clean because the script will modify it.
+
 ```
 $ pushd ~/workplace/EKSKubernetesPatches/src/EKSKubernetesPatches/
 $ ./hack/prepare_patches.sh ~/workplace/EKSKubernetesPatches/src/EKSDataPlaneKubernetes/ patches/1.22/
 $ popd
 ```
 
-Check the diff and commit patches accordingly.
-For example:
-- if your intention was only to add one patch, it's not necessary to commit the other patches whose commit hash changed but content did not.
-- if you dropped or reordered patches, then it's necessary to commit all patches because they need to be renamed.
-- if you edited a patch X that modifies a file also modified by a subsequent patch Y then it's necessary to commit both patch X and Y.
-Submit a CR with the prepared patches.
+Check the diff and commit patches accordingly.  For example:
+- if your intention was only to add one patch, it's not necessary to commit the
+  other patches whose commit hash changed but content did not.
+- if you dropped or reordered patches, then it's necessary to commit all
+  patches because they need to be renamed.
+- if you edited a patch X that modifies a file also modified by a subsequent
+  patch Y then it's necessary to commit both patch X and Y.  Submit a CR with
+  the prepared patches.
+
 ```
 $ pushd ~/workplace/EKSKubernetesPatches/src/EKSKubernetesPatches/
 $ git diff
 $ git add patches/1.22/private/0099-PATCH-1234
 $ cr
-$ popd
-```
-
-### Rebasing patches on a new kubernetes version
-
-For a new minor version, copy the preceding directory then edit the GIT_TAG to the new version you wish to rebase the patches on.
-
-For a new patch version, find the existing directory then edit the GIT_TAG.
-
-Then the process is the same as above. When you apply the patches you should expect a patch to fail in which case you must decide to edit or drop it. Submit a cr with the patch edited or dropped. Repeat this process until all patches succeed for the new GIT_TAG.
-
-# ekspatch
-
-Optionally you may use the ekspatch cli to help with some common patch
-manipulation operations.
-
-## Usage
-
-*Always execute ekspatch from the project root*
-
-Execute ekspatch with brazil-runtime-exec:
-```
-brazil-runtime-exec ekspatch --help
-
-Usage: ekspatch [OPTIONS] COMMAND [ARGS]...
-
-Options:
-  --help  Show this message and exit.
-
-Commands:
-  clone
-  create
-  pr
-```
-
-## Clone
-```
-Usage: ekspatch clone [OPTIONS]
-
-Options:
-  --help  Show this message and exit.
+The branch you're on doesn't track a GitFarm remote. Inferring your --parent to be '201ccfcfb' on branch 'mainline'.
+ Running pre-cr hook /home/ANT.AMAZON.COM/nic/workspace/EKSKubernetesPatches/src/EKSKubernetesPatches/pre-cr
+ Apply patches and create an EKSDataPlaneKubernetes CR too? It will be easier to review your EKSDataPlanePatches CR with a corresponding EKSDataPlaneKubernetes CR showing the applied patches. y/n?
 ```
 
-Clone the kubernetes codecommit repository which is a mirror of the gitfarm repository.
+You should choose yes when working on a change to patches.
 
-## Create Patches
 ```
-brazil-runtime-exec ekspatch create --help
-
-Usage: ekspatch create [OPTIONS]
-
-Options:
-  -e, --eks-tag TEXT  The eks tag, formatted as
-                      v<major>.<minor>.<patch>-eks-<short sha>
-                      (v1.12.10-eks-a26503).
-  --help              Show this message and exit.
-```
-
-Create creates patch files from a branch that already exists on the EKS codecommit repo.  When releasing a new Kubernetes version for EKS, you should ensure the following things to be true:
-
-1. A release branch is created from an upstream version tag.  For example, if the upstream version 1.12.10 is chosen, then v1.12.10 is the upstream version tag.  The release branch should be called release-1.12.10-eks (yes, it should have the patch version).
-2. If necessary, a number of patches are applied to the release branch.
-3. After all the patches are applied, the last commit is tagged with the eks tag.  For example, 1.12.10-eks-abc123, where abc123 is first 6 digits of the commit SHA.
-
-As long as all the following conventions are followed, then you can create the patches by running the command:
-```
-brazil-runtime-exec ekspatch create --eks-tag v1.12.10-eks-abc123
+$ popd
 ```
 
-And the formatted patch files will be added to `./patches/v1.12.10-eks-abc123/...`.
+### Rebasing Patches on a New Kubernetes Version
 
+For a new minor version, copy the preceding directory then edit the GIT_TAG to
+the new version you wish to rebase the patches on.  For a new patch version,
+find the existing directory then edit the GIT_TAG.  Commit these changes before
+you make any changes to patches.
 
-## Pull Request
-```
-brazil-runtime-exec ekspatch pr --help
+Then the process is the same as above. When you apply the patches you should
+expect a patch to fail in which case you must decide to edit or drop it. Submit
+a cr with the patch edited or dropped. Repeat this process until all patches
+succeed for the new GIT_TAG.
 
-Usage: ekspatch pr [OPTIONS]
-
-Options:
-  --id TEXT  The github PR id to create a patch from.
-  --help     Show this message and exit.
-```
 
-Create a patch from a github pull request.  This creates `./patches/<pr-id>/<pr-id>.patch` and `./patches/<pr-id>/<pr-id>-metadata.json`, where the metadata file hold some information about the pull request and patch.  After the patch file is created here, it should still be applied to a release branch.

hack/lib/patch.sh

@@ -59,18 +59,17 @@ function apply_patches {
 function apply_patches_all {
   local PARENT_PATCHES_DIR=$1
   local KUBERNETES_DIR=$2
+  local PUBLIC_PATCHES_DIR="$PARENT_PATCHES_DIR/0-public"
+  local PRIVATE_PATCHES_DIR="$PARENT_PATCHES_DIR/1-private"
 
-  local VERSION
-  VERSION=$(get_version "$PARENT_PATCHES_DIR")
+  local VERSION=$(get_version "$PARENT_PATCHES_DIR")
   echo "Checking out $VERSION in $KUBERNETES_DIR"
   checkout_kubernetes "$VERSION" "$KUBERNETES_DIR"
   echo "$VERSION checked out!"
 
   echo "Applying patches in $PARENT_PATCHES_DIR to $KUBERNETES_DIR..."
-  local PUBLIC_PATCHES_DIR="$PARENT_PATCHES_DIR/public"
   echo "Applying public patches in $PUBLIC_PATCHES_DIR to $KUBERNETES_DIR..."
   if apply_patches "$PUBLIC_PATCHES_DIR" "$KUBERNETES_DIR"; then
-    local PRIVATE_PATCHES_DIR="$PARENT_PATCHES_DIR/private"
     echo "Applying private patches in $PRIVATE_PATCHES_DIR to $KUBERNETES_DIR..."
     if apply_patches "$PRIVATE_PATCHES_DIR" "$KUBERNETES_DIR"; then
       echo "All patches succeeded!"
@@ -104,39 +103,68 @@ function prepare_patches {
 function prepare_patches_public {
   local KUBERNETES_DIR=$1
   local PARENT_PATCHES_DIR=$2
+  local PUBLIC_PATCHES_DIR="$PARENT_PATCHES_DIR/0-public"
 
-  local PRIVATE_PATCHES_DIR="$PARENT_PATCHES_DIR/public"
-  local NUM_PRIVATE
-  NUM_PRIVATE=$(ls "$PRIVATE_PATCHES_DIR" | wc -l)
-
-  local PUBLIC_PATCHES_DIR="$PARENT_PATCHES_DIR/public"
+  local RANGE_START=$(cat "$PARENT_PATCHES_DIR"/GIT_TAG)
+  local NUM_PRIVATE=$(num_private $KUBERNETES_DIR $RANGE_START HEAD)
   local START_NUM=1
-  local RANGE_START
-  RANGE_START=$(cat "$PARENT_PATCHES_DIR"/GIT_TAG)
   local RANGE_END=HEAD~"$NUM_PRIVATE"
 
+  remove_patches "${PUBLIC_PATCHES_DIR}"
+  mkdir -p "${PUBLIC_PATCHES_DIR}"
   prepare_patches "$KUBERNETES_DIR" "$PUBLIC_PATCHES_DIR" "$START_NUM" "$RANGE_START" "$RANGE_END"
 }
 
 function prepare_patches_private {
   local KUBERNETES_DIR=$1
   local PARENT_PATCHES_DIR=$2
+  local PRIVATE_PATCHES_DIR="$PARENT_PATCHES_DIR/1-private"
 
-  local PRIVATE_PATCHES_DIR="$PARENT_PATCHES_DIR/private"
-  local NUM_PRIVATE
-  NUM_PRIVATE=$(ls "$PRIVATE_PATCHES_DIR" | wc -l)
-
-  local START_NUM=$(("$NUM_PRIVATE" + 1))
+  local RANGE_START=$(cat "$PARENT_PATCHES_DIR"/GIT_TAG)
+  local NUM_PUBLIC=$(num_public $KUBERNETES_DIR $RANGE_START HEAD)
+  local NUM_PRIVATE=$(num_private $KUBERNETES_DIR $RANGE_START HEAD)
+  local START_NUM="$(($NUM_PUBLIC + 1))"
   local RANGE_START=HEAD~"$NUM_PRIVATE"
   local RANGE_END=HEAD
 
+  remove_patches "${PRIVATE_PATCHES_DIR}"
+  mkdir -p "${PRIVATE_PATCHES_DIR}"
   prepare_patches "$KUBERNETES_DIR" "$PRIVATE_PATCHES_DIR" "$START_NUM" "$RANGE_START" "$RANGE_END"
 }
 
+function num_private() {
+  local KUBERNETES_DIR=$1
+  local RANGE_START=$2
+  local RANGE_END=$3
+
+  pushd "$KUBERNETES_DIR"
+  echo "$(git rev-list --reverse --grep='--EKS-PRIVATE--' ${RANGE_START}..${RANGE_END})" | wc -l
+  popd
+}
+
+function num_public() {
+  local KUBERNETES_DIR=$1
+  local RANGE_START=$2
+  local RANGE_END=$3
+
+  pushd "$KUBERNETES_DIR"
+  echo "$(git rev-list --reverse --invert-grep --grep='--EKS-PRIVATE--' ${RANGE_START}..${RANGE_END})" | wc -l
+  popd
+}
+
 function prepare_patches_all {
   local KUBERNETES_DIR=$1
   local PARENT_PATCHES_DIR=$2
 
   prepare_patches_public "$KUBERNETES_DIR" "$PARENT_PATCHES_DIR"
   prepare_patches_private "$KUBERNETES_DIR" "$PARENT_PATCHES_DIR"
 }
+
+function remove_patches() {
+  local PATCHES_DIR=$1
+
+  files=($(ls $PATCHES_DIR))
+  for file in ${files[@]}; do
+      rm ${PATCHES_DIR}/$file
+  done
+}

hack/prepare_patches.sh

@@ -11,10 +11,10 @@ PARENT_PATCHES_DIR=$(realpath "$2")
 
 check_dirty "$REPO_ROOT"
 PARENT_PATCHES_DIR_PATTERN=".*/patches/1.[0-9][0-9]"
-if [[ "$PARENT_PATCHES_DIR" =~ $PARENT_PATCHES_DIR_PATTERN/public ]]; then
+if [[ "$PARENT_PATCHES_DIR" =~ $PARENT_PATCHES_DIR_PATTERN/0-public ]]; then
   PARENT_PATCHES_DIR=$(realpath "$PARENT_PATCHES_DIR"/..)
   prepare_patches_public "$KUBERNETES_DIR" "$PARENT_PATCHES_DIR"
-elif [[ "$PARENT_PATCHES_DIR" =~ $PARENT_PATCHES_DIR_PATTERN/private ]]; then
+elif [[ "$PARENT_PATCHES_DIR" =~ $PARENT_PATCHES_DIR_PATTERN/1-private ]]; then
   PARENT_PATCHES_DIR=$(realpath "$PARENT_PATCHES_DIR"/..)
   prepare_patches_private "$KUBERNETES_DIR" "$PARENT_PATCHES_DIR"
 elif [[ "$PARENT_PATCHES_DIR" =~ $PARENT_PATCHES_DIR_PATTERN ]]; then

patches/1.22/public/0001-EKS-PATCH-Added-allowlist-CIDR-flag-use-klog.patch → patches/1.22/0-public/0001-EKS-PATCH-Added-allowlist-CIDR-flag-use-klog.patch

@@ -1,7 +1,7 @@
-From b67824287c8b6102e6c50530392fc80d75453223 Mon Sep 17 00:00:00 2001
+From 15876272c586521e887d6549e4e5eddcf474b1e6 Mon Sep 17 00:00:00 2001
 From: Micah Hausler <mhausler@amazon.com>
 Date: Wed, 19 Sep 2018 18:16:23 -0700
-Subject: [PATCH 1/7] --EKS-PATCH-- Added allowlist CIDR flag, use "klog"
+Subject: [PATCH] --EKS-PATCH-- Added allowlist CIDR flag, use "klog"
 
 Alternative to https://github.com/kubernetes/kubernetes/pull/71980
 
@@ -821,6 +821,3 @@ index 00000000000..d3044053ca4
 +	}
 +	return service
 +}
--- 
-2.17.1
-

patches/1.22/public/0002-EKS-PATCH-Pass-region-to-sts-client.patch → patches/1.22/0-public/0002-EKS-PATCH-Pass-region-to-sts-client.patch

@@ -1,7 +1,7 @@
-From a26766258d6725c99e7cf360b45092121e41268d Mon Sep 17 00:00:00 2001
+From 46b270ba5099eae037a563ac06da74c773bd134e Mon Sep 17 00:00:00 2001
 From: Rasita Pai <prasita@amazon.com>
 Date: Wed, 13 Oct 2021 10:46:18 -0700
-Subject: [PATCH 2/7] --EKS-PATCH-- Pass region to sts client
+Subject: [PATCH] --EKS-PATCH-- Pass region to sts client
 
 First, this change explicitly sets the region from metadata.
 Second, it also uses .WithSTSRegionalEndpoint(endpoints.RegionalSTSEndpoint)
@@ -64,6 +64,3 @@ index d21e2cd7d44..b8a67e0693f 100644
  // readAWSCloudConfig reads an instance of AWSCloudConfig from config reader.
  func readAWSCloudConfig(config io.Reader) (*CloudConfig, error) {
  	var cfg CloudConfig
--- 
-2.17.1
-

patches/1.22/public/0003-EKS-PATCH-Bypassed-admission-controller-webhook-for-.patch → patches/1.22/0-public/0003-EKS-PATCH-Bypassed-admission-controller-webhook-for-.patch

@@ -1,7 +1,7 @@
-From 69c26420e07ce2ef5d2ac592e752c305feaa4028 Mon Sep 17 00:00:00 2001
+From 5f4a1446d47eb469488187cc9f9d2d5ac45a69b7 Mon Sep 17 00:00:00 2001
 From: Qing Ju <juqing@amazon.com>
 Date: Sun, 18 Oct 2020 10:31:39 -0700
-Subject: [PATCH 3/7] --EKS-PATCH-- Bypassed admission controller webhook for
+Subject: [PATCH] --EKS-PATCH-- Bypassed admission controller webhook for
  cluster critical resources
 
 Workaround for https://github.com/kubernetes/kubernetes/issues/92157
@@ -203,6 +203,3 @@ index 26dbefc1991..f2db46707b2 100644
  func TestShouldCallHook(t *testing.T) {
  	a := &Webhook{namespaceMatcher: &namespace.Matcher{}, objectMatcher: &object.Matcher{}}
 
--- 
-2.17.1
-

patches/1.22/public/0004-EKS-PATCH-Use-GNU-date.patch → patches/1.22/0-public/0004-EKS-PATCH-Use-GNU-date.patch

@@ -1,7 +1,7 @@
-From e0cb409b887ffebf1b8ca0f2680ba3c65fe9844c Mon Sep 17 00:00:00 2001
+From 315ff0f78bbcdc97a7caa280a0fa91ae063a3e39 Mon Sep 17 00:00:00 2001
 From: Micah Hausler <mhausler@amazon.com>
 Date: Sun, 1 Nov 2020 09:24:08 -0800
-Subject: [PATCH 4/7] --EKS-PATCH-- Use GNU date
+Subject: [PATCH] --EKS-PATCH-- Use GNU date
 
 Use GNU date if available for builds on darwin
 
@@ -30,6 +30,3 @@ index a14c7261aaa..e861312c45f 100644
    if [[ -n ${KUBE_GIT_COMMIT-} ]]; then
      add_ldflag "gitCommit" "${KUBE_GIT_COMMIT}"
      add_ldflag "gitTreeState" "${KUBE_GIT_TREE_STATE}"
--- 
-2.17.1
-