docs: Add Makefile and documentation for "fast" development targets

This commit introduces documentation for "fast" development targets in
Cilium. The "fast" targets allow for quicker development workflows by
building Cilium locally and deploying it on an existing kind cluster.
This approach eliminates the need to create and deploy Docker images
each time a source code change is made.

Additionally, the documentation has been updated to include these new
Makefile targets and provides a quick start guide with essential
information for working with Cilium.

When a cluster is already running with a Cilium container, we can
simply volume mount the binaries into the container and restart the
pod to pick up the changes. A similar approach can be used for C code,
with the caveat that the bpf/lib directory needs write permissions by
Cilium to generate the C Header features file, making it impossible
to use the volume as read-only. Since all nodes share the same kernel
features, there won't be any conflicts when multiple nodes write
to the same volume mount.

Signed-off-by: André Martins <andre@cilium.io>

Documentation/contributing/development/dev_setup.rst

@@ -10,35 +10,52 @@
 Development Setup
 =================
 
-Dev Container
-~~~~~~~~~~~~~
+This page provides an overview of different methods for efficient
+development on Cilium. Depending on your needs, you can choose the most
+suitable method.
 
-Cilium provides `Dev Container <https://code.visualstudio.com/docs/devcontainers/containers>`_ configuration for Visual Studio Code Remote Containers
-and `Github Codespaces <https://docs.github.com/en/codespaces/setting-up-your-project-for-codespaces/introduction-to-dev-containers>`_.
-This allows you to use a preconfigured development environment in the cloud or locally.
-The container is based on the official Cilium builder image and provides all the dependencies
-required to build Cilium.
+Quick Start
+-----------
 
-.. note::
+If you're in a hurry, here are the essential steps to get started:
 
-    The current Dev Container is running as root. Non-root user support requires non-root
-    user in Cilium builder image, which is related to :gh-issue:`23217`.
+On Linux:
+
+1. ``make kind`` - Provisions a Kind cluster.
+2. ``make kind-install-cilium-fast`` - Installs Cilium on the Kind cluster.
+3. ``make kind-image-fast`` - Builds Cilium and deploys it.
+
+On any OS:
+
+1. ``make kind`` - Provisions a Kind cluster.
+2. ``make kind-image`` - Builds Docker images.
+3. ``make kind-install-cilium`` - Installs Cilium on the Kind cluster.
+
+Detailed Instructions
+---------------------
+
+Depending on your specific development environment and requirements, you
+can follow the detailed instructions below.
 
 Verifying Your Development Setup
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Assuming you have Go installed, you can quickly verify many elements of your
-development setup by running:
+development setup by running the following command:
 
 .. code-block:: shell-session
 
     $ make dev-doctor
 
-Requirements
-~~~~~~~~~~~~
+Depending on your end-goal, not all dependencies listed are required to develop
+on Cilium. For example, "Ginkgo" is not required if you want to improve our
+documentation. Thus, do not consider that you need to have all tools installed.
+
+Version Requirements
+~~~~~~~~~~~~~~~~~~~~
 
-You need to have the following tools available in order to effectively
-contribute to Cilium:
+If using these tools, you need to have the following versions from them
+in order to effectively contribute to Cilium:
 
 +--------------------------------------------------------------+------------------------------+-----------------------------------------------------------------+
 | Dependency                                                   | Version / Commit ID          | Download Command                                                |
@@ -90,16 +107,25 @@ You can find the setup for a `kind <https://kind.sigs.k8s.io/>`_ environment in
 VirtualBox on Linux, but does require `Docker for Mac
 <https://docs.docker.com/desktop/install/mac-install/>`_ for Mac OS.
 
-Makefile targets automate the task of spinning up an environment and building
-Cilium images:
+Makefile targets automate the task of spinning up an environment:
 
 * ``make kind``: Creates a kind cluster based on the configuration passed in.
-  For more information, see _`Configuration for clusters`.
+  For more information, see `configurations_for_clusters`.
+* ``make kind-down``: Tears down and deletes the cluster.
+
+Depending on your environment you can build Cilium by using the following
+makefile targets:
+
+For Linux and Mac OS
+^^^^^^^^^^^^^^^^^^^^
+
+Makefile targets automate building Cilium images:
+
 * ``make kind-image``: Builds all Cilium images and loads them into the
   cluster.
-* ``make kind-image-agent``: Builds the Cilium Agent image only and loads it
+* ``make kind-image-agent``: Builds only the Cilium Agent image and loads it
   into the cluster.
-* ``make kind-image-operator``: Builds the Cilium Operator (generic) image only
+* ``make kind-image-operator``: Builds only the Cilium Operator (generic) image
   and loads it into the cluster.
 * ``make kind-debug``: Builds all Cilium images with optimizations disabled and
   ``dlv`` embedded for live debugging enabled and loads the images into the
@@ -108,10 +134,25 @@ Cilium images:
   Use if only the agent image needs to be rebuilt for faster iteration.
 * ``make kind-install-cilium``: Installs Cilium into the cluster using the
   Cilium CLI.
-* ``make kind-down``: Tears down and deletes the cluster.
 
-The preceding list includes the most used commands for convenience. For more
-targets, see the ``Makefile``.
+The preceding list includes the most used commands for **convenience**. For more
+targets, see the ``Makefile`` (or simply run ``make help``).
+
+For Linux only - with shorter development workflow time
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+On Linux environments, or on environments where you can compile and run
+Cilium, it is possible to use "fast" targets. These fast targets will build
+Cilium in the local environment and mount that binary, as well the bpf source
+code, in an pre-existing running Cilium container.
+
+* ``make kind-install-cilium-fast``: Installs Cilium into the cluster using the
+  Cilium CLI with the volume mounts defined.
+
+* ``make kind-image-fast``: Builds all Cilium binaries and loads them into all
+  kind clusters available in the host.
+
+.. _configurations_for_clusters:
 
 Configuration for clusters
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -135,8 +176,8 @@ The setup for the Vagrantfile in the root of the Cilium tree depends on a
 number of environment variables and network setup that are managed via
 ``contrib/vagrant/start.sh``.
 
-Option 1 - Using the Provided Vagrantfiles (Recommended)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Option 1 - Using the Provided Vagrantfiles
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To bring up a Vagrant VM with Cilium plus dependencies installed, run:
 
@@ -187,7 +228,7 @@ The box is currently available for the following providers:
 * virtualbox
 
 Configuration Options
----------------------
+^^^^^^^^^^^^^^^^^^^^^
 
 The following environment variables can be set to customize the VMs
 brought up by vagrant:
@@ -266,7 +307,7 @@ If you have any issue with the provided vagrant box
 build the box yourself using the `packer scripts <https://github.com/cilium/packer-ci-build>`_
 
 Launch CI VMs
--------------
+^^^^^^^^^^^^^
 
 The ``test`` directory also contains a ``Vagrantfile`` that can be
 used to bring up the CI VM images that will cache a Vagrant box
@@ -491,7 +532,7 @@ finishes.
 .. _making_changes:
 
 Making Changes
-~~~~~~~~~~~~~~
+--------------
 
 #. Make sure the ``main`` branch of your fork is up-to-date:
 
@@ -541,11 +582,25 @@ Making Changes
    This make target works both inside and outside the Vagrant VM, assuming that ``docker``
    is running in the environment.
 
+Dev Container
+-------------
+
+Cilium provides `Dev Container <https://code.visualstudio.com/docs/devcontainers/containers>`_ configuration for Visual Studio Code Remote Containers
+and `Github Codespaces <https://docs.github.com/en/codespaces/setting-up-your-project-for-codespaces/introduction-to-dev-containers>`_.
+This allows you to use a preconfigured development environment in the cloud or locally.
+The container is based on the official Cilium builder image and provides all the dependencies
+required to build Cilium.
+
+.. note::
+
+    The current Dev Container is running as root. Non-root user support requires non-root
+    user in Cilium builder image, which is related to :gh-issue:`23217`.
+
 Update a golang version
-~~~~~~~~~~~~~~~~~~~~~~~
+-----------------------
 
 Minor version
-^^^^^^^^^^^^^
+~~~~~~~~~~~~~
 
 Each Cilium release is tied to a specific version of Golang via an explicit constraint
 in our Renovate configuration.
@@ -585,13 +640,13 @@ Once the CI is passing, the PR will be merged as part of the standard version
 upgrade process.
 
 Patch version
-^^^^^^^^^^^^^
+~~~~~~~~~~~~~
 
 New patch versions of Golang are picked up automatically by the CI; there should
 normally be no need to update the version manually.
 
 Add/update a golang dependency
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+------------------------------
 
 Let's assume we want to add ``github.com/containernetworking/cni`` version ``v0.5.2``:
 
@@ -620,7 +675,7 @@ change:
     $ git add go.mod go.sum vendor/
 
 Add/update a new Kubernetes version
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+-----------------------------------
 
 Let's assume we want to add a new Kubernetes version ``v1.19.0``:
 
@@ -630,7 +685,7 @@ Let's assume we want to add a new Kubernetes version ``v1.19.0``:
    update.
 
 Minor version
-^^^^^^^^^^^^^
+~~~~~~~~~~~~~
 
 #. Check if it is possible to remove the last supported Kubernetes version from
    :ref:`k8scompatibility`, :ref:`k8s_requirements`, :ref:`test_matrix`,
@@ -715,7 +770,7 @@ Minor version
 .. _Cilium CI matrix: https://docs.google.com/spreadsheets/d/1TThkqvVZxaqLR-Ela4ZrcJ0lrTJByCqrbdCjnI32_X0
 
 Patch version
-^^^^^^^^^^^^^
+~~~~~~~~~~~~~
 
 #. Bump the Kubernetes version in ``contrib/vagrant/scripts/helpers.bash``.
 
@@ -724,7 +779,7 @@ Patch version
 #. Submit all your changes into a new PR.
 
 Making changes to the Helm chart
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+--------------------------------
 
 The Helm chart is located in the ``install/kubernetes`` directory. The
 ``values.yaml.tmpl`` file contains the values for the Helm chart which are being used into the ``values.yaml`` file.
@@ -759,7 +814,7 @@ At last you might want to check the chart using the ``lint`` target:
 
 
 Optional: Docker and IPv6
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+-------------------------
 
 Note that these instructions are useful to you if you care about having IPv6
 addresses for your Docker containers.
@@ -808,10 +863,11 @@ If you'd like IPv6 addresses, you will need to follow these steps:
 Now new containers will have an IPv6 address assigned to them.
 
 Debugging
-~~~~~~~~~
+---------
 
 Datapath code
-^^^^^^^^^^^^^
+~~~~~~~~~~~~~
+
 The tool ``cilium monitor`` can also be used to retrieve debugging information
 from the eBPF based datapath. To enable all log messages:
 

Makefile

@@ -563,6 +563,61 @@ kind-image: ## Build cilium and operator images and import them into kind.
 	$(MAKE) kind-image-agent
 	$(MAKE) kind-image-operator
 
+.PHONY: kind-install-cilium-fast
+kind-install-cilium-fast: kind-ready ## Install a local Cilium version into the cluster.
+	@echo "  INSTALL cilium"
+	# cilium-cli doesn't support idempotent installs, so we uninstall and
+	# reinstall here. https://github.com/cilium/cilium-cli/issues/205
+	-$(CILIUM_CLI) uninstall >/dev/null
+	# cilium-cli's --wait flag doesn't work, so we just force it to run
+	# in the background instead and wait for the resources to be available.
+	# https://github.com/cilium/cilium-cli/issues/1070
+	$(CILIUM_CLI) install \
+		--chart-directory=$(ROOT_DIR)/install/kubernetes/cilium \
+		--helm-values=$(ROOT_DIR)/contrib/testing/kind-common.yaml \
+		--helm-values=$(ROOT_DIR)/contrib/testing/kind-fast.yaml \
+		--version= \
+		>/dev/null 2>&1 &
+
+.PHONY: kind-image-fast
+kind-image-fast: ## Build cilium, operator, clustermesh-apiserver and mount them into a volume for all available Kind clusters.
+	$(eval dst:=/cilium-binaries)
+	$(MAKE) -C cilium
+	$(MAKE) -C daemon
+	$(MAKE) -C operator cilium-operator-generic
+	$(MAKE) -C clustermesh-apiserver
+
+	for cluster_name in $${KIND_CLUSTERS:-$(shell kind get clusters)}; do \
+	    for node_name in $(shell kind get nodes -n "${cluster_name}"); do \
+			docker exec -ti $${node_name} mkdir -p "${dst}"; \
+			\
+			docker exec -ti $${node_name} rm -rf "${dst}/var/lib/cilium"; \
+			docker exec -ti $${node_name} mkdir -p "${dst}/var/lib/cilium"; \
+			docker cp "./bpf/" $${node_name}:"${dst}/var/lib/cilium/bpf"; \
+			docker exec -ti $${node_name} find "${dst}/var/lib/cilium/bpf" -type f -exec chmod 0644 {} + ;\
+			\
+			docker exec -ti $${node_name} rm -f "${dst}/cilium"; \
+			docker cp "./cilium/cilium" $${node_name}:"${dst}"; \
+			docker exec -ti $${node_name} chmod +x "${dst}/cilium"; \
+			\
+			docker exec -ti $${node_name} rm -f "${dst}/cilium-agent"; \
+			docker cp "./daemon/cilium-agent" $${node_name}:"${dst}"; \
+			docker exec -ti $${node_name} chmod +x "${dst}/cilium-agent"; \
+			\
+			docker exec -ti $${node_name} rm -f "${dst}/cilium-operator-generic"; \
+			docker cp "./operator/cilium-operator-generic" $${node_name}:"${dst}"; \
+			docker exec -ti $${node_name} chmod +x "${dst}/cilium-operator-generic"; \
+			\
+			docker exec -ti $${node_name} rm -f "${dst}/clustermesh-apiserver"; \
+			docker cp "./clustermesh-apiserver/clustermesh-apiserver" $${node_name}:"${dst}"; \
+			docker exec -ti $${node_name} chmod +x "${dst}/clustermesh-apiserver"; \
+			\
+			kubectl --context=kind-$${cluster_name} delete pods -n kube-system -l k8s-app=cilium --force; \
+			kubectl --context=kind-$${cluster_name} delete pods -n kube-system -l name=cilium-operator --force; \
+			kubectl --context=kind-$${cluster_name} delete pods -n kube-system -l k8s-app=clustermesh-apiserver --force; \
+		done; \
+	done
+
 .PHONY: kind-install-cilium
 kind-install-cilium: kind-ready ## Install a local Cilium version into the cluster.
 	@echo "  INSTALL cilium"
@@ -574,6 +629,7 @@ kind-install-cilium: kind-ready ## Install a local Cilium version into the clust
 	# https://github.com/cilium/cilium-cli/issues/1070
 	$(CILIUM_CLI) install \
 		--chart-directory=$(ROOT_DIR)/install/kubernetes/cilium \
+		--helm-values=$(ROOT_DIR)/contrib/testing/kind-common.yaml \
 		--helm-values=$(ROOT_DIR)/contrib/testing/kind-values.yaml \
 		--version= \
 		>/dev/null 2>&1 &

contrib/testing/kind-common.yaml

@@ -0,0 +1,20 @@
+debug:
+  enabled: true
+operator:
+  nodeSelector:
+    kubernetes.io/os: linux
+    kubernetes.io/hostname: kind-worker
+ipam:
+  mode: kubernetes
+ipv6:
+  enabled: true
+ipv4:
+  enabled: true
+bpf:
+  monitorAggregation: none
+livenessProbe:
+  failureThreshold: 9999
+readinessProbe:
+  failureThreshold: 9999
+startupProbe:
+  failureThreshold: 9999