clustermesh: Adapt clustermesh script to install cilium-dbg

[joestringer]

Use the 'cilium' binary from a Cilium container to copy into
/usr/bin/cilium and /usr/bin/cilium-dbg so that this script will be
compatible with newer CI workflows in future that rely on running
cilium-dbg commands rather than cilium commands directly within the Pod.

The 'cp -L' symbolic link dereferencing will make this compatible with
upcoming v1.15-dev based images after cilium/cilium#28085 is merged.

This PR must be merged first, otherwise older versions of this script will
shallow-copy the symlink for 'cilium' into the host and then the 'cilium'
commands will not work. This also implies that when users upgrade to Cilium
v1.15, they will need to update this CLI tool if using external workloads
support, otherwise the script will fail.

Other than that, the code with this patch should be compatible with all
older releases of the cilium CLI.

Related: cilium/cilium#28085

[joestringer]

(We need to confirm the exact naming of the binary before merging this)

[giorio94]

Overall looks good to me. Just one comment inline to prevent incompatibilities with older installations, and one possible nit to reduce disk usage.

[joestringer]

Applying this diff on top:

$ git dc
diff --git a/clustermesh/clustermesh.go b/clustermesh/clustermesh.go
index eb49a0e3544f..55aba9934455 100644
--- a/clustermesh/clustermesh.go
+++ b/clustermesh/clustermesh.go
@@ -1585,9 +1585,13 @@ if [ "$1" = "uninstall" ] ; then
         echo "Shutting down running Cilium agent"
         ${SUDO} docker rm -f cilium || true
     fi
-    if [ -f /usr/bin/cilium ] ; then
+    if [ -e /usr/bin/cilium ]; then
         echo "Removing /usr/bin/cilium"
-        ${SUDO} rm /usr/bin/cilium /usr/bin/cilium-dbg
+        ${SUDO} rm /usr/bin/cilium
+    fi
+    if [ -e /usr/bin/cilium-dbg ] ; then
+        echo "Removing /usr/bin/cilium-dbg"
+        ${SUDO} rm /usr/bin/cilium-dbg
     fi
     pushd /etc
     if [ -f resolv.conf.orig ] ; then
@@ -1682,15 +1686,16 @@ retries=%[7]s
 while [ $cilium_started = false ]; do
     if [ -n "$(${SUDO} docker ps -a -q -f name=cilium)" ]; then
         echo "Shutting down running Cilium agent"
-        ${SUDO} docker rm -f cilium cilium-dbg || true
+        ${SUDO} docker rm -f cilium || true
     fi
 
     echo "Launching Cilium agent $CILIUM_IMAGE..."
     ${SUDO} docker run --name cilium $DOCKER_OPTS $CILIUM_IMAGE cilium-agent $CILIUM_OPTS
 
     # Copy Cilium CLI
-    ${SUDO} docker cp -L cilium:/usr/bin/cilium /usr/bin/cilium
     ${SUDO} docker cp -L cilium:/usr/bin/cilium /usr/bin/cilium-dbg
+    ln -s cilium-dbg cilium-symlink
+    ${SUDO} docker cp cilium-symlink /usr/bin/cilium
 
     # Wait for cilium agent to become available
     for ((i = 0 ; i < 12; i++)); do

EDIT: And this on top to fix the symlink from my first iteration:

diff --git a/clustermesh/clustermesh.go b/clustermesh/clustermesh.go
index 55aba9934455..5a575a2aec8b 100644
--- a/clustermesh/clustermesh.go
+++ b/clustermesh/clustermesh.go
@@ -1694,8 +1694,7 @@ while [ $cilium_started = false ]; do
 
     # Copy Cilium CLI
     ${SUDO} docker cp -L cilium:/usr/bin/cilium /usr/bin/cilium-dbg
-    ln -s cilium-dbg cilium-symlink                                                                                                         
-    ${SUDO} docker cp cilium-symlink /usr/bin/cilium                                                                                        
+    ${SUDO} ln -s /usr/bin/cilium-dbg /usr/bin/cilium                                                                                       
 
     # Wait for cilium agent to become available
     for ((i = 0 ; i < 12; i++)); do

[giorio94]

/lgtm, thanks!

[joestringer]

I was hitting cilium/cilium#28402, but the underlying issue should be resolved now. Resubmitting through CI.