Host agent throw "imgpkg: Error: Cannot use --recursive (-r) flag when pulling a bundle"

[phamquy]

What steps did you take and what happened:
I follow the instruction from this link https://github.com/vmware-tanzu/cluster-api-provider-bringyourownhost/blob/main/docs/getting_started.md with the following setup

• Create a management K8s cluster on 3 EC2 instances
• Deploy Boyh cluster api component on my management cluster
• Create 3 Ubuntu 20.04 instance as boyh hosts

cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

• Start agent in the 3 byoh

sudo ./byoh-hostagent-linux-amd64 --bootstrap-kubeconfig bootstrap-kubeconfig.conf

confirm that three hosts are picked up by cluster api

~/capi kubectl get byohosts                                                                                             05:19:13
NAME              OSNAME   OSIMAGE              ARCH
ip-172-31-4-123   linux    Ubuntu 20.04.6 LTS   amd64
ip-172-31-6-20    linux    Ubuntu 20.04.6 LTS   amd64
ip-172-31-8-92    linux    Ubuntu 20.04.6 LTS   amd64

• Generate cluster manifest with clusterctl

export CONTROL_PLANE_ENDPOINT_IP=172.31.2.232 # this is IP of a byoh host
clusterctl generate cluster byoh-cluster \
  --infrastructure byoh \
  --kubernetes-version v1.24.2 \
  --control-plane-machine-count 3 \
  --worker-machine-count

-- Apply the manifest

• Observe log in one of the host and saw

+ imgpkg pull -r -i projects.registry.vmware.com/cluster_api_provider_bringyourownhost/byoh-bundle-ubuntu_20.04.1_x86-64_k8s:v1.24.2 -o /var/lib/byoh/bundles/projects.registry.vmware.com/cluster_api_provider_bringyourownhost/byoh-bundle-ubuntu_20.04.1_x86-64_k8s:v1.24.2
imgpkg: Error: Cannot use --recursive (-r) flag when pulling a bundle

What did you expect to happen:
Cluster is created

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Cluster-api-provider-bringyourownhost version:

clusterctl version: &version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.3", GitCommit:"14b88ca091102b72c591cb0357199834385df478", GitTreeState:"clean", BuildDate:"2023-06-06T15:41:40Z", GoVersion:"go1.19.6", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes version: (use kubectl version --short):

Client Version: v1.24.0
Kustomize Version: v4.5.4
Server Version: v1.24.0

• OS (e.g. from /etc/os-release):

cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

[phamquy]

With help Dharmjit i found that there is imgpkg installed in my host with version 0.3x.* that does not support --recursive. after remove the imgpkg in my host, it got pass the error but stuck at

I0629 18:16:25.992579   45187 host_reconciler.go:89] controller/byohost "msg"="reconcile normal" "ByoHost"="ip-172-31-6-20" "name"="ip-172-31-6-20" "namespace"="default" "reconciler group"="infrastructure.cluster.x-k8s.io" "reconciler kind"="ByoHost"

although it got pass the error, i think we should still fix the installation script.

[haiwu]

I hit the same issue, while trying to get byoh to use latest imgpkg on purpose.

[dharmjit]

although it got pass the error, i think we should still fix the installation script.

@phamquy, This is a good find, Script[here] needs to be updated to check for the imgpkg version as well in the Host to limit any issues due to version mismatch. Let me know if you are interested in raising a PR for this.

[phamquy]

i'm interested.