Fetching dynamic kubeconfig

[akgalwas]

Description

Changes proposed in this pull request:

• KubeconfigProvider implemented
• KubeconfigProvider initialized on application startup

Related issue(s)

[kyma-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[Disper]

Should we already capture somewhere (maybe in some issue or documentation) that this should be overridden for other environments?

[Disper]

I was unable to test that successfully.
After applying the following sample CR:

apiVersion: infrastructuremanager.kyma-project.io/v1
kind: GardenerCluster
metadata:
  labels:
    kyma-project.io/instance-id: instance-id
    kyma-project.io/runtime-id: md-im3
    kyma-project.io/broker-plan-id: plan-id
    kyma-project.io/broker-plan-name: plan-name
    kyma-project.io/global-account-id: global-account-id
    kyma-project.io/subaccount-id: subAccount-id
    kyma-project.io/shoot-name: md-im3
    kyma-project.io/region: region
    operator.kyma-project.io/kyma-name: kymaName
  name: md-im3
  namespace: kcp-system
spec:
  shoot:
    name: md-im3
  kubeconfig:
    secret:
      name: kubeconfig-md-im3
      namespace: kcp-system
      key: "config"

I'm getting

2023-09-29T06:30:44Z    ERROR   Reconciler error        {"controller": "gardenercluster", "controllerGroup": "infrastructuremanager.kyma-project.io", "controllerKind": "GardenerCluster", "GardenerCluster": {"name":"md-im3","namespace":"kcp-system"}, "namespace": "kcp-system", "name": "md-im3", "reconcileID": "027f9eac-5c41-426b-944d-a41b1bc5b274", "error": "failed to get shoot: Get \"https://api.canary.gardener.cloud.sap/apis/core.gardener.cloud/v1beta1/namespaces/garden-frog-dev/shoots/md-im3\": getting credentials: exec: executable kubectl not found", "errorVerbose": "Get \"https://api.canary.gardener.cloud.sap/apis/core.gardener.cloud/v1beta1/namespaces/garden-frog-dev/shoots/md-im3\": getting credentials: exec: executable kubectl not found\nfailed to get shoot\ngithub.com/kyma-project/infrastructure-manager/internal/gardener.KubeconfigProvider.Fetch\n\t/workspace/internal/gardener/KubeConfigProvider.go:44\ngithub.com/kyma-project/infrastructure-manager/internal/controller.(*GardenerClusterController).newSecret\n\t/workspace/internal/controller/gardener_cluster_controller.go:183\ngithub.com/kyma-project/infrastructure-manager/internal/controller.(*GardenerClusterController).createSecret\n\t/workspace/internal/controller/gardener_cluster_controller.go:166\ngithub.com/kyma-project/infrastructure-manager/internal/controller.(*GardenerClusterController).Reconcile\n\t/workspace/internal/controller/gardener_cluster_controller.go:107\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:118\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:314\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:265\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:226\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1598"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.2/pkg/internal/controller/controller.go:226

Tested on

• image: europe-docker.pkg.dev/kyma-project/dev/infrastructure-manager:PR-47
• k8s clusters 1.27 & 1.26

By

• Downloading either static or dynamic kubeconfig
• running make deploy, make gardener-secret-deploy, kubectl apply -f config/samples/infrastructuremanager_v1_gardenercluster.yaml
• Editing deployment to use mentioned previously image

[Disper]

I've had a screen sharing session with @akgalwas where I've seen that this feature works. I still don't know what is causing the issue I've mentioned in the previous comment, possibly some inconsistencies with the manifest generated via kustomize or some tooling that I use like kubectl.

[Disper]

I'm giving an approval (see the previous comment) however I've didn't managed to have a closer look too see if there would be a good way to cover this with automated tests. @akgalwas Could you maybe consult other frog if there would be some nice way to introduce some automated test, or to make sure that we have a follow-up for that?

I've set the PR to [WiP] so it won't be merged automatically before you look at this comment but I'm also leaving an approval.