Global operator mode on Minikube not working

[christophd]

Integrations get stuck in phase Waiting For Platform

Camel-K version: 1.2.0-SNAPSHOT
Steps to reproduce:

• Create namespace kube-operators
• Install Camel-K in global mode kamel install --global -n kube-operators
• Switch to another namespace e.g. camel-k-demo
• Run Camel-K sample integration kamel run helloworld.groovy

Integration is created and has following events:

Condition "IntegrationPlatformAvailable" is "False" for Integration helloworld: IntegrationPlatform.camel.apache.org "camel-k" not found

Integration helloworld in phase "Waiting For Platform"
camel-k-integration-controller

Condition "IntegrationPlatformAvailable" is "False" for Integration helloworld: IntegrationPlatform.camel.apache.org "camel-k" not found
camel-k-integration-controller

Integration helloworld in phase "Waiting For Platform"
camel-k-integration-controller

Operator logs:

{"level":"info","ts":1602761845.9070475,"logger":"cmd","msg":"Go Version: go1.15.2"}
{"level":"info","ts":1602761845.9071462,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":1602761845.90717,"logger":"cmd","msg":"Buildah Version: 1.14.0"}
{"level":"info","ts":1602761845.907233,"logger":"cmd","msg":"Kaniko Version: 0.17.1"}
{"level":"info","ts":1602761845.9072526,"logger":"cmd","msg":"Camel K Operator Version: 1.2.0-SNAPSHOT"}
{"level":"info","ts":1602761845.9073274,"logger":"cmd","msg":"Camel K Default Runtime Version: 1.5.0"}
{"level":"info","ts":1602761845.9073784,"logger":"cmd","msg":"Camel K Git Commit: b3ae43691dea0afa6eb05a9a46c9d486f13656e4"}
{"level":"info","ts":1602761845.907538,"logger":"leader","msg":"Trying to become the leader."}
I1015 11:37:26.958026       1 request.go:621] Throttling request took 1.03595117s, request: GET:https://10.96.0.1:443/apis/networking.internal.knative.dev/v1alpha1?timeout=32s
{"level":"info","ts":1602761847.3164706,"logger":"leader","msg":"No pre-existing lock was found."}
{"level":"info","ts":1602761847.3210428,"logger":"leader","msg":"Became the leader."}
{"level":"info","ts":1602761850.1321387,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":1602761850.1324246,"logger":"cmd","msg":"Registering Components."}
{"level":"info","ts":1602761850.1349628,"logger":"cmd","msg":"Cannot detect user cluster role: skipping."}
{"level":"info","ts":1602761850.1404867,"logger":"cmd","msg":"Starting the Cmd."}
{"level":"info","ts":1602761850.1411188,"logger":"controller","msg":"Starting EventSource","controller":"integrationplatform-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.1410644,"logger":"controller","msg":"Starting EventSource","controller":"build-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.1416163,"logger":"controller","msg":"Starting EventSource","controller":"kamelet-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.141695,"logger":"controller","msg":"Starting EventSource","controller":"kamelet-binding-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.1417336,"logger":"controller","msg":"Starting EventSource","controller":"integration-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.1418538,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
{"level":"info","ts":1602761850.141239,"logger":"controller","msg":"Starting EventSource","controller":"integrationkit-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.242047,"logger":"controller","msg":"Starting EventSource","controller":"build-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.242017,"logger":"controller","msg":"Starting Controller","controller":"integrationplatform-controller"}
{"level":"info","ts":1602761850.2428153,"logger":"controller","msg":"Starting EventSource","controller":"integration-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.3427386,"logger":"controller","msg":"Starting Controller","controller":"kamelet-controller"}
{"level":"info","ts":1602761850.3436842,"logger":"controller","msg":"Starting workers","controller":"kamelet-controller","worker count":1}
{"level":"info","ts":1602761850.343308,"logger":"controller","msg":"Starting EventSource","controller":"integration-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.3442454,"logger":"controller","msg":"Starting EventSource","controller":"integration-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.3434167,"logger":"controller","msg":"Starting EventSource","controller":"integrationkit-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.3452816,"logger":"controller","msg":"Starting EventSource","controller":"integrationkit-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.3435223,"logger":"controller","msg":"Starting Controller","controller":"kamelet-binding-controller"}
{"level":"info","ts":1602761850.3433867,"logger":"controller","msg":"Starting Controller","controller":"build-controller"}
{"level":"info","ts":1602761850.3429363,"logger":"controller","msg":"Starting workers","controller":"integrationplatform-controller","worker count":1}
{"level":"info","ts":1602761850.3457706,"logger":"controller","msg":"Starting Controller","controller":"integrationkit-controller"}
{"level":"info","ts":1602761850.44582,"logger":"controller","msg":"Starting workers","controller":"build-controller","worker count":1}
{"level":"info","ts":1602761850.4461293,"logger":"controller","msg":"Starting workers","controller":"kamelet-binding-controller","worker count":1}
{"level":"info","ts":1602761850.4462824,"logger":"controller","msg":"Starting workers","controller":"integrationkit-controller","worker count":1}
{"level":"info","ts":1602761850.4465146,"logger":"controller","msg":"Starting EventSource","controller":"integration-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1602761850.5477488,"logger":"controller","msg":"Starting Controller","controller":"integration-controller"}
{"level":"info","ts":1602761850.547923,"logger":"controller","msg":"Starting workers","controller":"integration-controller","worker count":1}
{"level":"info","ts":1602761926.19488,"logger":"camel-k.controller.integration","msg":"Reconciling Integration","request-namespace":"yaks-demo","request-name":"helloworld"}
{"level":"info","ts":1602761926.1949658,"logger":"camel-k.controller.integration","msg":"Invoking action platform-setup","request-namespace":"yaks-demo","request-name":"helloworld","api-version":"camel.apache.org/v1","kind":"Integration","ns":"yaks-demo","name":"helloworld"}
{"level":"info","ts":1602761926.1965635,"logger":"camel-k.trait","msg":"Apply trait: platform"}
{"level":"info","ts":1602761926.196697,"logger":"camel-k.trait","msg":"Apply trait: deployer"}
{"level":"info","ts":1602761926.2029784,"logger":"camel-k.controller.integration","msg":"state transition","request-namespace":"yaks-demo","request-name":"helloworld","api-version":"camel.apache.org/v1","kind":"Integration","ns":"yaks-demo","name":"helloworld","phase-from":"","phase-to":"Waiting For Platform"}
{"level":"info","ts":1602761926.2031074,"logger":"camel-k.controller.integration","msg":"Reconciling Integration","request-namespace":"yaks-demo","request-name":"helloworld"}
{"level":"info","ts":1602761926.2039945,"logger":"camel-k.controller.integration","msg":"Invoking action platform-setup","request-namespace":"yaks-demo","request-name":"helloworld","api-version":"camel.apache.org/v1","kind":"Integration","ns":"yaks-demo","name":"helloworld"}
{"level":"info","ts":1602761926.2058463,"logger":"camel-k.trait","msg":"Apply trait: platform"}
{"level":"info","ts":1602761926.2059417,"logger":"camel-k.trait","msg":"Apply trait: deployer"}

[astefanutti]

@christophd My understanding is that it is the expected behaviour. When installing the operator globally, no default IntegrationPlatform is created, as there are expected to be created manually in the namespaces in which Camel K integrations should run.

[nicolaferraro]

I'm sure I did some changes before Christmas holiday (1.3) but I'm not sure if I merged them...

Or probably I hit too many issues trying to automate all things. Specifically on Minikube the kamel CLI is able to detect the presence of the registry and its address, while the operator is not. So when installing globally, the CLI should create a kinda "template integration platform" in the global namespace, that then should be used to generate other namespaced integrationplatforms in the specific namespaces.

An alternative (and better) solution would be to have a single platform in the operator namespace... But now there are too many ways the operator can panic because presence of an integration platform is always assumed true...

So I was thinking to instruct kamel run to also install a platform...

I'm still not sure which solution is best...

[astefanutti]

Right, it's worth thinking about it more, also in the context of #1799.

To keep complexity under control, maybe it's better to design a generic solution, not specific to Minikube.

I was just trying to clean the issue tracking a bit, not opening the pandora box :)

[nicolaferraro]

Right, it's worth thinking about it more, also in the context of #1799.

To keep complexity under control, maybe it's better to design a generic solution, not specific to Minikube.

I was just trying to clean the issue tracking a bit, not opening the pandora box :)

Yeah, in the general case, I would like to setup things like registry and credentials once upon installation, then just use Camel K in any ns. But I guess this mechanism needs a big code cleanup in order to work.

[astefanutti]

Maybe we can move the creation of the IntegrationPlatform operator-side, instead of CLI-side, so that the operator can decide what to do when there isn't any found.

For registry auto-configuration, we can grant the operator whatever permission is required, for example to handle auto-configuration with Minikube registry addon, that would be :

ca_role.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: camel-k-minikube
rules:
- apiGroups:
  - ""
  resources:
  - services
  resourceNames:
  - registry
  verbs:
  - get

ca_role_binding.yaml:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: camel-k-minikube
  namespace: kube-system
subjects:
- kind: ServiceAccount
  name: camel-k-operator
  namespace: camel-k
roleRef:
  kind: ClusterRole
  name: camel-k-minikube
  apiGroup: rbac.authorization.k8s.io

The only issue I see is with OLM packaging, as it's not possible to describe RoleBinding of ClusterRole in the CSV, and it does not seem like these are supported types that can be added as optional resources in the OLM bundle manifests directory. Anyway, as it's read-only permissions, granting ClusterRole seems acceptable.

Also in the future context of #1802, the operator would be able to read the LocalRegistryHosting ConfigMap, and auto-configure the IntegrationPlatform accordingly.

[nicolaferraro]

Maybe we can move the creation of the IntegrationPlatform operator-side, instead of CLI-side, so that the operator can decide what to do when there isn't any found.

For registry auto-configuration, we can grant the operator whatever permission is required, from example to handle auto-configuration with Minikube registry addon, that would be :

ca_role.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: camel-k-minikube
rules:
- apiGroups:
  - ""
  resources:
  - services
  resourceNames:
  - registry
  verbs:
  - get

ca_role_binding.yaml:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: camel-k-minikube
  namespace: kube-system
subjects:
- kind: ServiceAccount
  name: camel-k-operator
  namespace: camel-k
roleRef:
  kind: ClusterRole
  name: camel-k-minikube
  apiGroup: rbac.authorization.k8s.io

The only issue I see is with OLM packaging, as it's not possible to describe RoleBinding of ClusterRole in the CSV, and it does not seem like these are supported types that can be added as optional resources in the OLM bundle manifests directory. Anyway, as it's read-only permissions, granting ClusterRole seems acceptable.

Also in the future context of #1802, the operator would be able to read the LocalRegistryHosting ConfigMap, and auto-configure the IntegrationPlatform accordingly.

Yeah, that would solve the cases when auto-creation is possible, so we can remove the special cases for minikube and also support e.g. Kind oob. Currently only on OpenShift we're able to auto-create platforms via the platform trait.

But I think there are many cases, e.g. Kubernetes on GKE, Azure, AWS.. where there's need for user input and the CLI is the main way to inject those into an integration platform at installation time.

[astefanutti]

Yeah, that would solve the cases when auto-creation is possible, so we can remove the special cases for minikube and also support e.g. Kind oob. Currently only on OpenShift we're able to auto-create platforms via the platform trait.

Exactly. That would move the logic into a single place, and solve that particular issue.

But I think there are many cases, e.g. Kubernetes on GKE, Azure, AWS.. where there's need for user input and the CLI is the main way to inject those into an integration platform at installation time.

I'm not sure how the auto-configuration case and the user-provided case intersect. I would rather see the later leveraging the former, or just orthogonal.

[github-actions]

This issue has been automatically marked as stale due to 90 days of inactivity.
It will be closed if no further activity occurs within 15 days.
If you think that’s incorrect or the issue should never stale, please simply write any comment.
Thanks for your contributions!