Deploying simplenificluster does nothing

[juldrixx]

From nifikop created by Docteur-RS: Orange-OpenSource/nifikop#166

Type of question

Requesting help on strange behavior

Question

Hey,

Followed the installation tutorial from the documentation but after the last step nothing happend.

What I did :

# Installing zookeeper (in default namespace...)

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install zookeeper bitnami/zookeeper \
    --set resources.requests.memory=256Mi \
    --set resources.requests.cpu=250m \
    --set resources.limits.memory=256Mi \
    --set resources.limits.cpu=250m \
    --set global.storageClass=standard \
    --set networkPolicy.enabled=true \
    --set replicaCount=3

# Installing certmanager

kubectl apply -f \
    https://github.com/jetstack/cert-manager/releases/download/v1.2.0/cert-manager.yaml

# Installing nifikops

helm repo add orange-incubator https://orange-kubernetes-charts-incubator.storage.googleapis.com/

kubectl create ns nifi

helm install nifikop \
    orange-incubator/nifikop \
    --namespace=nifi \
    --version 0.7.5 \
    --set image.tag=v0.7.5-release \
    --set resources.requests.memory=256Mi \
    --set resources.requests.cpu=250m \
    --set resources.limits.memory=256Mi \
    --set resources.limits.cpu=250m \
    --set namespaces={"nifi"}

Next, I didn't setup the customstorage class because it was optional.

# Applying the simple nifi cluster

## Updated the storage StorageClassName to the one availble inside my cluster

kubectl create -n nifi -f config/samples/simplenificluster.yaml

And now..... Nothing.

Pods and logs

kubectl get po -n default

NAME          READY   STATUS    RESTARTS   AGE
zookeeper-0   1/1     Running   0          40m
zookeeper-1   1/1     Running   0          40m
zookeeper-2   1/1     Running   0          40m

kubectl get po -n nifi

NAME                       READY   STATUS    RESTARTS   AGE
nifikop-5d4c9b6d6d-mz922   1/1     Running   0          33m

 kubectl get nificluster
NAME         AGE
simplenifi   32m

#Nifikop logs

2021-11-29T17:27:07.597Z        INFO    setup   manager set up with multiple namespaces {"namespaces": "nifi"}
2021-11-29T17:27:07.597Z        INFO    setup   Writing ready file.
I1129 17:27:08.718212       1 request.go:655] Throttling request took 1.013748334s, request: GET:https://10.3.0.1:443/apis/apm.k8s.elastic.co/v1?timeout=32s
2021-11-29T17:27:09.984Z        INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": ":8080"}
2021-11-29T17:27:09.991Z        INFO    setup   starting manager
2021-11-29T17:27:09.992Z        INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
2021-11-29T17:27:09.993Z        INFO    controller-runtime.manager.controller.nificluster       Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster", "source": "kind source: /, Kind="}
2021-11-29T17:27:09.993Z        INFO    controller-runtime.manager.controller.nifiuser  Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiUser", "source": "kind source: /, Kind="}
2021-11-29T17:27:09.994Z        INFO    controller-runtime.manager.controller.nifiusergroup     Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiUserGroup", "source": "kind source: /, Kind="}
2021-11-29T17:27:09.994Z        INFO    controller-runtime.manager.controller.nifidataflow      Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiDataflow", "source": "kind source: /, Kind="}
2021-11-29T17:27:09.994Z        INFO    controller-runtime.manager.controller.nifiparametercontext      Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiParameterContext", "source": "kind source: /, Kind="}
2021-11-29T17:27:09.994Z        INFO    controller-runtime.manager.controller.nifiregistryclient        Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiRegistryClient", "source": "kind source: /, Kind="}
2021-11-29T17:27:09.995Z        INFO    controller-runtime.manager.controller.nificluster       Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster", "source": "kind source: /, Kind="}
2021-11-29T17:27:10.180Z        INFO    controller-runtime.manager.controller.nifiusergroup     Starting Controller     {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiUserGroup"}
2021-11-29T17:27:10.180Z        INFO    controller-runtime.manager.controller.nifiuser  Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiUser", "source": "kind source: /, Kind="}
2021-11-29T17:27:10.181Z        INFO    controller-runtime.manager.controller.nifiregistryclient        Starting Controller     {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiRegistryClient"}
2021-11-29T17:27:10.181Z        INFO    controller-runtime.manager.controller.nifiparametercontext      Starting Controller     {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiParameterContext"}
2021-11-29T17:27:10.181Z        INFO    controller-runtime.manager.controller.nifidataflow      Starting Controller     {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiDataflow"}
2021-11-29T17:27:10.284Z        INFO    controller-runtime.manager.controller.nifidataflow      Starting workers        {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiDataflow", "worker count": 1}
2021-11-29T17:27:10.294Z        INFO    controller-runtime.manager.controller.nificluster       Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster", "source": "kind source: /, Kind="}
2021-11-29T17:27:10.381Z        INFO    controller-runtime.manager.controller.nificluster       Starting Controller     {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster"}
2021-11-29T17:27:10.381Z        INFO    controller-runtime.manager.controller.nifiusergroup     Starting workers        {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiUserGroup", "worker count": 1}
2021-11-29T17:27:10.381Z        INFO    controller-runtime.manager.controller.nifiuser  Starting Controller     {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiUser"}
2021-11-29T17:27:10.381Z        INFO    controller-runtime.manager.controller.nifiregistryclient        Starting workers        {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiRegistryClient", "worker count": 1}
2021-11-29T17:27:10.381Z        INFO    controller-runtime.manager.controller.nifiparametercontext      Starting workers        {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiParameterContext", "worker count": 1}
W1129 17:27:10.385667       1 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W1129 17:27:10.389258       1 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
2021-11-29T17:27:10.394Z        INFO    controller-runtime.manager.controller.nificluster       Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster", "source": "kind source: /, Kind="}
2021-11-29T17:27:10.481Z        INFO    controller-runtime.manager.controller.nificluster       Starting workers        {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster", "worker count": 1}
2021-11-29T17:27:10.495Z        INFO    controller-runtime.manager.controller.nificluster       Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster", "source": "kind source: /, Kind="}
2021-11-29T17:27:10.582Z        INFO    controller-runtime.manager.controller.nifiuser  Starting workers        {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiUser", "worker count": 1}
2021-11-29T17:27:10.596Z        INFO    controller-runtime.manager.controller.nificluster       Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster", "source": "kind source: /, Kind="}
2021-11-29T17:27:10.697Z        INFO    controller-runtime.manager.controller.nificluster       Starting EventSource    {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster", "source": "kind source: /, Kind="}
2021-11-29T17:27:10.798Z        INFO    controller-runtime.manager.controller.nificluster       Starting Controller     {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster"}
2021-11-29T17:27:10.798Z        INFO    controller-runtime.manager.controller.nificluster       Starting workers        {"reconciler group": "nifi.orange.com", "reconciler kind": "NifiCluster", "worker count": 1}
W1129 17:34:30.395634       1 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W1129 17:42:32.401953       1 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W1129 17:50:11.406944       1 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W1129 17:56:23.413301       1 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget

Environment

• nifikop version:

Helm chart version : 0.7.5

• Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:10:45Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:04:16Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes cluster kind:
  Managed by OVH

Please advise...

Thx !

[juldrixx]

same problem on start pods have this error: The processing instruction target matching "[xX][mM][lL]" is not allowed

[juldrixx]

the problem is on bootstrap-notification-services.xml last release introduce the correctly filename and cause this "problem"

[juldrixx]

@erdrix have method for override file: bootstrap-notification-services.xml ?

[juldrixx]

Same Problem here:
│ nifi Bootstrap Config File: /opt/nifi/nifi-current/conf/bootstrap.conf
│ nifi [Fatal Error] :1:7: The processing instruction target matching "[xX][mM][lL]" is not allowed.

[w0oh0o]

I'm also getting the same fatal. I have tried every release of the new repo and got the same error.

edit: I'm using konpyutaika repo and not the old orange one.

[grant-guo]

i got the same error, however i was told it is not a problem and i can ignore it through nifikop slack channel. i checked the nifi node logs and found it is complaining no coordinator. looks like node can't access to zookeeper although i can ping zk from the node. still debugging...

[wintersommer]

@grant-guo same here, cluster starts , port 8080 open (cat /proc/net/tcp) , but nifi does not answer on that port. On my side zookeeper is reached, i checked with PrettyZoo.

[grant-guo]

Update: i got everything working. Just installed a new zookeeper cluster(using bitnami's helm chart) in k8s instead of relying on the existing external zookeeper cluster.

One tricky thing is I gave 1G memory to zk instead of the default 256Mi because I got OOM

[grant-guo]

@grant-guo same here, cluster starts , port 8080 open (cat /proc/net/tcp) , but nifi does not answer on that port. On my side zookeeper is reached, i checked with PrettyZoo.

See my new update