Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Operator crashes the when NATS unavailable in the cluster #50

Closed
BuddhiWathsala opened this issue Jul 15, 2019 · 1 comment · Fixed by #51 or #65
Closed

Operator crashes the when NATS unavailable in the cluster #50

BuddhiWathsala opened this issue Jul 15, 2019 · 1 comment · Fixed by #51 or #65

Comments

@BuddhiWathsala
Copy link
Contributor

Description:
In the distributed deployment of the Siddhi operator installing NATS is a prerequisite. Thus, Siddhi operator always tries to watch NatsCluster and NatsStreamingCluster resources. If those resources are not there then the operator throws an error. Due to this error throw, the Siddhi operator tends to crash unexpectedly. The error log shows as below.

no matches for kind "NatsCluster" in version "nats.io/v1alpha2" 
github.com/siddhi-io/siddhi-operator/vendor/github.com/go-logr/zapr.(*zapLogger).Error
	/Users/buddhi/Documents/MyWork/Go/src/github.com/siddhi-io/siddhi-operator/vendor/github.com/go-logr/zapr/zapr.go:128
github.com/siddhi-io/siddhi-operator/vendor/sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start
	/Users/buddhi/Documents/MyWork/Go/src/github.com/siddhi-io/siddhi-operator/vendor/sigs.k8s.io/controller-runtime/pkg/source/source.go:89
github.com/siddhi-io/siddhi-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Watch
	/Users/buddhi/Documents/MyWork/Go/src/github.com/siddhi-io/siddhi-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:122
github.com/siddhi-io/siddhi-operator/pkg/controller/siddhiprocess.add
	/Users/buddhi/Documents/MyWork/Go/src/github.com/siddhi-io/siddhi-operator/pkg/controller/siddhiprocess/siddhiprocess_controller.go:136
github.com/siddhi-io/siddhi-operator/pkg/controller/siddhiprocess.Add
	/Users/buddhi/Documents/MyWork/Go/src/github.com/siddhi-io/siddhi-operator/pkg/controller/siddhiprocess/siddhiprocess_controller.go:55
github.com/siddhi-io/siddhi-operator/pkg/controller.AddToManager
	/Users/buddhi/Documents/MyWork/Go/src/github.com/siddhi-io/siddhi-operator/pkg/controller/controller.go:31
main.main
	/Users/buddhi/Documents/MyWork/Go/src/github.com/siddhi-io/siddhi-operator/cmd/manager/main.go:131
runtime.main
	/usr/local/bin/go/src/runtime/pr

The error log is identical to NatsStreamingCluster too.

Suggested Labels:
fix

Affected Product Version:
0.2.0-m1

Steps to reproduce:
Install the operator without NATS

@BuddhiWathsala
Copy link
Contributor Author

When we deploy Siddhi operator without NATS which would be totally fine when we deploy stateless Siddhi apps, then it prints an error like below saying unable to find NATS.

"msg":"if kind is a CRD, it should be installed before calling Start","kind":"NatsCluster.nats.io","error":"no matches for kind \"NatsCluster\" in version \"nats.io/v1alpha2\""
"msg":"if kind is a CRD, it should be installed before calling Start","kind":"NatsStreamingCluster.streaming.nats.io","error":"no matches for kind \"NatsStreamingCluster\" in version \"streaming.nats.io/v1alpha1\""

After this error, if we deploy NATS and NATS streaming operators and then tries to deploy a stateful distributed Siddhi app, the operator gives the following error.

{"level":"error","ts":1564141174.2775638,"logger":"siddhi","msg":"no matches for kind \"NatsCluster\" in version \"nats.io/v1alpha2\"","Request.Namespace":"default","Request.Name":"power-consume-app","error":"no matches for kind \"NatsCluster\" in version \"nats.io/v1alpha2\"","stacktrace":"github.com/siddhi-io/siddhi-operator/vendor/github.com/go-logr/zapr.(*zapLogger).Error
	src/github.com/siddhi-io/siddhi-operator/vendor/github.com/go-logr/zapr/zapr.go:128
github.com/siddhi-io/siddhi-operator/pkg/controller/siddhiprocess.(*ReconcileSiddhiProcess).updateErrorStatus
	src/github.com/siddhi-io/siddhi-operator/pkg/controller/siddhiprocess/operator.go:195
github.com/siddhi-io/siddhi-operator/pkg/controller/siddhiprocess.(*ReconcileSiddhiProcess).Reconcile
	src/github.com/siddhi-io/siddhi-operator/pkg/controller/siddhiprocess/siddhiprocess_controller.go:186
github.com/siddhi-io/siddhi-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	src/github.com/siddhi-io/siddhi-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215
github.com/siddhi-io/siddhi-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
	src/github.com/siddhi-io/siddhi-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158
github.com/siddhi-io/siddhi-operator/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
	src/github.com/siddhi-io/siddhi-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133
github.com/siddhi-io/siddhi-operator/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil
	src/github.com/siddhi-io/siddhi-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134
github.com/siddhi-io/siddhi-operator/vendor/k8s.io/apimachinery/pkg/util/wait.Until
	src/github.com/siddhi-io/siddhi-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

At that time I have to do as follow for a successful distributed Siddhi app deployment.

  1. Remove Siddhi Operator
  2. Install NATS and NATS streaming operator
  3. Redeploy the Siddhi Operator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant