Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Botkube 0.8.0 - crash loop - invalid memory address or nil pointer deference #126

Closed
bilaloslo92 opened this issue Jul 11, 2019 · 14 comments · Fixed by #184
Closed

[BUG] Botkube 0.8.0 - crash loop - invalid memory address or nil pointer deference #126

bilaloslo92 opened this issue Jul 11, 2019 · 14 comments · Fixed by #184
Labels
Milestone

Comments

@bilaloslo92
Copy link

@bilaloslo92 bilaloslo92 commented Jul 11, 2019

Describe the bug
I have tried to deploy Botkube version 0.8.0 (which was recently released), I am using the deploy-all-in-one-tls.yaml to deploy it in my Kubernetes cluster.

To Reproduce
Steps to reproduce the behavior:

  1. I am running mattermost server version: 5.11.0
  2. Kubebot version 0.8.0
  3. Deploy the deploy-all-in-one-tls.yaml with TLS (self-signed certificates)
  4. See error:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x10a8e3c] 

goroutine 78 [running]:
github.com/infracloudio/botkube/pkg/bot.(*mmBot).Start.func1(0xc00010e150, 0xc0002af140) 
       /go/src/github.com/infracloudio/botkube/pkg/bot/mattermost.go:104 +0x5c 
created by github.com/infracloudio/botkube/pkg/bot.(*mmBot).Start 
      /go/src/github.com/infracloudio/botkube/pkg/bot/mattermost.go:101 +0x546​
@bilaloslo92 bilaloslo92 changed the title [BUG] Botkube 8.0 - crash loop - invalid memory address or nil pointer deference [BUG] Botkube 0.8.0 - crash loop - invalid memory address or nil pointer deference Jul 11, 2019
@PrasadG193
Copy link
Member

@PrasadG193 PrasadG193 commented Jul 12, 2019

@bilaloslo92 Could you please post your configuration here (make sure you remove sensitive info like certs and tokens)?

@PrasadG193
Copy link
Member

@PrasadG193 PrasadG193 commented Jul 12, 2019

Are you encoding certificate before creating secret?
cat <YOUR_CERTIFICATE> | base64 -w 0

@bilaloslo92
Copy link
Author

@bilaloslo92 bilaloslo92 commented Jul 15, 2019

Hi @PrasadG193 ! Thanks for your reply. See attached config (removed credentials, clustername, and mattermost configuration).

---
# Configmap
apiVersion: v1
kind: ConfigMap
metadata:
  name: botkube-configmap
  labels:
    app: botkube
data:
  config.yaml: |
    ## Resources you want to watch
    resources:
      - name: pods                # Name of the resources e.g pods, deployments, ingresses, etc. (Resource name must be in plural form)
        namespaces:               # List of namespaces, "all" will watch all the namespaces
          - all
        events:                   # List of lifecycle events you want to receive, e.g create, update, delete, error OR all
          - create
          - delete
          - error
      - name: services
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: deployments
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: ingresses
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: nodes
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: namespaces
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: persistentvolumes
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: persistentvolumeclaims
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: secrets
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: configmaps
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: daemonsets
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: jobs
        namespaces:
          - all
        events:
          - create
          - update
          - delete
          - error
                  
      - name: roles
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: rolebindings
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: clusterroles
        namespaces:
          - all
        events:
          - create
          - delete
          - error
      - name: clusterrolebindings
        namespace:
          - all
        events:
          - create
          - delete
          - error
      - name: nodes
        namespaces:
          - all
        events:
          - create
          - delete
          - error
    # Check true if you want to receive recommendations
    # about the best practices for the created resource
    recommendations: true

    # Channels configuration
    communications:
      # Settings for Slack
      slack:
        enabled: false
        channel: 'SLACK_CHANNEL'
        token: 'SLACK_API_TOKEN'

      # Settings for Mattermost
      mattermost:
        enabled: true
        url: ''                # URL where Mattermost is running. e.g https://example.com:9243
        token: ''                   # Personal Access token generated by BotKube user
        team: ''                     # Mattermost Team to configure with BotKube
        channel: ''               # Mattermost Channel for receiving BotKube alerts

      # Settings for ELS
      elasticsearch:
        enable: false
        server: 'ELASTICSEARCH_ADDRESS'              # e.g https://example.com:9243
        username: 'ELASTICSEARCH_USERNAME'
        password: 'ELASTICSEARCH_PASSWORD'
        # ELS index settings
        index:
          name: botkube
          type: botkube-event
          shards: 1
          replicas: 0

    # Setting to support multiple clusters
    settings:
      # Cluster name to differentiate incoming messages
      clustername: cluster_name
      # Set true to enable kubectl commands execution
      allowkubectl: false
      # Set false to disable upgrade notification
      upgradeNotifier: true
---
# serviceaccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: botkube-sa
  labels:
    app: botkube
---
# Source: botkube/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: botkube-clusterrole
  labels:
    app: botkube
rules:
  - apiGroups: ["*"]
    resources: ["*"]
    verbs: ["get", "watch", "list"]
---
# clusterrolebinding
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: botkube-clusterrolebinding
  labels:
    app: botkube
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: botkube-clusterrole
subjects:
- kind: ServiceAccount
  name: botkube-sa
  namespace: botkube
---
# Secret
apiVersion: v1
kind: Secret
metadata:
  name: botkube-secret
  labels:
    app: botkube
data:
  ca-certificates.crt: 'ENCODED_CERTIFICATE '
---
# deployment
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: botkube
  labels:
    component: controller
    app: botkube
spec:
  replicas: 1
  selector:
    matchLabels:
      component: controller
      app: botkube
  template:
    metadata:
      labels:
        component: controller
        app: botkube
    spec:
      serviceAccountName: botkube-sa
      containers:
        - name: botkube
          image: "infracloud/botkube:v0.8.0"
          imagePullPolicy: Always
          volumeMounts:
            - name: config-volume
              mountPath: "/config"
            - name: certs
              mountPath: "/etc/ssl/certs"
          env:
            - name: CONFIG_PATH
              value: "/config/"
            # set one of the log levels- info, warn, debug, error, fatal, panic
            - name: LOG_LEVEL
              value: "info"
            # set BotKube release version
            - name: BOTKUBE_VERSION
              value: v0.8.0
          resources:
            requests:
              cpu: 100m
              memory: "512Mi"
            limits:
              memory: "1024Mi"
      volumes:
        - name: config-volume
          configMap:
            name: botkube-configmap
        - name: certs
          secret:
            secretName: botkube-secret

@PrasadG193
Copy link
Member

@PrasadG193 PrasadG193 commented Jul 15, 2019

Just wanted to make sure:

  1. Are you passing correct config in
      # Settings for Mattermost
      mattermost:
        enabled: true
        url: ''                # URL where Mattermost is running. e.g https://example.com:9243
        token: ''                   # Personal Access token generated by BotKube user
        team: ''                     # Mattermost Team to configure with BotKube
        channel: ''               # Mattermost Channel for receiving BotKube alerts
  1. Are you passing encoded self signed certs in
# Secret
apiVersion: v1
kind: Secret
metadata:
  name: botkube-secret
  labels:
    app: botkube
data:
  ca-certificates.crt: 'ENCODED_CERTIFICATE '

@bilaloslo92
Copy link
Author

@bilaloslo92 bilaloslo92 commented Jul 15, 2019

  1. Yes, I am passing Mattermost token, url, etc in that section which works.
  2. Yes, I am passing the base64 encoded CA certificate.

Thanks for quick reply.

@PrasadG193
Copy link
Member

@PrasadG193 PrasadG193 commented Jul 16, 2019

@bilaloslo92 Are you creating BotKube in botkube namespace?

@PrasadG193
Copy link
Member

@PrasadG193 PrasadG193 commented Jul 16, 2019

Also, please verify if secret is getting populated properly with -
kubectl get secret botkube-secret -n botkube -oyaml

@bilaloslo92
Copy link
Author

@bilaloslo92 bilaloslo92 commented Jul 17, 2019

@PrasadG193 - I have tried to create BotKube in multiple namespaces. Is there any restrictions in which namespace it is created in?

I can confirm the secret is populated. BotKube is able to communicate with Mattermost and send notifications, however the pod is restarted because of the error in the initial post. Thats why I am also being spammed by the notification: ...and now my watch begins for cluster clustername!" in my mattermost channel.

@PrasadG193
Copy link
Member

@PrasadG193 PrasadG193 commented Jul 17, 2019

@bilaloslo92 If you want to deploy BotKube in a different namespace, you need to modify the namespace field in the ClusterRoleBinding resource accordingly.

The Pod crashing issue indeed looks strange to me. I am not able reproduce this on my setup. I'll keep looking into this.

@bilaloslo92
Copy link
Author

@bilaloslo92 bilaloslo92 commented Jul 17, 2019

@PrasadG193 I have changed the namespace in the ClusterRoleBinding accoundingly. It is very strange, I am also unsure why this happens. I have made as few modifications as nececcary. Hopefully you are able to find out the issue. Do you get any suggestion on what it could be based on the error?

@sanketsudake sanketsudake added this to the v0.9.0 milestone Jul 30, 2019
@gangseok514
Copy link
Contributor

@gangseok514 gangseok514 commented Oct 2, 2019

Hello, I faced the same issue. I solved with adding event == nil check in mattermost.go.
In fact I don't know why event is nil. :)

@bilaloslo92
Copy link
Author

@bilaloslo92 bilaloslo92 commented Oct 2, 2019

Hi @gangseok514. Good that you managed to reproduce this issue. I hope this can be fixed in the next release, since I don't want to compile it from source. @PrasadG193 - When is next release comming up and could this be included?

@PrasadG193
Copy link
Member

@PrasadG193 PrasadG193 commented Oct 3, 2019

@gangseok514 Could you please raise the PR for the fix. Unless there are any surprises, we'll be releasing v0.9.0 by the end of this week or early next week

@bilaloslo92
Copy link
Author

@bilaloslo92 bilaloslo92 commented Oct 3, 2019

@gangseok514 - Could you please create an PR ? Since you managed to solve the issue. So the fix can be released in v0.9.0 :)

gangseok514 pushed a commit to gangseok514/botkube that referenced this issue Oct 3, 2019
infracloudio#126

event is nil in this case. adding nil check is needed

Signed-off-by: gangseok.lee <gangseok.lee@samsung.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

4 participants