Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Voyager pod is restarting itself when attached backend pod restarts #69

Closed
skuda opened this issue May 5, 2017 · 2 comments

Comments

Projects
None yet
3 participants
@skuda
Copy link

commented May 5, 2017

I talked with @tamalsaha in Slack about this, basically, when I delete the attached pod, the Voyager pod terminates too and it keeps restarting for a while until it finally keeps running.

skuda@skuda ~ $ kubectl get pods -o wide | grep user
mysql-user-0                            2/2       Running   0          2d        10.200.65.19   master-03
mysql-user-1                            2/2       Running   0          1h        10.200.75.4    master-02
voyager-mysql-user-ingress-rd3gn        1/1       Running   0          1h        10.0.0.3       master-03

skuda@skuda ~ $ kubectl delete pod mysql-user-0
pod "mysql-user-0" deleted

skuda@skuda ~ $ kubectl get pods -o wide | grep user
mysql-user-0                            2/2       Terminating   0          2d        10.200.65.19   master-03
mysql-user-1                            2/2       Running       0          1h        10.200.75.4    master-02
voyager-mysql-user-ingress-rd3gn        0/1       Terminating   0          1h        10.0.0.3       master-03

This is the yaml file related to this setup:

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mysql-user
spec:
  serviceName: mysql-user-headless
  replicas: 2
  template:
    metadata:
      labels:
        app: mysql-user
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
        
    spec:
      restartPolicy: Always
      terminationGracePeriodSeconds: 10
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: "app"
                    operator: In
                    values: 
                    - mysql-user-headless
                topologyKey: "kubernetes.io/hostname"

      containers:
      - name: mysql
        imagePullPolicy: Always
        image: percona:5.7
        resources:
          requests:
            memory: "4Gi"
            cpu: "2"
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: config-volume
          mountPath: /etc/mysql/mysql.cnf
          subPath: mysql.cnf
        - name: config-volume
          mountPath: /etc/mysql/conf.d/extra.cnf
          subPath: extra.cnf
        - name: data
          mountPath: /var/lib/mysql
        - name: local-fs
          mountPath: /tmp
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: percona
        readinessProbe:
          tcpSocket:
            port: 3306
          initialDelaySeconds: 10
          periodSeconds: 30
        livenessProbe:
          tcpSocket:
            port: 3306
          initialDelaySeconds: 20
          periodSeconds: 30
      - name: phpmyadmin
        image: phpmyadmin/phpmyadmin:4.6
        ports:
        - containerPort: 80
          name: phpmyadmin
        resources:
          requests:
            memory: "500Mi"
            cpu: "500m"
        env:
        - name: PMA_HOST
          value: 127.0.0.1
        readinessProbe:
          httpGet:
            path: /index.php
            port: 80
          initialDelaySeconds: 10
          periodSeconds: 30
        livenessProbe:
          httpGet:
            path: /index.php
            port: 80
          initialDelaySeconds: 20
          periodSeconds: 30
      volumes:
      - name: config-volume
        configMap:
          name: mysql-user-configmap
      - name: local-fs
        emptyDir: {}
  volumeClaimTemplates:
  - metadata:
      name: data
      annotations:
        volume.beta.kubernetes.io/storage-class: fastio
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 32Gi


---

apiVersion: v1
kind: Service
metadata:
  name: mysql-user-headless
  labels:
    app: mysql-user
spec:
  ports:
  - port: 3306
    targetPort: 3306
    name: mysql
    protocol: TCP
  - port: 80
    targetPort: 80
    name: phpmyadmin
    protocol: TCP
  clusterIP: None
  selector:
    app: mysql-user

---

apiVersion: appscode.com/v1beta1
kind: Ingress
metadata:
  name: mysql-user-ingress
  annotations:
     ingress.appscode.com/type: Daemon
     ingress.appscode.com/daemon.nodeSelector: "kubernetes.io/hostname=master-03"
spec:
  rules:
  - host: phpmyadmin.local
    http:
      paths:
      - path: '/1'
        backend:
          hostNames:
          - mysql-user-0
          serviceName: mysql-user-headless
          servicePort: '80'
          rewriteRule:
          - ^(GET|POST|HEAD)\ /1/(.*)     \1\ /\2
      - path: '/2'
        backend:
          hostNames:
          - mysql-user-1
          serviceName: mysql-user-headless
          servicePort: '80'
          rewriteRule:
          - ^(GET|POST|HEAD)\ /2/(.*)     \1\ /\2

Thanks!

@tamalsaha

This comment has been minimized.

Copy link
Member

commented May 8, 2017

@skuda , can you please try Voyager 1.5.4? This should fix this issue. Please update the appscode-voyager deployment to version 1.5.4

@skuda skuda changed the title Voyager pod is restarting itself when attachd backend pod restarts Voyager pod is restarting itself when attached backend pod restarts May 8, 2017

@skuda

This comment has been minimized.

Copy link
Author

commented May 8, 2017

Tested, it's fixed, thanks @tamalsaha!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.