Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s启动2.2.0版本集群失败,日志报Fail to get leader of group naming_service_metadata, Unknown leader #372

Closed
InGramViking opened this issue Dec 22, 2022 · 4 comments

Comments

@InGramViking
Copy link
Contributor

k8s版本: 1.20.11-aliyun.1
nacos版本: nacos/nacos-server:v2.2.0
使用阿里云NAS做的pvc,挂载的nfs3
集群主机连接RDS是可以通的,命名空间中有一个1.4版本的在运行
数据库帐号密码正确,使用的初始化sql为 https://github.com/alibaba/nacos/blob/master/distribution/conf/mysql-schema.sql

---
apiVersion: v1
kind: Service
metadata:
  name: nacos220-headless
  namespace: pluginalpha
  labels:
    app: nacos220-headless
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: lb-abcdefghijklmcdcdcdcd
    service.beta.kubernetes.io/alicloud-loadbalancer-force-override-listeners: 'true'
spec:
  publishNotReadyAddresses: true
  ports:
    - port: 28848
      name: server
      targetPort: 8848
    - port: 9848
      name: client-rpc
      targetPort: 9848
    - port: 9849
      name: raft-rpc
      targetPort: 9849
    # 与1.4版本的nacos共用了一个SLB,所以注释掉了这个端口
    # ## 兼容1.4.x版本的选举端口
    # - port: 7848
    #   name: old-raft-rpc
    #   targetPort: 7848
  selector:
    app: nacos220
  sessionAffinity: None
  type: LoadBalancer
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos220-cm
  namespace: pluginalpha
data:
  mysql.host: "rm-2zabcdefghijklmncd.mysql.rds.aliyuncs.com"
  mysql.db.name: "nacos220_test"
  mysql.port: "3306"
  mysql.user: "nacos220_test"
  mysql.password: "rlllllllllllllll+Sa"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  # namespace里面有老版本的nacos了,所以改了名
  name: nacos220
  namespace: pluginalpha
spec:
  serviceName: nacos220-headless
  podManagementPolicy: Parallel
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos220
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos220-headless
              topologyKey: "kubernetes.io/hostname"
      # 因为要用SLB不能设置  clusterIP: None, 所以注释掉了peer
      # initContainers:
      #   - name: peer-finder-plugin-install
      #     image: registry-vpc.cn-beijing.aliyuncs.com/acegear/nacos-peer-finder-plugin:1.1
      #     imagePullPolicy: Always
      #     volumeMounts:
      #       - mountPath: /home/nacos/plugins/peer-finder
      #         name: data
      #         subPath: peer-finder
      containers:
        - name: nacos
          imagePullPolicy: Always
          image: nacos/nacos-server:v2.2.0
          resources:
            requests:
              memory: "2Gi"
          #     cpu: "500m"
          ports:
            - containerPort: 8848
              name: client-port
            - containerPort: 9848
              name: client-rpc
            - containerPort: 9849
              name: raft-rpc
            - containerPort: 7848
              name: old-raft-rpc
          env:
            - name: NACOS_REPLICAS
              value: "3"
            # 没用peer就注释掉这个了
            # - name: SERVICE_NAME
            #   value: "nacos220-headless"
            # - name: DOMAIN_NAME
            #   value: "cluster.local"
            # - name: POD_NAMESPACE
            #   valueFrom:
            #     fieldRef:
            #       apiVersion: v1
            #       fieldPath: metadata.namespace
            - name: MYSQL_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.host
            - name: MYSQL_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.db.name
            - name: MYSQL_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.port
            - name: MYSQL_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.user
            - name: MYSQL_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.password
            - name: MODE
              value: "cluster"
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              value: "nacos220-0.nacos220-headless.pluginalpha.svc.cluster.local:8848 nacos220-1.nacos220-headless.pluginalpha.svc.cluster.local:8848 nacos220-2.nacos220-headless.pluginalpha.svc.cluster.local:8848"
          volumeMounts:
            # - name: data
            #   mountPath: /home/nacos/plugins/peer-finder
            #   subPath: peer-finder
            - name: data
              mountPath: /home/nacos/data
              subPath: data
            - name: data
              mountPath: /home/nacos/logs
              subPath: logs
      volumes:
      # 通过阿里云NAS做的nfs的pvc
        - name: data
          persistentVolumeClaim:
            claimName: nacos220-alpha
  selector:
    matchLabels:
      app: nacos220

执行部署后的日志

+ export CUSTOM_SEARCH_NAMES=application
+ CUSTOM_SEARCH_NAMES=application
+ export CUSTOM_SEARCH_LOCATIONS=file:/home/nacos/conf/
+ CUSTOM_SEARCH_LOCATIONS=file:/home/nacos/conf/
+ export MEMBER_LIST=
+ MEMBER_LIST=
+ PLUGINS_DIR=/home/nacos/plugins/peer-finder
+ [[ cluster == \s\t\a\n\d\a\l\o\n\e ]]
+ [[ '' == \e\m\b\e\d\d\e\d ]]
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m'
+ [[ n == \y ]]
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof'
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages'
+ print_servers
+ [[ ! -d /home/nacos/plugins/peer-finder ]]
+ echo ''
+ for server in '${NACOS_SERVERS}'
+ echo nacos220-0.nacos220-headless.pluginalpha.svc.cluster.local:8848
+ for server in '${NACOS_SERVERS}'
+ echo nacos220-1.nacos220-headless.pluginalpha.svc.cluster.local:8848
+ for server in '${NACOS_SERVERS}'
+ echo nacos220-2.nacos220-headless.pluginalpha.svc.cluster.local:8848
+ [[ all == \c\o\n\f\i\g ]]
+ [[ all == \n\a\m\i\n\g ]]
+ [[ ! -z '' ]]
+ [[ ! -z '' ]]
+ [[ ! -z '' ]]
+ [[ ! -z '' ]]
+ [[ ! -z '' ]]
+ [[ hostname == \h\o\s\t\n\a\m\e ]]
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true'
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list='
++ /usr/lib/jvm/java-1.8.0-openjdk/bin/java -version
++ sed -E -n 's/.* version "([0-9]*).*$/\1/p'
+ JAVA_MAJOR_VERSION=1
+ [[ 1 -ge 9 ]]
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql'
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql -Xloggc:/home/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M'
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql -Xloggc:/home/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dnacos.home=/home/nacos'
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql -Xloggc:/home/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dnacos.home=/home/nacos -jar /home/nacos/target/nacos-server.jar'
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql -Xloggc:/home/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dnacos.home=/home/nacos -jar /home/nacos/target/nacos-server.jar '
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql -Xloggc:/home/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dnacos.home=/home/nacos -jar /home/nacos/target/nacos-server.jar  --spring.config.additional-location=file:/home/nacos/conf/'
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql -Xloggc:/home/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dnacos.home=/home/nacos -jar /home/nacos/target/nacos-server.jar  --spring.config.additional-location=file:/home/nacos/conf/ --spring.config.name=application'
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql -Xloggc:/home/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dnacos.home=/home/nacos -jar /home/nacos/target/nacos-server.jar  --spring.config.additional-location=file:/home/nacos/conf/ --spring.config.name=application --logging.config=/home/nacos/conf/nacos-logback.xml'
+ JAVA_OPT=' -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql -Xloggc:/home/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dnacos.home=/home/nacos -jar /home/nacos/target/nacos-server.jar  --spring.config.additional-location=file:/home/nacos/conf/ --spring.config.name=application --logging.config=/home/nacos/conf/nacos-logback.xml --server.max-http-header-size=524288'
+ echo 'Nacos is starting, you can docker logs your container'
Nacos is starting, you can docker logs your container
+ exec /usr/lib/jvm/java-1.8.0-openjdk/bin/java -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/nacos/logs/java_heapdump.hprof -XX:-UseLargePages -Dnacos.preferHostnameOverIp=true -Dnacos.member.list= -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk/lib/ext:/home/nacos/plugins/health:/home/nacos/plugins/cmdb:/home/nacos/plugins/mysql -Xloggc:/home/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dnacos.home=/home/nacos -jar /home/nacos/target/nacos-server.jar --spring.config.additional-location=file:/home/nacos/conf/ --spring.config.name=application --logging.config=/home/nacos/conf/nacos-logback.xml --server.max-http-header-size=524288
         ,--.
       ,--.'|
   ,--,:  : |                                           Nacos 2.2.0
,`--.'`|  ' :                       ,---.               Running in cluster mode, All function modules
|   :  :  | |                      '   ,'\   .--.--.    Port: 8848
:   |   \ | :  ,--.--.     ,---.  /   /   | /  /    '   Pid: 1
|   : '  '; | /       \   /     \.   ; ,. :|  :  /`./   Console: http://nacos220-0.nacos220-headless.pluginalpha.svc.cluster.local:8848/nacos/index.html
'   ' ;.    ;.--.  .-. | /    / ''   | |: :|  :  ;_
|   | | \   | \__\/: . ..    ' / '   | .; : \  \    `.      https://nacos.io
'   : |  ; .' ," .--.; |'   ; :__|   :    |  `----.   \
|   | '`--'  /  /  ,.  |'   | '.'|\   \  /  /  /`--'  /
'   : |     ;  :   .'   \   :    : `----'  '--'.     /
;   |.'     |  ,     .-./\   \  /            `--'---'
'---'        `--`---'     `----'
2022-12-22 11:27:31,748 INFO The server IP list of Nacos is [nacos220-0.nacos220-headless.pluginalpha.svc.cluster.local:8848, nacos220-1.nacos220-headless.pluginalpha.svc.cluster.local:8848, nacos220-2.nacos220-headless.pluginalpha.svc.cluster.local:8848]
2022-12-22 11:27:32,750 INFO Nacos is starting...
2022-12-22 11:27:33,754 INFO Nacos is starting...
2022-12-22 11:27:34,767 INFO Nacos is starting...
2022-12-22 11:27:35,771 INFO Nacos is starting...
2022-12-22 11:27:36,775 INFO Nacos is starting...
2022-12-22 11:27:37,782 INFO Nacos is starting...
2022-12-22 11:27:38,786 INFO Nacos is starting...
2022-12-22 11:27:39,792 INFO Nacos is starting...
2022-12-22 11:27:40,802 INFO Nacos is starting...
2022-12-22 11:27:41,514 INFO Nacos Log files: /home/nacos/logs
2022-12-22 11:27:41,515 INFO Nacos Log files: /home/nacos/conf
2022-12-22 11:27:41,515 INFO Nacos Log files: /home/nacos/data
2022-12-22 11:27:41,516 ERROR Startup errors : 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'memoryMonitor' defined in URL [jar:file:/home/nacos/target/nacos-server.jar!/BOOT-INF/lib/nacos-config-2.2.0.jar!/com/alibaba/nacos/config/server/monitor/MemoryMonitor.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'asyncNotifyService': Unsatisfied dependency expressed through field 'dumpService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'externalDumpService': Invocation of init method failed; nested exception is ErrCode:500, ErrMsg:Nacos Server did not start because dumpservice bean construction failure :
errCode: 102, errMsg: dataSource or tableName is null 
        at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800)
        at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
        at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:745)
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:420)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:307)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1317)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
        at com.alibaba.nacos.Nacos.main(Nacos.java:35)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'asyncNotifyService': Unsatisfied dependency expressed through field 'dumpService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'externalDumpService': Invocation of init method failed; nested exception is ErrCode:500, ErrMsg:Nacos Server did not start because dumpservice bean construction failure :
errCode: 102, errMsg: dataSource or tableName is null 
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:659)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:639)
        at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
        at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1389)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1309)
        at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887)
        at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791)
        ... 27 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'externalDumpService': Invocation of init method failed; nested exception is ErrCode:500, ErrMsg:Nacos Server did not start because dumpservice bean construction failure :
errCode: 102, errMsg: dataSource or tableName is null 
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:160)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
        at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1389)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1309)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:656)
        ... 42 common frames omitted
Caused by: com.alibaba.nacos.api.exception.NacosException: Nacos Server did not start because dumpservice bean construction failure :
errCode: 102, errMsg: dataSource or tableName is null 
        at com.alibaba.nacos.config.server.service.dump.DumpService.dumpOperate(DumpService.java:260)
        at com.alibaba.nacos.config.server.service.dump.ExternalDumpService.init(ExternalDumpService.java:61)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157)
        ... 54 common frames omitted
Caused by: com.alibaba.nacos.api.exception.runtime.NacosRuntimeException: errCode: 102, errMsg: dataSource or tableName is null 
        at com.alibaba.nacos.plugin.datasource.MapperManager.findMapper(MapperManager.java:102)
        at com.alibaba.nacos.config.server.service.repository.extrnal.ExternalConfigInfoPersistServiceImpl.findConfigMaxId(ExternalConfigInfoPersistServiceImpl.java:612)
        at com.alibaba.nacos.config.server.service.dump.processor.DumpAllProcessor.process(DumpAllProcessor.java:51)
        at com.alibaba.nacos.config.server.service.dump.DumpService.dumpConfigInfo(DumpService.java:317)
        at com.alibaba.nacos.config.server.service.dump.DumpService.dumpOperate(DumpService.java:230)
        ... 62 common frames omitted
2022-12-22 11:27:41,738 WARN [WatchFileCenter] start close
2022-12-22 11:27:41,739 WARN [WatchFileCenter] start to shutdown this watcher which is watch : /home/nacos/conf
2022-12-22 11:27:41,744 WARN [WatchFileCenter] already closed
2022-12-22 11:27:41,744 WARN [NotifyCenter] Start destroying Publisher
2022-12-22 11:27:41,744 WARN [NotifyCenter] Destruction of the end
2022-12-22 11:27:41,745 ERROR Nacos failed to start, please see /home/nacos/logs/nacos.log for more details.
2022-12-22 11:27:41,807 WARN [ThreadPoolManager] Start destroying ThreadPool
2022-12-22 11:27:41,807 WARN [ThreadPoolManager] Destruction of the end
@InGramViking
Copy link
Contributor Author

报上面错误的原因疑似最新版2.2.0镜像sha256:be794368c96f90e42a6dd9f288d3fb12d69608cebd782eb091a489ccb48cfd28 里面,没有设置spring.sql.init.platform=${SPRING_DATASOURCE_PLATFORM:} 默认值mysql,手动添加后nacos-0和nacos-1启动成功,nacos-2启动失败

- name: SPRING_DATASOURCE_PLATFORM
  value: "mysql"

image

@InGramViking
Copy link
Contributor Author

修改了配置信息:
将pvc挂载的nfs3改为了nfs4

  mountOptions:
  - tcp,noresvport,minorversion=0,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2
  - vers=4

更新yaml为:

---
apiVersion: v1
kind: Service
metadata:
  name: nacos220-headless
  namespace: pluginalpha
  labels:
    app: nacos220-headless
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: lb-abcdefghijklmcdcdcdcd
    service.beta.kubernetes.io/alicloud-loadbalancer-force-override-listeners: 'true'
spec:
  publishNotReadyAddresses: true
  ports:
    - port: 28848
      name: server
      targetPort: 8848
    - port: 9848
      name: client-rpc
      targetPort: 9848
    - port: 9849
      name: raft-rpc
      targetPort: 9849
    # 与1.4版本的nacos共用了一个SLB,所以注释掉了这个端口
    # ## 兼容1.4.x版本的选举端口
    # - port: 7848
    #   name: old-raft-rpc
    #   targetPort: 7848
  selector:
    app: nacos220
  sessionAffinity: None
  type: LoadBalancer
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos220-cm
  namespace: pluginalpha
data:
  mysql.host: "rm-2zabcdefghijklmncd.mysql.rds.aliyuncs.com"
  mysql.db.name: "nacos220_test"
  mysql.port: "3306"
  mysql.user: "nacos220_test"
  mysql.password: "rlllllllllllllll+Sa"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  # namespace里面有老版本的nacos了,所以改了名
  name: nacos220
  namespace: pluginalpha
spec:
  serviceName: nacos220-headless
  # podManagementPolicy: Parallel
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos220
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos220-headless
              topologyKey: "kubernetes.io/hostname"
      # 因为要用SLB不能设置  clusterIP: None, 所以注释掉了peer
      # initContainers:
      #   - name: peer-finder-plugin-install
      #     image: registry-vpc.cn-beijing.aliyuncs.com/acegear/nacos-peer-finder-plugin:1.1
      #     imagePullPolicy: Always
      #     volumeMounts:
      #       - mountPath: /home/nacos/plugins/peer-finder
      #         name: data
      #         subPath: peer-finder
      containers:
        - name: nacos
          imagePullPolicy: Always
          image: nacos/nacos-server:v2.2.0
          resources:
            requests:
              memory: "2Gi"
          #     cpu: "500m"
          ports:
            - containerPort: 8848
              name: client-port
            - containerPort: 9848
              name: client-rpc
            - containerPort: 9849
              name: raft-rpc
            - containerPort: 7848
              name: old-raft-rpc
          env:
            - name: NACOS_REPLICAS
              value: "3"
            # 没用peer就注释掉这个了
            # - name: SERVICE_NAME
            #   value: "nacos220-headless"
            # - name: DOMAIN_NAME
            #   value: "cluster.local"
            # - name: POD_NAMESPACE
            #   valueFrom:
            #     fieldRef:
            #       apiVersion: v1
            #       fieldPath: metadata.namespace
            - name: MYSQL_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.host
            - name: MYSQL_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.db.name
            - name: MYSQL_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.port
            - name: MYSQL_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.user
            - name: MYSQL_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos220-cm
                  key: mysql.password
            - name: SPRING_DATASOURCE_PLATFORM
              value: "mysql"
            - name: MODE
              value: "cluster"
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              value: "nacos220-0.nacos220-headless.pluginalpha.svc.cluster.local:8848 nacos220-1.nacos220-headless.pluginalpha.svc.cluster.local:8848 nacos220-2.nacos220-headless.pluginalpha.svc.cluster.local:8848"
          volumeMounts:
            # - name: data
            #   mountPath: /home/nacos/plugins/peer-finder
            #   subPath: peer-finder
            - name: data
              mountPath: /home/nacos/data
              subPath: data
            - name: data
              mountPath: /home/nacos/logs
              subPath: logs
      volumes:
      # 通过阿里云NAS做的nfs的pvc
        - name: data
          persistentVolumeClaim:
            claimName: nacos220-alpha
  selector:
    matchLabels:
      app: nacos220

将三节点同时启动关闭了,添加上了SPRING_DATASOURCE_PLATFORM参数。
启动后有一两个节点会启动失败,日志报Fail to init node, please see the logs to find the reason,附件附上了所有日志,好像是raft选举问题,是因为我把old-raft-rpc 注释掉的原因吗?
nacos2.2.0cluster-alinaspvc-alislb.tar.gz

@InGramViking InGramViking changed the title k8s启动2.2.0版本集群失败,启动报dataSource or tableName is null k8s启动2.2.0版本集群失败,日志报Fail to get leader of group naming_service_metadata, Unknown leader Dec 22, 2022
@InGramViking
Copy link
Contributor Author

去掉pvc,三个节点集群是正常的,但是相应的就丧失了动态扩缩容的能力,我们需要这个,能否有人看一下问题,集群+pvc模式的所有日志在上面的压缩包中

image

@InGramViking
Copy link
Contributor Author

已解决

alibaba/nacos#9766 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant