The storage and console components can't run

[yanruogu]

Please check the FAQ documentation before raising an issue

Describe the bug (required)

my docker-compose.yaml like this：

version: '3.4'
services:
  metad0:
    image: vesoft/nebula-metad:v3.2.0
    environment:
      USER: root
      TZ:   "Asia/Shanghai"
    command:
      - --meta_server_addrs=metad0:9559
      - --local_ip=metad0
      - --ws_ip=metad0
      - --port=9559
      - --ws_http_port=19559
      - --data_path=/data/meta
      - --log_dir=/logs
      - --v=0
      - --minloglevel=0
    healthcheck:
      test: ["CMD", "curl", "-sf", "http://metad0:19559/status"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s
    ports:
      - "9559:9559"
      - 19559
      - 19560
    volumes:
      - ./data/meta0:/data/meta
      - ./logs/meta0:/logs
    networks:
      - nebula-net
    restart: on-failure
    cap_add:
      - SYS_PTRACE

  storaged0:
    image: vesoft/nebula-storaged:v3.2.0
    environment:
      USER: root
      TZ:   "Asia/Shanghai"
    command:
      - --meta_server_addrs=metad0:9559
      - --local_ip=storaged0
      - --ws_ip=storaged0
      - --port=9779
      - --ws_http_port=19779
      - --data_path=/data/storage
      - --log_dir=/logs
      - --v=0
      - --minloglevel=0
    depends_on:
      - metad0
    healthcheck:
      test: ["CMD", "curl", "-sf", "http://storaged0:19779/status"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s
    ports:
      - "9779:9779"
      - 19779
      - 19780
    volumes:
      - ./data/storage0:/data/storage
      - ./logs/storage0:/logs
    networks:
      - nebula-net
    restart: on-failure
    cap_add:
      - SYS_PTRACE

  graphd:
    image: vesoft/nebula-graphd:v3.2.0
    environment:
      USER: root
      TZ:   "Asia/Shanghai"
    command:
      - --meta_server_addrs=metad0:9559
      - --port=9669
      - --local_ip=graphd
      - --ws_ip=graphd
      - --ws_http_port=19669
      - --log_dir=/logs
      - --v=0
      - --minloglevel=0
    depends_on:
      - storaged0
    healthcheck:
      test: ["CMD", "curl", "-sf", "http://graphd:19669/status"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s
    ports:
      - "9669:9669"
      - 19669
      - 19670
    volumes:
      - ./logs/graph:/logs
    networks:
      - nebula-net
    restart: on-failure
    cap_add:
      - SYS_PTRACE

  console:
    image: vesoft/nebula-console:v3.0.0
    entrypoint: ""
    command:
      - sh
      - -c
      - |
        sleep 3 &&
        nebula-console -addr graphd -port 9669 -u root -p nebula -e 'ADD HOSTS "storaged0":9779 &&
        sleep 36000
    depends_on:
      - graphd
    networks:
      - nebula-net

networks:
  nebula-net:

when I run it，the storage and console can't healthy：

b83e1900fb44   vesoft/nebula-console:v3.0.0                                           "sh -c 'sleep 3 &&\nn…"   20 minutes ago   Exited (2) 20 minutes ago                                                                                                                                                                      nebula-console-1
3a0a3f296990   vesoft/nebula-graphd:v3.2.0                                            "/usr/local/nebula/b…"    21 minutes ago   Up 20 minutes (healthy)     0.0.0.0:9669->9669/tcp, :::9669->9669/tcp, 0.0.0.0:49216->19669/tcp, :::49216->19669/tcp, 0.0.0.0:49215->19670/tcp, :::49215->19670/tcp                            nebula-graphd-1
50d4a0b8121d   vesoft/nebula-storaged:v3.2.0                                          "/usr/local/nebula/b…"    21 minutes ago   Up 20 minutes (unhealthy)   9777-9778/tcp, 9780/tcp, 0.0.0.0:9779->9779/tcp, :::9779->9779/tcp, 0.0.0.0:49214->19779/tcp, :::49214->19779/tcp, 0.0.0.0:49213->19780/tcp, :::49213->19780/tcp   nebula-storaged0-1
9a8fbf834404   vesoft/nebula-metad:v3.2.0                                             "/usr/local/nebula/b…"    21 minutes ago   Up 20 minutes (healthy)     0.0.0.0:9559->9559/tcp, :::9559->9559/tcp, 9560/tcp, 0.0.0.0:49212->19559/tcp, :::49212->19559/tcp, 0.0.0.0:49211->19560/tcp, :::49211->19560/tcp                  nebula-metad0-1

I find the error log for storage:

[root@VM-123-37-centos storage0]# tail -f nebula-storaged.ERROR 
E20221219 15:23:43.825707     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:23:56.830708     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:24:09.834656     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:24:22.838608     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:24:35.843055     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:24:48.847618     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:25:01.852696     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:25:14.856086     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:25:27.860548     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:25:40.865533     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
E20221219 15:25:53.870564     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!

[root@VM-123-37-centos storage0]# tail -f storaged-stderr.log 
I20221219 15:25:27.860584     1 MetaClient.cpp:137] Waiting for the metad to be ready!
W20221219 15:25:37.860674     1 FileBasedClusterIdMan.cpp:43] Open file failed, error No such file or directory
E20221219 15:25:40.865533     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
I20221219 15:25:40.865566     1 MetaClient.cpp:137] Waiting for the metad to be ready!
W20221219 15:25:50.865658     1 FileBasedClusterIdMan.cpp:43] Open file failed, error No such file or directory
E20221219 15:25:53.870564     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
I20221219 15:25:53.870604     1 MetaClient.cpp:137] Waiting for the metad to be ready!
W20221219 15:26:03.870697     1 FileBasedClusterIdMan.cpp:43] Open file failed, error No such file or directory
E20221219 15:26:06.874982     1 MetaClient.cpp:112] Heartbeat failed, status:Machine not existed!
I20221219 15:26:06.875025     1 MetaClient.cpp:137] Waiting for the metad to be ready!

the log told me the metad is not ready ， In fact, the metad is running.

I don't know why!

Your Environments (required)

• OS: Linux VM-123-37-centos 3.10.0-1160.66.1.el7.x86_64 [doc] add cn doc #1 SMP Wed May 18 16:02:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
• docker version:

Client: Docker Engine - Community
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:05:12 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:03:33 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.7
  GitCommit:        0197261a30bf81f1ee8e6a4dd2dea0ef95d67ccb
 runc:
  Version:          1.1.3
  GitCommit:        v1.1.3-0-g6724737
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker-compose version:

Docker Compose version v2.9.0

[wey-gu]

Did you try connecting to graphd and do add hosts on your own? the current healthiness/readiness checking mechanism isn't perfect, that is, when the storage host was not enabled(by adding hosts), it'll show as unhealthy.

[abby-cyber]

Close this issue. For more questions, submit an issue here.