Replies: 2 comments
-
this is because not all nodes have the same configuration or not all hosts resolve properly to the local to their respective hostnames. the crash is a side affect of the configuration issue in the environment. |
Beta Was this translation helpful? Give feedback.
0 replies
-
Hello, we have checked the config files, and the host files. We are sure that they are the same. Just for a try, we also could form a separate cluster with the new nodes (minio-9 to 12) without any problem. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
We have a working minio deployment with 8 nodes and 32 drives, in 2 server pools. (Originally there was one pool and we expanded it with new pool 4-6 months ago). Now we would like to expand it with another new pool but after the configuration of the nodes we can't restart the cluster. Some of the nodes simply stuck in the restart process and it seems that they are waiting to other nodes, some nodes crashing almost immediately after the restart.
Deployment:
Oroginal:
2x4 nodes, 4 5TB drive in each node.
OS: Ubuntu 20.04
RAM: 8GB
CPU 4
minio version: minio -version
minio version RELEASE.2024-06-26T01-06-18Z (commit-id=499531f0b5863ad5b3a5e92a24239138c3ba7a13)
Runtime: go1.22.4 linux/amd64
License: GNU AGPLv3 - https://www.gnu.org/licenses/agpl-3.0.html
Copyright: 2015-2024 MinIO, Inc.
hostnames: mino-1 to mino-8
disks: /mnt/disk1 to /mnt/disk4
So the orginal config was:
MINIO_VOLUMES="http://minio-{1...4}:9000/mnt/disk{1...4}/minio http://minio-{5...8}:9000/mnt/disk{1...4}/minio"
MINIO_OPTS="--console-address :9001"
MINIO_ROOT_USER=secret
MINIO_ROOT_PASSWORD=secret
MINIO_SERVER_URL="https://example.com"
MINIO_PROMETHEUS_URL="http://minio-1:9090/"
It is working perfectly
So we have created 4 new node (minio-9 to minio-12). HW is the same, hostnames are sequential, disk layout is the same. There is no firewall, all host in the same lan. The new configuration is (we have updated it on the old nodes too)
MINIO_VOLUMES="http://minio-{1...4}:9000/mnt/disk{1...4}/minio http://minio-{5...8}:9000/mnt/disk{1...4}/minio http://minio-{9...12}:9000/mnt/disk{1...4}/minio"
MINIO_OPTS="--console-address :9001"
MINIO_ROOT_USER=secret
MINIO_ROOT_PASSWORD=secret
MINIO_SERVER_URL="https://example.com"
MINIO_PROMETHEUS_URL="http://minio-1:9090/"
The logs for a node what is "waiting"
minio-7.log
And one what is "crashing":
minio-1.log
Any help would be appreciated, thank You
Pali
Beta Was this translation helpful? Give feedback.
All reactions