New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scylla_setup should fail with error when mdadm fails in Silent mode #3433

Closed
harelassayag opened this Issue May 15, 2018 · 0 comments

Comments

Projects
None yet
1 participant
@harelassayag

harelassayag commented May 15, 2018

Installation details
Scylla version (or git commit hash): 2.2.rc0-0.20180513.7ca5e7e99
Cluster size: 1TB
OS (RHEL/CentOS/Ubuntu/AWS AMI): Centos

scylla_setup in silent mode does not even though mdadm fails on both cases below

case 1:

[scylla-test@longevity-1tb-7d-2-2-db-node-b772c7d0-0-1 scylla]$ sudo /usr/lib/scylla/scylla_setup --nic eth0 --disks /dev/nvme0n1,/dev/nvme0n3,/dev/nvme0n5,/dev/nvme0n7,/dev/nvme0n2,/dev/nvme0n4,/dev/nvme0n6,/dev/nvme0n8
Skip any of the following steps by answering 'no'
This is a supported kernel version.
Unable to retrieve version information
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.mirror.constant.com
 * epel: epel.mirror.constant.com
 * extras: mirror.math.princeton.edu
 * updates: reflector.westga.edu
Package ntp-4.2.6p5-28.el7.centos.x86_64 already installed and latest version
Package ntpdate-4.2.6p5-28.el7.centos.x86_64 already installed and latest version
Nothing to do
15 May 11:45:59 ntpdate[12531]: adjust time server 169.254.169.254 offset -0.000057 sec
Creating RAID0 for scylla using 8 disk(s): /dev/nvme0n1 /dev/nvme0n3 /dev/nvme0n5 /dev/nvme0n7 /dev/nvme0n2 /dev/nvme0n4 /dev/nvme0n6 /dev/nvme0n8
mdadm: super1.x cannot open /dev/nvme0n1: Device or resource busy
mdadm: ddf: Cannot use /dev/nvme0n1: Device or resource busy
mdadm: Cannot use /dev/nvme0n1: It is busy
mdadm: cannot open /dev/nvme0n1: Device or resource busy
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e
Setting parameters on /etc/sysconfig/scylla-server
Generating evaluation file sized 10GB... timed out before we could write the entire file. Will continue but accuracy may suffer.
4.37402GB written in 144 seconds

case 2:

018-05-14 14:27:26,432 remote           L0897 DEBUG| Remote [scylla-test@10.142.0.4]: Running 'sudo /usr/lib/scylla/scylla_setup --nic eth0 --disks /dev/nvme0n1,/dev/nvme0n3,/dev/nvme0n5,/dev/nvme0n7,/dev/nvme0n2,/dev/nvme0n4,/dev/nvme0n6,/dev/nvme0n8'
2018-05-14 14:27:26,432 remote           L0097 INFO | [10.142.0.4] Running '/bin/ssh -tt -a -x   -o StrictHostKeyChecking=no -o UserKnownHostsFile=/var/tmp/tmpvw46NF -o BatchMode=yes -o ConnectTimeout=300 -o ServerAliveInterval=300 -l scylla-test -p 22 -i /jenkins/.ssh/scylla-test 10.142.0.4 "sudo /usr/lib/scylla/scylla_setup --nic eth0 --disks /dev/nvme0n1,/dev/nvme0n3,/dev/nvme0n5,/dev/nvme0n7,/dev/nvme0n2,/dev/nvme0n4,/dev/nvme0n6,/dev/nvme0n8"'
2018-05-14 14:27:38,751 remote           L0197 DEBUG| [10.142.0.4] [stdout] Creating RAID0 for scylla using 8 disk(s): /dev/nvme0n1 /dev/nvme0n3 /dev/nvme0n5 /dev/nvme0n7 /dev/nvme0n2 /dev/nvme0n4 /dev/nvme0n6 /dev/nvme0n8
2018-05-14 14:28:12,612 remote           L0197 DEBUG| [10.142.0.4] [stdout] mdadm: Defaulting to version 1.2 metadata
2018-05-14 14:28:12,669 remote           L0197 DEBUG| [10.142.0.4] [stdout] mdadm: RUN_ARRAY failed: Invalid argument

@harelassayag harelassayag changed the title from scylla_setup should fail with error when mdadm fails to scylla_setup should fail with error when mdadm fails in Silent mode May 16, 2018

avikivity added a commit that referenced this issue Jul 25, 2018

dist/common/scripts/scylla_setup: abort running script when one of se…
…tup failed in silent mode

Current script silently continues even one of setup fails, need to
abort.

Fixes #3433

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180522180355.1648-1-syuu@scylladb.com>
(cherry picked from commit 300af65)

avikivity added a commit that referenced this issue Jul 25, 2018

dist/common/scripts/scylla_setup: abort running script when one of se…
…tup failed in silent mode

Current script silently continues even one of setup fails, need to
abort.

Fixes #3433

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180522180355.1648-1-syuu@scylladb.com>
(cherry picked from commit 300af65)

avikivity added a commit that referenced this issue Jul 25, 2018

dist/common/scripts/scylla_setup: abort running script when one of se…
…tup failed in silent mode

Current script silently continues even one of setup fails, need to
abort.

Fixes #3433

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180522180355.1648-1-syuu@scylladb.com>
(cherry picked from commit 300af65)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment