Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ceph-volume: allow symlinks as devices #46893

Merged
merged 2 commits into from Jul 28, 2022
Merged

ceph-volume: allow symlinks as devices #46893

merged 2 commits into from Jul 28, 2022

Conversation

guits
Copy link
Contributor

@guits guits commented Jun 29, 2022 •
edited

This MR creates ability to use symlinks as devices for ceph-volume.
One example of use will include usage of custom udev rules to map disk slots in enclosures to block device labels like:

/dev/data1 -> /dev/sda
/dev/data2 -> /dev/sdb

Fixes: https://tracker.ceph.com/issues/49103

Signed-off-by: Jan Sobczak jsobczak@cloudferro.com
Co-authored-by: Guillaume Abrioux gabrioux@redhat.com

@guits guits requested a review from a team as a code owner June 29, 2022 11:47
@guits
Copy link
Contributor Author

guits commented Jun 29, 2022

@jansobczak FYI

@guits guits force-pushed the ceph-volume-symlinks-guits branch from da40140 to a2ca7c7 Compare June 29, 2022 11:52
@guits guits mentioned this pull request Jun 29, 2022
Closed
@guits guits force-pushed the ceph-volume-symlinks-guits branch 4 times, most recently from edebd9e to 58f7d4d Compare June 29, 2022 14:14
@guits guits closed this Jun 29, 2022
@guits guits reopened this Jun 29, 2022
@guits
Copy link
Contributor Author

guits commented Jun 29, 2022

jenkins test ceph-volume tox

@guits
Copy link
Contributor Author

guits commented Jul 1, 2022

jenkins test make check

@github-actions
Copy link

github-actions bot commented Jul 5, 2022

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@guits guits force-pushed the ceph-volume-symlinks-guits branch from 58f7d4d to 6683dc0 Compare July 5, 2022 13:42
@guits
Copy link
Contributor Author

guits commented Jul 5, 2022

jenkins test ceph-volume tox

@guits guits force-pushed the ceph-volume-symlinks-guits branch from 6683dc0 to dd255f2 Compare July 6, 2022 13:03
@guits
Copy link
Contributor Author

guits commented Jul 6, 2022

jenkins test ceph-volume tox

@guits guits force-pushed the ceph-volume-symlinks-guits branch from dd255f2 to c976511 Compare July 6, 2022 13:09
@guits
Copy link
Contributor Author

guits commented Jul 7, 2022

jenkins test make check arm64

@adk3798
Copy link
Contributor

adk3798 commented Jul 7, 2022

https://pulpito.ceph.com/adking-2022-07-07_11:19:22-orch:cephadm-wip-adk-testing-2022-07-06-2326-distro-default-smithi/

8 failures, all seem to be general teuthology issues.

Reran all the failed tests twice and they all passed
https://pulpito.ceph.com/adking-2022-07-07_13:17:44-orch:cephadm-wip-adk-testing-2022-07-06-2326-distro-default-smithi/

@guits guits force-pushed the ceph-volume-symlinks-guits branch from c976511 to 0b9c740 Compare July 8, 2022 13:12
@guits
Copy link
Contributor Author

guits commented Jul 8, 2022

jenkins test ceph-volume tox

@guits
Copy link
Contributor Author

guits commented Jul 8, 2022 

# ceph-volume lvm batch --prepare --yes --bluestore /dev/disk/by-id/ata-QEMU_HARDDISK_QM00001 --no-systemd --report
--> DEPRECATION NOTICE
--> You are using the legacy automatic disk sorting behavior
--> The Pacific release will change the default to --no-auto
--> passed data devices: 1 physical, 0 LVM
--> relative data size: 1.0

Total OSDs: 1

  Type            Path                                                    LV Size         % of device
----------------------------------------------------------------------------------------------------
  data            /dev/disk/by-id/ata-QEMU_HARDDISK_QM00001 -> /dev/sda   200.00 GB       100.00%
#

# ceph-volume lvm batch --prepare --yes --bluestore /dev/disk/by-id/ata-QEMU_HARDDISK_QM00001 --no-systemd
--> DEPRECATION NOTICE
--> You are using the legacy automatic disk sorting behavior
--> The Pacific release will change the default to --no-auto
--> passed data devices: 1 physical, 0 LVM
--> relative data size: 1.0
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 43b8e67f-ae53-4c91-90c1-58f61a4a72a9
Running command: nsenter --mount=/rootfs/proc/1/ns/mnt --ipc=/rootfs/proc/1/ns/ipc --net=/rootfs/proc/1/ns/net --uts=/rootfs/proc/1/ns/uts /sbin/vgcreate --force --yes ceph-2d6d7e16-c4e9-434e-8919-fceb2ba1cb71 /dev/sda
 stdout: Physical volume "/dev/sda" successfully created.
 stdout: Volume group "ceph-2d6d7e16-c4e9-434e-8919-fceb2ba1cb71" successfully created
Running command: nsenter --mount=/rootfs/proc/1/ns/mnt --ipc=/rootfs/proc/1/ns/ipc --net=/rootfs/proc/1/ns/net --uts=/rootfs/proc/1/ns/uts /sbin/lvcreate --yes -l 51199 -n osd-block-43b8e67f-ae53-4c91-90c1-58f61a4a72a9 ceph-2d6d7e16-c4e9-434e-8919-fceb2ba1cb71
 stdout: Logical volume "osd-block-43b8e67f-ae53-4c91-90c1-58f61a4a72a9" created.
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-1
Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-2d6d7e16-c4e9-434e-8919-fceb2ba1cb71/osd-block-43b8e67f-ae53-4c91-90c1-58f61a4a72a9
Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0
Running command: /usr/bin/ln -s /dev/ceph-2d6d7e16-c4e9-434e-8919-fceb2ba1cb71/osd-block-43b8e67f-ae53-4c91-90c1-58f61a4a72a9 /var/lib/ceph/osd/ceph-1/block
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-1/activate.monmap
 stderr: got monmap epoch 1
--> Creating keyring file for osd.1
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/keyring
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/
Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid 43b8e67f-ae53-4c91-90c1-58f61a4a72a9 --setuser ceph --setgroup ceph
 stderr: 2022-07-08T13:15:14.486+0000 7fb50196e440 -1 bluestore(/var/lib/ceph/osd/ceph-1/) _read_fsid unparsable uuid
--> ceph-volume lvm prepare successful for: /dev/sda
#

# ceph-volume lvm prepare --bluestore --data /dev/disk/by-id/ata-QEMU_HARDDISK_QM00002 --no-systemd
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 1161be04-fb4b-417c-9189-024a27fe1ba2
Running command: nsenter --mount=/rootfs/proc/1/ns/mnt --ipc=/rootfs/proc/1/ns/ipc --net=/rootfs/proc/1/ns/net --uts=/rootfs/proc/1/ns/uts /sbin/vgcreate --force --yes ceph-400c2d3f-4db9-4014-b7aa-865cc221cffc /dev/sdb
 stdout: Physical volume "/dev/sdb" successfully created.
 stdout: Volume group "ceph-400c2d3f-4db9-4014-b7aa-865cc221cffc" successfully created
Running command: nsenter --mount=/rootfs/proc/1/ns/mnt --ipc=/rootfs/proc/1/ns/ipc --net=/rootfs/proc/1/ns/net --uts=/rootfs/proc/1/ns/uts /sbin/lvcreate --yes -l 51199 -n osd-block-1161be04-fb4b-417c-9189-024a27fe1ba2 ceph-400c2d3f-4db9-4014-b7aa-865cc221cffc
 stdout: Logical volume "osd-block-1161be04-fb4b-417c-9189-024a27fe1ba2" created.
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-2
Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-400c2d3f-4db9-4014-b7aa-865cc221cffc/osd-block-1161be04-fb4b-417c-9189-024a27fe1ba2
Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1
Running command: /usr/bin/ln -s /dev/ceph-400c2d3f-4db9-4014-b7aa-865cc221cffc/osd-block-1161be04-fb4b-417c-9189-024a27fe1ba2 /var/lib/ceph/osd/ceph-2/block
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-2/activate.monmap
 stderr: got monmap epoch 1
--> Creating keyring file for osd.2
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2/keyring
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2/
Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 2 --monmap /var/lib/ceph/osd/ceph-2/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-2/ --osd-uuid 1161be04-fb4b-417c-9189-024a27fe1ba2 --setuser ceph --setgroup ceph
 stderr: 2022-07-08T13:16:51.572+0000 7fb5df509440 -1 bluestore(/var/lib/ceph/osd/ceph-2/) _read_fsid unparsable uuid
--> ceph-volume lvm prepare successful for: /dev/sdb
#

@guits guits force-pushed the ceph-volume-symlinks-guits branch from 0b9c740 to dc05012 Compare July 8, 2022 13:26
@adk3798
Copy link
Contributor

adk3798 commented Jul 14, 2022

https://pulpito.ceph.com/adking-2022-07-13_02:33:30-orch:cephadm-wip-adk-testing-2022-07-12-2037-distro-default-smithi/

11 Failures:

  • 10 failed RHEL tests due to issues with satellite in sepia
  • 1 failure to create osd with ObjectStore::mkfs failed with error (5) Input/output error This failure has been popping up occasionally and is nothing new, most likely an issue with either infra or the way we split up the nvme for osds in our tests

Overall, despite the raw number of failures, all are known things and unrelated to PRs in the run

guits and others added 2 commits July 27, 2022 16:41
@guits
ceph-volume: drop self.abspath in Device()
01112ba 
seems useless to have both self.path and self.abspath attributes.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
@guits
ceph-volume: support symlinks as devices
9ddc5b7 
This makes ceph-volume support passing symlinks as devices.

Fixes: https://tracker.ceph.com/issues/49103

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
@guits guits force-pushed the ceph-volume-symlinks-guits branch from dc05012 to 9ddc5b7 Compare July 27, 2022 14:49
@ceph ceph deleted a comment from github-actions bot Jul 27, 2022
@guits
Copy link
Contributor Author

guits commented Jul 27, 2022

jenkins test ceph-volume tox

@guits guits merged commit 3bf846a into ceph:main Jul 28, 2022
@guits guits deleted the ceph-volume-symlinks-guits branch July 28, 2022 18:11
This was referenced Jul 29, 2022
Closed
Closed
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jansobczak jansobczak jansobczak left review comments

@adk3798 adk3798 adk3798 approved these changes

Assignees
No one assigned
Labels
ceph-volume
Projects
None yet
Milestone
No milestone
3 participants
@guits @adk3798 @jansobczak