New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/bluestore: implement collect_metadata #14115

Merged
merged 14 commits into from Apr 6, 2017

Conversation

Projects
None yet
2 participants
@liewegas
Member

liewegas commented Mar 24, 2017

Here's what I get when running bluestore on /dev/sdb:

    "bluefs": "1",
    "bluefs_db_access_mode": "file",
    "bluefs_db_block_size": "4096",
    "bluefs_db_driver": "KernelDevice",
    "bluefs_db_path": "\/home\/sage\/src\/ceph\/build\/dev\/osd0\/block.db",
    "bluefs_db_rotational": "0",
    "bluefs_db_size": "67108864",
    "bluefs_db_type": "ssd",
    "bluefs_single_shared_device": "0",
    "bluefs_slow_access_mode": "blk",
    "bluefs_slow_block_size": "4096",
    "bluefs_slow_dev": "8:16",
    "bluefs_slow_dev_node": "sdb",
    "bluefs_slow_driver": "KernelDevice",
    "bluefs_slow_model": "WDC WD6001FZWX-0",
    "bluefs_slow_partition_path": "\/dev\/sdb",
    "bluefs_slow_rotational": "1",
    "bluefs_slow_size": "6001175126016",
    "bluefs_slow_type": "hdd",
    "bluefs_wal_access_mode": "file",
    "bluefs_wal_block_size": "4096",
    "bluefs_wal_driver": "KernelDevice",
    "bluefs_wal_path": "\/home\/sage\/src\/ceph\/build\/dev\/osd0\/block.wal",
    "bluefs_wal_rotational": "0",
    "bluefs_wal_size": "1048576000",
    "bluefs_wal_type": "ssd",
    "bluestore_bdev_access_mode": "blk",
    "bluestore_bdev_block_size": "4096",
    "bluestore_bdev_dev": "8:16",
    "bluestore_bdev_dev_node": "sdb",
    "bluestore_bdev_driver": "KernelDevice",
    "bluestore_bdev_model": "WDC WD6001FZWX-0",
    "bluestore_bdev_partition_path": "\/dev\/sdb",
    "bluestore_bdev_rotational": "1",
    "bluestore_bdev_size": "6001175126016",
    "bluestore_bdev_type": "hdd",

and on a file backed by nvme:

    "bluefs": "1",
    "bluefs_db_access_mode": "file",
    "bluefs_db_block_size": "4096",
    "bluefs_db_driver": "KernelDevice",
    "bluefs_db_path": "\/home\/sage\/src\/ceph\/build\/dev\/osd0\/block.db",
    "bluefs_db_rotational": "0",
    "bluefs_db_size": "67108864",
    "bluefs_db_type": "ssd",
    "bluefs_single_shared_device": "0",
    "bluefs_slow_access_mode": "file",
    "bluefs_slow_block_size": "4096",
    "bluefs_slow_driver": "KernelDevice",
    "bluefs_slow_path": "\/home\/sage\/src\/ceph\/build\/dev\/osd0\/block",
    "bluefs_slow_rotational": "0",
    "bluefs_slow_size": "10737418240",
    "bluefs_slow_type": "ssd",
    "bluefs_wal_access_mode": "file",
    "bluefs_wal_block_size": "4096",
    "bluefs_wal_driver": "KernelDevice",
    "bluefs_wal_path": "\/home\/sage\/src\/ceph\/build\/dev\/osd0\/block.wal",
    "bluefs_wal_rotational": "0",
    "bluefs_wal_size": "1048576000",
    "bluefs_wal_type": "ssd",
    "bluestore_bdev_access_mode": "file",
    "bluestore_bdev_block_size": "4096",
    "bluestore_bdev_driver": "KernelDevice",
    "bluestore_bdev_path": "\/home\/sage\/src\/ceph\/build\/dev\/osd0\/block",
    "bluestore_bdev_rotational": "0",
    "bluestore_bdev_size": "10737418240",
    "bluestore_bdev_type": "ssd",

and directly on an nvme:

        "bluefs": "1",
        "bluefs_db_access_mode": "file",
        "bluefs_db_block_size": "4096",
        "bluefs_db_driver": "KernelDevice",
        "bluefs_db_path": "\/home\/sage\/src\/ceph\/build\/dev\/osd0\/block.db",
        "bluefs_db_rotational": "0",
        "bluefs_db_size": "67108864",
        "bluefs_db_type": "ssd",
        "bluefs_single_shared_device": "0",
        "bluefs_slow_access_mode": "blk",
        "bluefs_slow_block_size": "4096",
        "bluefs_slow_dev": "259:0",
        "bluefs_slow_dev_node": "nvme1n1",
        "bluefs_slow_driver": "KernelDevice",
        "bluefs_slow_model": "INTEL SSDPE2ME400G4                     ",
        "bluefs_slow_partition_path": "\/dev\/nvme1n1",
        "bluefs_slow_rotational": "0",
        "bluefs_slow_serial": "CVMD54420001400FGN  ",
        "bluefs_slow_size": "400088457216",
        "bluefs_slow_type": "nvme",
        "bluefs_wal_access_mode": "file",
        "bluefs_wal_block_size": "4096",
        "bluefs_wal_driver": "KernelDevice",
        "bluefs_wal_path": "\/home\/sage\/src\/ceph\/build\/dev\/osd0\/block.wal",
        "bluefs_wal_rotational": "0",
        "bluefs_wal_size": "1048576000",
        "bluefs_wal_type": "ssd",
        "bluestore_bdev_access_mode": "blk",
        "bluestore_bdev_block_size": "4096",
        "bluestore_bdev_dev": "259:0",
        "bluestore_bdev_dev_node": "nvme1n1",
        "bluestore_bdev_driver": "KernelDevice",
        "bluestore_bdev_model": "INTEL SSDPE2ME400G4                     ",
        "bluestore_bdev_partition_path": "\/dev\/nvme1n1",
        "bluestore_bdev_rotational": "0",
        "bluestore_bdev_serial": "CVMD54420001400FGN  ",
        "bluestore_bdev_size": "400088457216",
        "bluestore_bdev_type": "nvme",

@liewegas liewegas requested review from dmick and markhpc and removed request for dmick Mar 24, 2017

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Mar 27, 2017

Member

retest this please

Member

liewegas commented Mar 27, 2017

retest this please

Show outdated Hide outdated src/common/blkdev.cc
if (r < 0) {
return -EINVAL; // hrm.
}
char *t = blkid_devno_to_devname(st.st_rdev);

This comment has been minimized.

@dmick

dmick Mar 28, 2017

Member

someone needs to free t

@dmick

dmick Mar 28, 2017

Member

someone needs to free t

TEST(blkdev, device_model)
{
const char* env = getenv("CEPH_ROOT");

This comment has been minimized.

@dmick

dmick Mar 28, 2017

Member

Can't these four lines be in a SetUp() function somewhere?

@dmick

dmick Mar 28, 2017

Member

Can't these four lines be in a SetUp() function somewhere?

{
char val[1024] = {0};
get_block_device_string_property(dev, property, val, sizeof(val));
return val;

This comment has been minimized.

@dmick

dmick Mar 28, 2017

Member

doesn't val go out of scope on return?

@dmick

dmick Mar 28, 2017

Member

doesn't val go out of scope on return?

This comment has been minimized.

@liewegas

liewegas Mar 28, 2017

Member

i'll make the conversion to std::string explicit so that it's clear it happens first. (I think it does on its... but not really sure actually!)

@liewegas

liewegas Mar 28, 2017

Member

i'll make the conversion to std::string explicit so that it's clear it happens first. (I think it does on its... but not really sure actually!)

This comment has been minimized.

@dmick

dmick Mar 28, 2017

Member

Oh, because promotion of return type. Hm. Yeah, I dunno either

@dmick

dmick Mar 28, 2017

Member

Oh, because promotion of return type. Hm. Yeah, I dunno either

This comment has been minimized.

Show outdated Hide outdated src/os/bluestore/KernelDevice.cc
(*pm)[prefix + "serial"] = serial;
}
// nvme has a driver/driver/* structure; infer from that. there

This comment has been minimized.

@dmick

dmick Mar 28, 2017

Member

presumably you meant device/device/*

@dmick

dmick Mar 28, 2017

Member

presumably you meant device/device/*

This comment has been minimized.

@liewegas

liewegas Mar 28, 2017

Member

eyah

@liewegas
@@ -865,6 +865,7 @@ int NVMEDevice::open(const string& p)
driver->register_device(this);
block_size = driver->get_block_size();
size = driver->get_size();
name = serial_number;

This comment has been minimized.

@dmick

dmick Mar 28, 2017

Member

not, like, path?

and so I'm confused by this; looks like spdk sets it to the result of spdk_pci_device_get_device_name, but
spdk/spdk@dca887b. Granted, it's only used in messages apparently, but, it seems like it ought to be useful as an identifier for the device to humans. What's the intent?

@dmick

dmick Mar 28, 2017

Member

not, like, path?

and so I'm confused by this; looks like spdk sets it to the result of spdk_pci_device_get_device_name, but
spdk/spdk@dca887b. Granted, it's only used in messages apparently, but, it seems like it ought to be useful as an identifier for the device to humans. What's the intent?

This comment has been minimized.

@liewegas

liewegas Mar 28, 2017

Member

huh, yeah dunno about that spdk change. in any case, though, this is how we're identifying the device. @yuyuyu101 is there something better to do here?

@liewegas

liewegas Mar 28, 2017

Member

huh, yeah dunno about that spdk change. in any case, though, this is how we're identifying the device. @yuyuyu101 is there something better to do here?

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Mar 28, 2017

Member

Speaking of 'path', what I really wanted was the string that is used for /dev/disk/by-path which shows how the device is connected across the various buses and controllers in the system, but it wasn't obvious to me where that can be found in /sys/block. Maybe it's something simple referenced by the block udev rule?

Member

liewegas commented Mar 28, 2017

Speaking of 'path', what I really wanted was the string that is used for /dev/disk/by-path which shows how the device is connected across the various buses and controllers in the system, but it wasn't obvious to me where that can be found in /sys/block. Maybe it's something simple referenced by the block udev rule?

@dmick

This comment has been minimized.

Show comment
Hide comment
@dmick

dmick Mar 28, 2017

Member

Maybe you have to find the end thru /sys/block and then search /dev/disk for symlinks that point there? I have often wanted a full graph of both namespaces...

Member

dmick commented Mar 28, 2017

Maybe you have to find the end thru /sys/block and then search /dev/disk for symlinks that point there? I have often wanted a full graph of both namespaces...

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Mar 29, 2017

Member

retest this please

Member

liewegas commented Mar 29, 2017

retest this please

liewegas added some commits Mar 24, 2017

common/blkdev: implement get_device_by_fd
Complement to get_device_by_uuid, but we provide an fd.

Signed-off-by: Sage Weil <sage@redhat.com>
common/blkdev: take queue/ prefix as part of property name
This way we can get at properties in device/ too.

Signed-off-by: Sage Weil <sage@redhat.com>
common/blkdev: add get string property
Signed-off-by: Sage Weil <sage@redhat.com>
common/blkdev: add block_device_model
Signed-off-by: Sage Weil <sage@redhat.com>
common/blkdev: pass devname to block_device_*
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore/BlockDevice: collect metadata
Implement for KernelDevice and NVMEDevice

Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: implement collect_metadata for BlueStore, BlueFS
Signed-off-by: Sage Weil <sage@redhat.com>
unittest_blkdev: more tests!
Signed-off-by: Sage Weil <sage@redhat.com>
common/blkdev: clean up header a bit
Signed-off-by: Sage Weil <sage@redhat.com>
common/blkdev: take fd for device node or regular file
Either way, return the partition and device name for
the underlying device.

Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore/KernelDevice: set rotational flag for files too
If we're backed by a file we can still tell whether the
underlying device is rotational or not.

Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore/KernelDevice: fix uninit value
Signed-off-by: Sage Weil <sage@redhat.com>
common/blkdev: pass size_t to get_device_by_fd
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore/KernelDevice: assume rotational if we can't get devname
Signed-off-by: Sage Weil <sage@redhat.com>
@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Apr 5, 2017

Member

retest this please

Member

liewegas commented Apr 5, 2017

retest this please

@liewegas liewegas merged commit bb6bef0 into ceph:master Apr 6, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment