osd/objectstore: change osd_objectstore default to bluestore #24642

vumrao · 2018-10-17T17:56:02Z

osd/objectstore: change osd_objectstore default to bluestore
Fixes: http://tracker.ceph.com/issues/36494

Signed-off-by: Vikhyat Umrao vumrao@redhat.com

tchaikov

this breaks some tests inspecting the underlying files/directories of filestore for verifying its behavior. for instance, http://pulpito.ceph.com/kchai-2018-10-22_01:58:26-rados-wip-kefu-testing-2018-10-20-1204-distro-basic-smithi/3170191/

tchaikov

this breaks some tests inspecting the underlying files/directories of filestore for verifying its behavior. for instance, http://pulpito.ceph.com/kchai-2018-10-22_01:58:26-rados-wip-kefu-testing-2018-10-20-1204-distro-basic-smithi/3170191/

vumrao · 2018-10-23T22:28:34Z

@tchaikov thank you.

We need to fix this test:

File - qa/standalone/mon/osd-pool-create.sh

function TEST_pool_create_rep_expected_num_objects() {
    local dir=$1
    setup $dir || return 1

    # disable pg dir merge
    export CEPH_ARGS
    run_mon $dir a || return 1
    run_mgr $dir x || return 1
    run_osd $dir 0 || return 1

    ceph osd pool create rep_expected_num_objects 64 64 replicated  replicated_rule 100000 || return 1
    # wait for pg dir creating
    sleep 30
    ceph pg ls
    find ${dir}/0/current -ls
    ret=$(find ${dir}/0/current/1.0_head/ | grep DIR | wc -l)
    if [ "$ret" -le 2 ];
    then
        return 1
    else
        echo "TEST_pool_create_rep_expected_num_objects PASS"
    fi
}

Because this is where it is failing:

2018-10-22T03:01:39.014 INFO:tasks.workunit.client.0.smithi142.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/osd-pool-create.sh:225: TEST_pool_create_rep_expected_num_objects:  find td/osd-pool-create/0/current -ls
2018-10-22T03:01:39.016 INFO:tasks.workunit.client.0.smithi142.stderr:find: 'td/osd-pool-create/0/current': No such file or directory

2018-10-22T03:01:39.017 INFO:tasks.workunit.client.0.smithi142.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/osd-pool-create.sh:226: TEST_pool_create_rep_expected_num_objects:  find td/osd-pool-create/0/current/1.0_head/
2018-10-22T03:01:39.018 INFO:tasks.workunit.client.0.smithi142.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/osd-pool-create.sh:226: TEST_pool_create_rep_expected_num_objects:  grep DIR
2018-10-22T03:01:39.018 INFO:tasks.workunit.client.0.smithi142.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/osd-pool-create.sh:226: TEST_pool_create_rep_expected_num_objects:  wc -l
2018-10-22T03:01:39.019 INFO:tasks.workunit.client.0.smithi142.stderr:find: 'td/osd-pool-create/0/current/1.0_head/': No such file or directory

2018-10-22T03:01:39.021 INFO:tasks.workunit.client.0.smithi142.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/osd-pool-create.sh:226: TEST_pool_create_rep_expected_num_objects:  ret=0
2018-10-22T03:01:39.022 INFO:tasks.workunit.client.0.smithi142.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/osd-pool-create.sh:227: TEST_pool_create_rep_expected_num_objects:  '[' 0 -le 2 ']'

2018-10-22T03:01:39.022 INFO:tasks.workunit.client.0.smithi142.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/osd-pool-create.sh:229: TEST_pool_create_rep_expected_num_objects:  return 1

I think we should be running this test only when the objectstore is filestore.

liewegas · 2018-10-23T22:32:07Z

The fix is to change this

export CEPH_ARGS

to export CEPH_ARGS="--osd-objectstore filestore" for this or any similar test that needs filestore.

vumrao · 2018-10-23T22:36:18Z

Thank you @liewegas will make the changes.

tchaikov · 2018-10-24T02:39:37Z

@vumrao there are more of them, could you fix following tests as well?

$ grep -w '"current"' qa -r 2>/dev/null
qa/btrfs/test_rmdir_async_snap.c:	strcpy(vold.name, "current");
qa/btrfs/test_rmdir_async_snap.c:	va.fd = open("current", O_RDONLY);
qa/tasks/mgr/test_insights.py:            if check not in report["health"]["current"]["checks"]:
qa/tasks/mgr/test_insights.py:            if "MGR_INSIGHTS_WARNING" in report["health"]["current"]["checks"]:
qa/standalone/special/ceph_objectstore_tool.py:    DIR = os.path.join(SUBDIR, "current")
qa/standalone/special/ceph_objectstore_tool.py:        SUBDIR = os.path.join(DIRL2, "current")
qa/standalone/special/ceph_objectstore_tool.py:        SUBDIR = os.path.join(DIRL2, "current")
qa/standalone/special/ceph_objectstore_tool.py:                    DIR = os.path.join(OSDDIR, os.path.join(osd, os.path.join("current", "{pg}_head".format(pg=pg))))
qa/standalone/special/ceph_objectstore_tool.py:                    DIR = os.path.join(OSDDIR, os.path.join(osd, os.path.join("current", "{pg}_head".format(pg=pg))))
qa/standalone/special/ceph_objectstore_tool.py:                    DIR = os.path.join(OSDDIR, os.path.join(osd, os.path.join("current", "{pg}_head".format(pg=pg))))
qa/standalone/special/ceph_objectstore_tool.py:                    DIR = os.path.join(OSDDIR, os.path.join(osd, os.path.join("current", "{pg}_head".format(pg=pg))))
qa/standalone/special/ceph_objectstore_tool.py:                    DIR = os.path.join(OSDDIR, os.path.join(osd, os.path.join("current", "{pg}_head".format(pg=pg))))

$ grep '/current' qa -r 2>/dev/null
qa/workunits/rados/test_alloc_hint.sh:        local fns=$(sudo sh -c "ls ${OSD_DATA[i]}/current/${PGID}*_head/${OBJ}_*")
qa/standalone/mon/osd-pool-create.sh:    find ${dir}/0/current -ls
qa/standalone/mon/osd-pool-create.sh:    ret=$(find ${dir}/0/current/1.0_head/ | grep DIR | wc -l)

vumrao · 2018-10-24T23:32:49Z

Thank you, @tchaikov I have updated ceph_objectstore_tool and test_alloc_hint tests as they are related tofilestore. Looks like Mgr one is not related and btrfs one I am not sure please let me know if btrfs also needs to be updated.

The osd-pool-create one I had already covered in yesterday commit.

smithfarm · 2018-11-27T18:13:49Z

@liewegas OK to backport this to mimic and luminous?

liewegas · 2018-11-27T19:09:29Z

@smithfarm sure!

jdurgin

Test updates look good, let's run this through a suite

tchaikov · 2018-11-29T05:28:13Z

qa/workunits/rados/test_alloc_hint.sh

@@ -51,6 +51,7 @@ function setup_pgid() {
 }

 function expect_alloc_hint_eq() {
+    export CEPH_ARGS="--osd-objectstore=filestore"


this does not change how osd is initialized. you will need to update theoverrides section in qa/suites/rados/objectstore/backends/alloc-hint.yaml to specify the objectstore type.

and you need to audit other places you set --osd-objectstore=filestore to make sure it can actually take effect before the OSD/cluster is created.

2018-11-29T02:58:03.956 INFO:tasks.workunit.client.0.smithi083.stderr:+ sudo ceph daemon osd.0 flush_journal 2018-11-29T02:58:04.143 INFO:tasks.workunit.client.0.smithi083.stderr:++ sudo sh -c 'ls /var/lib/ceph/osd/ceph-0/current/2.6*_head/foo_*' 2018-11-29T02:58:04.158 INFO:tasks.workunit.client.0.smithi083.stderr:ls: cannot access /var/lib/ceph/osd/ceph-0/current/2.6*_head/foo_*: No such file or directory

see http://pulpito.ceph.com/kchai-2018-11-29_01:22:06-rados-wip-kefu2-testing-2018-11-29-0053-distro-basic-smithi/3289928/

tchaikov · 2018-11-30T02:54:35Z

@vumrao could you also enlarge the size of OSD, see @ifed01 's analysis at http://tracker.ceph.com/issues/37452 ?

but i wonder why we haven't run into such an issue if filestore is used. does this imply that bluestore requires more device space compared to filestore? is this expected?

ifed01 · 2018-11-30T07:56:34Z

@tchaikov - I think there is no explicit limit for filestore for these test cases - they simply are able to use all the available space as they run on top of regular file system.
Contrary bluestore has such a limit as it requires predefined (i.e. allocated) slow/DB/WAL volumes.

tchaikov · 2018-12-11T07:57:53Z

@vumrao ping?

vumrao · 2018-12-11T23:07:23Z

@tchaikov sorry for the delay I will take a look tomorrow.

Fixes: http://tracker.ceph.com/issues/36494 Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

for only osd_objectstore as filestore. Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

vumrao · 2018-12-13T22:56:24Z

@tchaikov I have tried to fix it hope this time it will go through :)

vumrao · 2019-01-03T19:36:53Z

@tchaikov Hi Kefu - can you please add this one to your next test branch. Thank you.

liewegas · 2019-01-13T22:08:03Z

see #25937 to address additional rados suite failures.

liewegas · 2019-01-16T19:10:07Z

replaced by #25937

batrick added core needs-review labels Oct 17, 2018

neha-ojha self-requested a review October 17, 2018 21:26

liewegas approved these changes Oct 17, 2018

View reviewed changes

liewegas added needs-qa and removed needs-review labels Oct 17, 2018

neha-ojha approved these changes Oct 17, 2018

View reviewed changes

tchaikov added the wip-kefu-testing label Oct 19, 2018

tchaikov requested changes Oct 22, 2018

View reviewed changes

tchaikov removed needs-qa wip-kefu-testing labels Oct 22, 2018

tchaikov requested changes Oct 22, 2018

View reviewed changes

vumrao force-pushed the wip-vumrao-36494 branch from d9ed42f to 529e2f5 Compare October 23, 2018 22:53

vumrao force-pushed the wip-vumrao-36494 branch from 529e2f5 to e6b6954 Compare October 24, 2018 23:22

cbodley mentioned this pull request Nov 27, 2018

options: set default osd_objectstore to bluestore #25288

Closed

jdurgin added the needs-qa label Nov 27, 2018

jdurgin approved these changes Nov 27, 2018

View reviewed changes

tchaikov added the wip-kefu2-testing label Nov 28, 2018

tchaikov requested changes Nov 29, 2018

View reviewed changes

tchaikov removed needs-qa wip-kefu2-testing labels Nov 29, 2018

vumrao added 3 commits December 13, 2018 14:26

osd/objectstore: change osd_objectstore default to bluestore

4c19d8c

Fixes: http://tracker.ceph.com/issues/36494 Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

test: Run pool_create_rep_expected_num_objects only with filestore

da71558

Fixes: http://tracker.ceph.com/issues/36494 Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

test: update ceph-objectstore-tool and test_alloc_hint tests

113221c

for only osd_objectstore as filestore. Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

vumrao force-pushed the wip-vumrao-36494 branch from e6b6954 to 113221c Compare December 13, 2018 22:55

neha-ojha added the needs-qa label Jan 3, 2019

liewegas added the wip-sage4-testing label Jan 4, 2019

liewegas removed wip-sage4-testing labels Jan 13, 2019

liewegas mentioned this pull request Jan 13, 2019

osd: default osd_objectstore=bluestore #25937

Merged

liewegas added the wip-sage-testing label Jan 15, 2019

liewegas closed this Jan 16, 2019

osd/objectstore: change osd_objectstore default to bluestore #24642

osd/objectstore: change osd_objectstore default to bluestore #24642

vumrao commented Oct 17, 2018

tchaikov left a comment

tchaikov left a comment •

edited

vumrao commented Oct 23, 2018

liewegas commented Oct 23, 2018 via email

vumrao commented Oct 23, 2018

tchaikov commented Oct 24, 2018

vumrao commented Oct 24, 2018 •

edited

smithfarm commented Nov 27, 2018

liewegas commented Nov 27, 2018

jdurgin left a comment

tchaikov Nov 29, 2018 •

edited

tchaikov Nov 29, 2018

tchaikov commented Nov 30, 2018 •

edited

ifed01 commented Nov 30, 2018

tchaikov commented Dec 11, 2018

vumrao commented Dec 11, 2018

vumrao commented Dec 13, 2018

vumrao commented Jan 3, 2019

liewegas commented Jan 13, 2019

liewegas commented Jan 16, 2019

osd/objectstore: change osd_objectstore default to bluestore #24642

osd/objectstore: change osd_objectstore default to bluestore #24642

Conversation

vumrao commented Oct 17, 2018

tchaikov left a comment

Choose a reason for hiding this comment

tchaikov left a comment • edited

Choose a reason for hiding this comment

vumrao commented Oct 23, 2018

liewegas commented Oct 23, 2018 via email

vumrao commented Oct 23, 2018

tchaikov commented Oct 24, 2018

vumrao commented Oct 24, 2018 • edited

smithfarm commented Nov 27, 2018

liewegas commented Nov 27, 2018

jdurgin left a comment

Choose a reason for hiding this comment

tchaikov Nov 29, 2018 • edited

Choose a reason for hiding this comment

tchaikov Nov 29, 2018

Choose a reason for hiding this comment

tchaikov commented Nov 30, 2018 • edited

ifed01 commented Nov 30, 2018

tchaikov commented Dec 11, 2018

vumrao commented Dec 11, 2018

vumrao commented Dec 13, 2018

vumrao commented Jan 3, 2019

liewegas commented Jan 13, 2019

liewegas commented Jan 16, 2019

tchaikov left a comment •

edited

vumrao commented Oct 24, 2018 •

edited

tchaikov Nov 29, 2018 •

edited

tchaikov commented Nov 30, 2018 •

edited