New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/bluestore: reduce some overhead for _do_clone_range() and _do_remove() #15944

Merged
merged 3 commits into from Jul 7, 2017

Conversation

Projects
None yet
2 participants
@xiexingguo
Member

xiexingguo commented Jun 27, 2017

No description provided.

@xiexingguo xiexingguo requested a review from liewegas Jun 27, 2017

Show outdated Hide outdated src/os/bluestore/BlueStore.cc
Show outdated Hide outdated src/os/bluestore/BlueStore.cc
@xiexingguo

This comment has been minimized.

Show comment
Hide comment
@xiexingguo
Member

xiexingguo commented Jun 30, 2017

@xiexingguo xiexingguo changed the title from os/bluestore: reduce some overhead for _do_clone_range() and _do_remove() to DNM: os/bluestore: reduce some overhead for _do_clone_range() and _do_remove() Jun 30, 2017

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Jun 30, 2017

Member

/a/sage-2017-06-30_05:44:03-rados-wip-sage-testing-distro-basic-smithi/1344955 got

2017-06-30T06:18:58.257 INFO:tasks.ceph.osd.1.smithi112.stderr:/build/ceph-12.0.3-2457-g4f25a7f/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::ExtentMap::dirty_range(uint32_t, uint32_t)' thread 7f7b36fd6700 time 2017-06-30 06:18:58.260362
2017-06-30T06:18:58.257 INFO:tasks.ceph.osd.1.smithi112.stderr:/build/ceph-12.0.3-2457-g4f25a7f/src/os/bluestore/BlueStore.cc: 2666: FAILED assert(0 == "can't mark unloaded shard dirty")
2017-06-30T06:18:58.258 INFO:tasks.ceph.osd.3.smithi176.stderr:2017-06-30 06:18:58.258698 7f4c9ac68700 -1 received  signal: Hangup from  PID: 30778 task name: /usr/bin/python /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3  UID: 0
2017-06-30T06:18:58.264 INFO:tasks.ceph.osd.1.smithi112.stderr: ceph version 12.0.3-2457-g4f25a7f (4f25a7f9ca89a19499f31fe81b7d65f70b1dcd74) luminous (rc)
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x5589807ee272]
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 2: (BlueStore::ExtentMap::dirty_range(unsigned int, unsigned int)+0x54a) [0x558980642bda]
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 3: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr&, boost::intrusive_ptr&, boost::intrusive_ptr&, unsigned long, unsigned long, unsigned long)+0x1298) [0x55898068d448]
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 4: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr&, boost::intrusive_ptr&, boost::intrusive_ptr&, unsigned long, unsigned long, unsigned long)+0x288) [0x5589806a7738]
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 5: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)+0x1fc2) [0x5589806acfa2]
2017-06-30T06:18:58.266 INFO:tasks.ceph.osd.1.smithi112.stderr: 6: (BlueStore::queue_transactions(ObjectStore::Sequencer*, std::vector >&, boost::intrusive_ptr, ThreadPool::TPHandle*)+0x52e) [0x5589806adbce]

need to investigate...

Member

liewegas commented Jun 30, 2017

/a/sage-2017-06-30_05:44:03-rados-wip-sage-testing-distro-basic-smithi/1344955 got

2017-06-30T06:18:58.257 INFO:tasks.ceph.osd.1.smithi112.stderr:/build/ceph-12.0.3-2457-g4f25a7f/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::ExtentMap::dirty_range(uint32_t, uint32_t)' thread 7f7b36fd6700 time 2017-06-30 06:18:58.260362
2017-06-30T06:18:58.257 INFO:tasks.ceph.osd.1.smithi112.stderr:/build/ceph-12.0.3-2457-g4f25a7f/src/os/bluestore/BlueStore.cc: 2666: FAILED assert(0 == "can't mark unloaded shard dirty")
2017-06-30T06:18:58.258 INFO:tasks.ceph.osd.3.smithi176.stderr:2017-06-30 06:18:58.258698 7f4c9ac68700 -1 received  signal: Hangup from  PID: 30778 task name: /usr/bin/python /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3  UID: 0
2017-06-30T06:18:58.264 INFO:tasks.ceph.osd.1.smithi112.stderr: ceph version 12.0.3-2457-g4f25a7f (4f25a7f9ca89a19499f31fe81b7d65f70b1dcd74) luminous (rc)
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x5589807ee272]
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 2: (BlueStore::ExtentMap::dirty_range(unsigned int, unsigned int)+0x54a) [0x558980642bda]
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 3: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr&, boost::intrusive_ptr&, boost::intrusive_ptr&, unsigned long, unsigned long, unsigned long)+0x1298) [0x55898068d448]
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 4: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr&, boost::intrusive_ptr&, boost::intrusive_ptr&, unsigned long, unsigned long, unsigned long)+0x288) [0x5589806a7738]
2017-06-30T06:18:58.265 INFO:tasks.ceph.osd.1.smithi112.stderr: 5: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)+0x1fc2) [0x5589806acfa2]
2017-06-30T06:18:58.266 INFO:tasks.ceph.osd.1.smithi112.stderr: 6: (BlueStore::queue_transactions(ObjectStore::Sequencer*, std::vector >&, boost::intrusive_ptr, ThreadPool::TPHandle*)+0x52e) [0x5589806adbce]

need to investigate...

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Jun 30, 2017

Member

and /a/sage-2017-06-30_05:44:03-rados-wip-sage-testing-distro-basic-smithi/1345006 (ceph-objectstore-tool failure)

Member

liewegas commented Jun 30, 2017

and /a/sage-2017-06-30_05:44:03-rados-wip-sage-testing-distro-basic-smithi/1345006 (ceph-objectstore-tool failure)

@xiexingguo

This comment has been minimized.

Show comment
Hide comment
@xiexingguo

xiexingguo Jun 30, 2017

Member

need to investigate...

sure, will do.

Member

xiexingguo commented Jun 30, 2017

need to investigate...

sure, will do.

xiexingguo added some commits Jun 27, 2017

os/bluestore: _do_clone_range:: narrow dirty range
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
os/bluestore: _do_remove::add sanity check
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
os/bluestore: _do_remove:: less overhead; less code indention
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
@xiexingguo

This comment has been minimized.

Show comment
Hide comment
@xiexingguo

xiexingguo Jul 6, 2017

Member

should be dirty_range_end = e.logical_end() - 1; here to exclude next potential shard

Repushed and passed my local ObjectStore/StoreTest, shall be ready for QA again now

Member

xiexingguo commented Jul 6, 2017

should be dirty_range_end = e.logical_end() - 1; here to exclude next potential shard

Repushed and passed my local ObjectStore/StoreTest, shall be ready for QA again now

@xiexingguo xiexingguo added the needs-qa label Jul 6, 2017

@xiexingguo xiexingguo changed the title from DNM: os/bluestore: reduce some overhead for _do_clone_range() and _do_remove() to os/bluestore: reduce some overhead for _do_clone_range() and _do_remove() Jul 6, 2017

@liewegas liewegas merged commit 0ec1089 into ceph:master Jul 7, 2017

4 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details
make check (arm64) make check succeeded
Details

@xiexingguo xiexingguo deleted the xiexingguo:wip-bluestore-20170627 branch Jul 8, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment