New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/bluestore: replace Blob ref_map with reference counting #12904

Merged
merged 8 commits into from Feb 2, 2017

Conversation

Projects
None yet
3 participants
@ifed01
Contributor

ifed01 commented Jan 12, 2017

This PR is based on some commits from PR #12700
This effectively reduces 4Mb Onode metadata in-memory size from 418K to 254K

Signed-off-by: Igor Fedotov ifedotov@mirantis.com

@ifed01 ifed01 requested a review from liewegas Jan 12, 2017

@xiexingguo

This comment has been minimized.

Show comment
Hide comment
@xiexingguo

xiexingguo Jan 13, 2017

Member

retest this please

Member

xiexingguo commented Jan 13, 2017

retest this please

@ifed01

This comment has been minimized.

Show comment
Hide comment
@ifed01

ifed01 Jan 16, 2017

Contributor

@liewegas - resolved.

Contributor

ifed01 commented Jan 16, 2017

@liewegas - resolved.

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Jan 16, 2017

Member
Member

liewegas commented Jan 16, 2017

}
bool empty = maybe_empty ? !is_not_empty() : false;
if (empty && release_units) {
release_units->clear();

This comment has been minimized.

@liewegas

liewegas Jan 16, 2017

Member

can you document the semantics of the empty return value vs release_units in the header?

@liewegas

liewegas Jan 16, 2017

Member

can you document the semantics of the empty return value vs release_units in the header?

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Jan 16, 2017

Member

The other patches look good!

Overall I think this PR is clearly the way to go. My main hope is that we can avoid the Blob::put_ref() rewrite and avoid building a bit set<> of individual au's ...

Member

liewegas commented Jan 16, 2017

The other patches look good!

Overall I think this PR is clearly the way to go. My main hope is that we can avoid the Blob::put_ref() rewrite and avoid building a bit set<> of individual au's ...

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Jan 17, 2017

Member
Member

liewegas commented Jan 17, 2017

@ifed01

This comment has been minimized.

Show comment
Hide comment
@ifed01

ifed01 Jan 20, 2017

Contributor

@liewegas - comments resolved, please review once more...

Contributor

ifed01 commented Jan 20, 2017

@liewegas - comments resolved, please review once more...

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Jan 26, 2017

Member
  MismatchedFree
  Mismatched free() / delete / delete []
  
    
      0x4C2918D
      /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so
      operator delete(void*)
      /builddir/build/BUILD/valgrind-3.11.0/coregrind/m_replacemalloc
      vg_replace_malloc.c
      576
    
    
      0x805FB0
      /usr/bin/ceph-osd
      clear
      /usr/src/debug/ceph-11.1.0-7031-g7e25bbc/src/os/bluestore
      bluestore_types.h
      304
    
    
      0x805FB0
      /usr/bin/ceph-osd
      ~bluestore_blob_use_tracker_t
      /usr/src/debug/ceph-11.1.0-7031-g7e25bbc/src/os/bluestore
      bluestore_types.h
      299
    
    
      0x805FB0
      /usr/bin/ceph-osd
      ~Blob
      /usr/src/debug/ceph-11.1.0-7031-g7e25bbc/src/os/bluestore
      BlueStore.h
      422
    

from /a/sage-2017-01-25_23:09:16-rados-wip-sage-testing---basic-smithi/748438

Member

liewegas commented Jan 26, 2017

  MismatchedFree
  Mismatched free() / delete / delete []
  
    
      0x4C2918D
      /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so
      operator delete(void*)
      /builddir/build/BUILD/valgrind-3.11.0/coregrind/m_replacemalloc
      vg_replace_malloc.c
      576
    
    
      0x805FB0
      /usr/bin/ceph-osd
      clear
      /usr/src/debug/ceph-11.1.0-7031-g7e25bbc/src/os/bluestore
      bluestore_types.h
      304
    
    
      0x805FB0
      /usr/bin/ceph-osd
      ~bluestore_blob_use_tracker_t
      /usr/src/debug/ceph-11.1.0-7031-g7e25bbc/src/os/bluestore
      bluestore_types.h
      299
    
    
      0x805FB0
      /usr/bin/ceph-osd
      ~Blob
      /usr/src/debug/ceph-11.1.0-7031-g7e25bbc/src/os/bluestore
      BlueStore.h
      422
    

from /a/sage-2017-01-25_23:09:16-rados-wip-sage-testing---basic-smithi/748438

Igor Fedotov added some commits Dec 23, 2016

Igor Fedotov
os/bluestore: replace Blob's ref_map with reference counting
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
Igor Fedotov
os/bluestore: raise ExtentMap/Blob encoding version to handle migrati…
…on from ref_map to ref counting properly.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
Igor Fedotov
os/bluestore: add bluestore_blob_use_tracker_t to bluestore types 'si…
…zeof' unittest.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
Igor Fedotov
os/bluestore: split SharedBlob into two-parts to improve in-mem footp…
…rint for non-shared blobs.

Signed-off-by: Igor Fedotov <ifedotor@mirantis.com>
Igor Fedotov
os/store_test: extend Onode in-mem size estimation with cached data p…
…resence case.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
Igor Fedotov
os/bluestore: optimize SharedBlob::sbid in-mem representation.
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
Igor Fedotov
os/bluestore: get rid off unused SharedBlob::hash_value function.
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
Igor Fedotov
os/bluestore: fix miscalculation in bluestore_extent_ref_map_t::bound…
…_encode

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
@ifed01

This comment has been minimized.

Show comment
Hide comment
@ifed01

ifed01 Jan 26, 2017

Contributor

@liewegas - fixed.

Contributor

ifed01 commented Jan 26, 2017

@liewegas - fixed.

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Jan 27, 2017

Member

Hit this from ObjectStore/StoreTest.SyntheticMatrixCsumVsCompression/2

2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:    -2> 2017-01-27 18:58:04.424198 7f6b6de38700 -1 freelist _verify_range key 0xfffffffffff80000 not present, expected 0x1
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:    -1> 2017-01-27 18:58:04.424227 7f6b6de38700 -1 freelist _verify_range saw 1 errors
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:     0> 2017-01-27 18:58:04.426027 7f6b6de38700 -1 /build/ceph-11.1.0-7072-g80df3d9/src/os/bluestore/BitmapFreelistManager.cc: In function 'void BitmapFreelistManager::_verify_range(uint64_t, uint64_t, int)' thread 7f6b6de38700 time 2017-01-27 1
8:58:04.424250
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:/build/ceph-11.1.0-7072-g80df3d9/src/os/bluestore/BitmapFreelistManager.cc: 454: FAILED assert(0 == "bitmap freelist errors")
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr: ceph version 11.1.0-7072-g80df3d9 (80df3d90ce6633aae8e985e346cc3a07f0ee8568)
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x7f6b7adc657e]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 2: (BitmapFreelistManager::_verify_range(unsigned long, unsigned long, int)+0x832) [0x7f6b83be4b82]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 3: (BitmapFreelistManager::release(unsigned long, unsigned long, std::shared_ptr)+0x2ab) [0x7f6b83be738b]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 4: (BlueStore::_txc_finalize_kv(BlueStore::TransContext*, std::shared_ptr)+0x23e) [0x7f6b83b4ed6e]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 5: (BlueStore::_kv_sync_thread()+0x1665) [0x7f6b83b58db5]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 6: (BlueStore::KVSyncThread::entry()+0xd) [0x7f6b83b85e0d]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 7: (()+0x8184) [0x7f6b7a935184]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 8: (clone()+0x6d) [0x7f6b7903f37d]

which looks pretty suspicious. Maybe try running it in a loop?

Member

liewegas commented Jan 27, 2017

Hit this from ObjectStore/StoreTest.SyntheticMatrixCsumVsCompression/2

2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:    -2> 2017-01-27 18:58:04.424198 7f6b6de38700 -1 freelist _verify_range key 0xfffffffffff80000 not present, expected 0x1
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:    -1> 2017-01-27 18:58:04.424227 7f6b6de38700 -1 freelist _verify_range saw 1 errors
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:     0> 2017-01-27 18:58:04.426027 7f6b6de38700 -1 /build/ceph-11.1.0-7072-g80df3d9/src/os/bluestore/BitmapFreelistManager.cc: In function 'void BitmapFreelistManager::_verify_range(uint64_t, uint64_t, int)' thread 7f6b6de38700 time 2017-01-27 1
8:58:04.424250
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:/build/ceph-11.1.0-7072-g80df3d9/src/os/bluestore/BitmapFreelistManager.cc: 454: FAILED assert(0 == "bitmap freelist errors")
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr:
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr: ceph version 11.1.0-7072-g80df3d9 (80df3d90ce6633aae8e985e346cc3a07f0ee8568)
2017-01-27T18:58:04.542 INFO:teuthology.orchestra.run.smithi180.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x7f6b7adc657e]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 2: (BitmapFreelistManager::_verify_range(unsigned long, unsigned long, int)+0x832) [0x7f6b83be4b82]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 3: (BitmapFreelistManager::release(unsigned long, unsigned long, std::shared_ptr)+0x2ab) [0x7f6b83be738b]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 4: (BlueStore::_txc_finalize_kv(BlueStore::TransContext*, std::shared_ptr)+0x23e) [0x7f6b83b4ed6e]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 5: (BlueStore::_kv_sync_thread()+0x1665) [0x7f6b83b58db5]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 6: (BlueStore::KVSyncThread::entry()+0xd) [0x7f6b83b85e0d]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 7: (()+0x8184) [0x7f6b7a935184]
2017-01-27T18:58:04.543 INFO:teuthology.orchestra.run.smithi180.stderr: 8: (clone()+0x6d) [0x7f6b7903f37d]

which looks pretty suspicious. Maybe try running it in a loop?

@ifed01

This comment has been minimized.

Show comment
Hide comment
@ifed01

ifed01 Feb 2, 2017

Contributor

Passed 32 iterations without any issues..

Contributor

ifed01 commented Feb 2, 2017

Passed 32 iterations without any issues..

@liewegas liewegas merged commit 013b96e into ceph:master Feb 2, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details

@ifed01 ifed01 deleted the ifed01:wip-bluestore-onode-refmap3 branch Feb 2, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment