Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bluestore: use btree_map for allocator #7269

Merged
merged 11 commits into from Jan 27, 2016

Conversation

liewegas
Copy link
Member

  • less memory. I saw it go from ~55MB per TB to ~20MB per TB; not sure how accurate that is, though, and the cpp-btree claims much better efficiency than that.
  • less cpu.

https://code.google.com/p/cpp-btree/wiki/UsageInstructions#Memory_usage_comparison
https://code.google.com/p/cpp-btree/wiki/UsageInstructions#Performance_comparison

#include <iterator>
#include <map>
#include <ostream>
using namespace std;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using namespace in header files is not a good practice in general, as it brings the entire namespace into the global namespace. as a user, I wouldn't expect that behavior from #include "btree_interval_set.h". the std namespace has tons of names, so conflicts are likely in a large codebase, and they're hard to track down because they generally show up in templates or function overload resolution

all of the std types appear to be qualified with std:: already, so you can probably just take that line out

A few small changes to interval_map to deal with the fact that any btree
change invalidates all iterators.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
btree_map is about twice as memory efficient.

Signed-off-by: Sage Weil <sage@redhat.com>
btree_map uses about half the memory.  According to the cpp-btree docs they
are also faster than STL maps... especially when they get big.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
@liewegas
Copy link
Member Author

fixed std usage in interval_set and btree_interval_set

if (p != kv_free.end())
next = p->first;
--p;
kv_free.erase(p);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like btree_map::erase returns an iterator for the next item:
See btree_map_container declaration:
...
// Erase the specified iterator from the btree. The iterator must be valid
// (i.e. not equal to end()). Return an iterator pointing to the node after
// the one that was erased (or end() if none exists).
iterator erase(const iterator &iter) {
return this->tree_.erase(iter);
}
...

Thus you probably can have something like
p = kv.free.erase(p);
and remove other stuff to locate the next entry

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The btree_map erase() returns an iterator for the next element.
Perfect!

Reported-by: Igor Fedotov <ifedotov@mirantis.com>
Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
This mirrors the condition above.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
@ifed01
Copy link
Contributor

ifed01 commented Jan 27, 2016

@liewegas, I made some unit tests to verify both interval sets implementations. They pass OK. PR is made to your repo since btree version isn't merged yet. Sorry if that's inappropriate.
PR is at
liewegas#6

UT for interval_set implementations
liewegas added a commit that referenced this pull request Jan 27, 2016
bluestore: use btree_map for allocator

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
@liewegas liewegas merged commit 9c30be9 into ceph:master Jan 27, 2016
@liewegas liewegas deleted the wip-bluestore-btree branch January 27, 2016 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants