New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jewel: core: interval_set: optimize intersect_of insert operations #17514

Merged
merged 2 commits into from Sep 7, 2017

Conversation

Projects
None yet
4 participants
@smithfarm
Contributor

smithfarm commented Sep 6, 2017

zmedico added some commits Aug 25, 2017

interval_set: optimize intersect_of insert operations
Use the std::map insert method with hint iterator to optimize
inserts. This increases performance more than 3.5 times for
large numbers of intervals. This will help performance
especially in the PGPool::update method, where profiling data
has shown that intersection operations are a hot spot. The
following benchmark data is for 400000 intervals:

    4 +-+--+----+----+----+----+----+----+----+----+--+-+
P     +    +    +    +    +    +    +    +  *************
E     |                             ********            |
R 3.5 +-+                       ****                  +-+
F     |                   ******                        |
O     |                 **                              |
R   3 +-+           ****                              +-+
M     |          ***                                    |
A     |        **                                       |
N 2.5 +-+     *                                       +-+
C     |     **                                          |
E     |     *                                           |
    2 +-+ **                                          +-+
R     |  **                                             |
A     | **                                              |
T 1.5 +**                                             +-+
I     |**                                               |
O     +*   +    +    +    +    +    +    +    +    +    +
    1 +*+--+----+----+----+----+----+----+----+----+--+-+
      0   0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9

                        SET SIZE RATIO

The above chart was generated using benchmark results
from the following program:

#include <iostream>
#include <sys/timeb.h>
#include "include/interval_set.h"

int main(int argc, char *argv[])
{
  const int interval_count = std::stoi(argv[1]);
  const int interval_distance = 4;
  const int interval_size = 2;
  const int sample_count = 8;
  const int max_offset = interval_count * interval_distance;
  interval_set<int> a, b, intersection;

  for (int i = 0; i < max_offset; i+=interval_distance) {
    a.insert(i, interval_size);
  }

  for (int m = 1; m < 100; m++) {
    float ratio = 1 / float(m);

    for (int i = 0; i < max_offset; i+=interval_distance*m) {
      b.insert(i, interval_size);
    }

    struct timeb start, end;
    int ms = 0;
    for (int i = 0; i < sample_count; i++) {
      ftime(&start);
      intersection.intersection_of(a, b);
      ftime(&end);
      ms += (int) (1000.0 * (end.time - start.time)
        + (end.millitm - start.millitm));
      intersection.clear();
    }
    b.clear();

    std::cout << ratio << "\t" << ms << std::endl << std::flush;
  }
}

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit 32bc043)
interval_set: optimize intersect_of for identical spans
Optimize comparisons for identical spans of intervals.
When this patch is combined with the previous map insert
optimization, a benchmark using 400000 identical
intervals shows a 7 times performance improvement in
comparison to without the patches.

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit b6a0356)

@smithfarm smithfarm self-assigned this Sep 6, 2017

@smithfarm smithfarm added this to the jewel milestone Sep 6, 2017

@liu-chunmei

This comment has been minimized.

Show comment
Hide comment
@liu-chunmei

liu-chunmei Sep 6, 2017

Contributor

what kind of benefit do we get by this patch?

Contributor

liu-chunmei commented Sep 6, 2017

what kind of benefit do we get by this patch?

@smithfarm

This comment has been minimized.

Show comment
Hide comment
@smithfarm

smithfarm Sep 7, 2017

Contributor

@liu-chunmei The commit messages describe performance benefits.

Contributor

smithfarm commented Sep 7, 2017

@liu-chunmei The commit messages describe performance benefits.

@smithfarm

This comment has been minimized.

Show comment
Hide comment
@smithfarm

smithfarm Sep 7, 2017

Contributor

Passed a rados suite at http://tracker.ceph.com/issues/20613#note-48 with some failures that are, I believe, unrelated.

Contributor

smithfarm commented Sep 7, 2017

Passed a rados suite at http://tracker.ceph.com/issues/20613#note-48 with some failures that are, I believe, unrelated.

@smithfarm smithfarm requested review from adamemerson and liewegas Sep 7, 2017

@smithfarm

This comment has been minimized.

Show comment
Hide comment
@smithfarm

smithfarm Sep 7, 2017

Contributor

@liewegas OK to have this in jewel?

Contributor

smithfarm commented Sep 7, 2017

@liewegas OK to have this in jewel?

@smithfarm smithfarm merged commit f212b36 into ceph:jewel Sep 7, 2017

4 checks passed

Docs: build check OK - docs built
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details

@smithfarm smithfarm deleted the smithfarm:wip-21232-jewel branch Sep 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment