New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

luminous: core: PGPool::update optimizations #23969

Merged
merged 4 commits into from Sep 14, 2018

Conversation

Projects
None yet
7 participants
@zmedico
Contributor

zmedico commented Sep 6, 2018

Backports for this pull requests:

#17121
#17239
#17410
#17820 (5c37758 already backported in 61c66a1)

@gregsfortytwo
@chutz
@robbat2

zmedico added some commits Aug 21, 2017

PGPool::update: optimize with interval_set.swap
Use constant complexity swap instead of linear complexity
assignment.

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit ae64179)
PGPool::update: avoid expensive union_of
Achieve the same result using a swap (constant complexity),
and an assignment (linear complexity).

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit bf20b66)
PGPool::update: optimize removed_snaps comparison when possible
In self/unmanaged snaps mode, optimize removed_snaps comparison
for cases where removed_snaps has not changed. This exploits the
fact that remove_unmanaged_snap adds a dummy removed snapshot
to the end of removed_snaps, allowing for inexpensive detection
of changes. In cases where removed_snaps is very large, this
optimization improves performance dramatically.

Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit caf6803)
osd/PGPool::update: optimize with subset_of
Replace expensive inverval_set intersection_of and operator==
calls with a single subset_of call. I borrowed this idea from
Piotr Dałek's "osd/PGPool: don't use intermediate interval set"
patch. The following benchmark program demonstrates a 38%
performance increase:

#include <iostream>
#include <chrono>
#include "include/interval_set.h"

#define NANOSECONDS(d) \
    std::chrono::duration_cast<std::chrono::nanoseconds>(d).count()

typedef uint64_t snapid_t;
typedef std::chrono::steady_clock::duration duration;

duration PGPool_update_old(const interval_set<snapid_t> &rs) {
  std::chrono::steady_clock::time_point start, end;
  interval_set<snapid_t> newly_removed_snaps, cached_removed_snaps;

  // initialize state
  cached_removed_snaps = rs;

  // start timed simulation
  start = std::chrono::steady_clock::now();

  {
    newly_removed_snaps = cached_removed_snaps;
    interval_set<snapid_t> intersection;
    intersection.intersection_of(newly_removed_snaps, cached_removed_snaps);

    assert(intersection == cached_removed_snaps);
    cached_removed_snaps.swap(newly_removed_snaps);
    newly_removed_snaps = cached_removed_snaps;
    newly_removed_snaps.subtract(intersection);
  }

  // end timed simulation
  end = std::chrono::steady_clock::now();

  return end - start;
}

duration PGPool_update_new(const interval_set<snapid_t> &rs) {
  std::chrono::steady_clock::time_point start, end;
  interval_set<snapid_t> newly_removed_snaps, cached_removed_snaps;

  // initialize state
  cached_removed_snaps = rs;

  // start timed simulation
  start = std::chrono::steady_clock::now();

  {
    newly_removed_snaps = cached_removed_snaps;

    assert(cached_removed_snaps.subset_of(newly_removed_snaps));
    interval_set<snapid_t> removed_snaps = newly_removed_snaps;
    newly_removed_snaps.subtract(cached_removed_snaps);
    cached_removed_snaps.swap(removed_snaps);
  }

  // end timed simulation
  end = std::chrono::steady_clock::now();

  return end - start;
}

int main(int argc, char *argv[])
{
  assert(argc == 3);
  const int sample_count = std::stoi(argv[1]);
  const int interval_count = std::stoi(argv[2]);
  const int interval_distance = 4;
  const int interval_size = 2;
  const int max_offset = interval_count * interval_distance;
  interval_set<snapid_t> removed_snaps;

  for (int i = 0; i < max_offset; i += interval_distance)
    removed_snaps.insert(i, interval_size);

  duration old_delta(0), new_delta(0);

  for (int i = 0; i < sample_count; ++i) {
    old_delta += PGPool_update_old(removed_snaps);
    new_delta += PGPool_update_new(removed_snaps);
  }

  float ratio = float(NANOSECONDS(old_delta)) / NANOSECONDS(new_delta);

  std::cout << ratio << std::endl;
}

Suggested-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
Signed-off-by: Zac Medico <zmedico@gmail.com>
(cherry picked from commit 18cbba4)

@tchaikov tchaikov added this to the luminous milestone Sep 7, 2018

@smithfarm smithfarm changed the title from lumious: PGPool::update optimizations to luminous: PGPool::update optimizations Sep 11, 2018

@smithfarm smithfarm requested review from gregsfortytwo, jdurgin and robbat2 Sep 11, 2018

@yuriw

This comment has been minimized.

Contributor

yuriw commented Sep 12, 2018

@gregsfortytwo

Reviewed-by: Greg Farnum gfarnum@redhat.com

@yuriw yuriw merged commit 768e4b7 into ceph:luminous Sep 14, 2018

4 checks passed

Docs: build check OK - docs built
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details

@smithfarm smithfarm changed the title from luminous: PGPool::update optimizations to luminous: core: PGPool::update optimizations Oct 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment