hammer: tools: heavy memory shuffling in rados bench #5810
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This function iterates over all bufferlist internal buffers and calls their invalidate_crc() method. Required for rados bench to work correctly, because it modifies buffers outside buffer api, invalidating internal CRC cache in the process - this method clears that cache, so another call for buffer::get_crc() to return correct checksum. Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com> (cherry picked from commit 55a6f9e)
Abhishekvrshny
force-pushed
the
wip-12948-hammer
branch
from
eb92452
to
eb9a58e
Compare
Fixes the high CPU usage and corrects rados bench scores on fast SSDs and ramdisks/memstore. For bench run on SSD, on Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz before this patch, times are: write: real 5m0.169s, user 2m33.565s, sys 4m39.791s seq: real 4m28.642s, user 1m35.250s, sys 6m42.948s rand: real 5m0.258s, user 1m19.656s, sys 6m47.145s After this patch: write: real 5m1.162s, user 0m27.788s, sys 3m11.707s seq: real 5m1.149s, user 2m23.278s, sys 4m14.427s rand: real 5m1.021s, user 2m30.514s, sys 4m20.347s Bench run: rados -p ssd bench 300 write|seq|read --no-cleanup Note the increase in user time cpu on seq/read tests, along with decreased sys cpu time; this is because there's additional memcmp() that compares read objects with expected contents. With less time spent memory juggling, more time is spent performing more reads per second, increasing memcmp call count and increasing amount of user cpu time used. Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com> (cherry picked from commit b894fc7) Conflicts: src/common/obj_bencher.cc return -5 was replaced with return r
Abhishekvrshny
force-pushed
the
wip-12948-hammer
branch
from
eb9a58e
to
dba8b5b
Compare
ghost
pushed a commit
that referenced
this pull request
Oct 2, 2015
Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost
pushed a commit
that referenced
this pull request
Oct 3, 2015
Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost
pushed a commit
that referenced
this pull request
Oct 3, 2015
Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost
pushed a commit
that referenced
this pull request
Oct 3, 2015
Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost
pushed a commit
that referenced
this pull request
Oct 7, 2015
Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost
pushed a commit
that referenced
this pull request
Oct 8, 2015
Reviewed-by: Loic Dachary <ldachary@redhat.com>
|
@tchaikov does this backport look good to merge ? It passed a run of the hammer rados suite ( see http://tracker.ceph.com/issues/13356#note-11 and for details ). |
|
@Abhishekvrshny do you want to backport #5152 as well? |
|
@dachary #5152 was mentioned in http://tracker.ceph.com/issues/12946, but it was not explicitly associated with any issue. i am just concerned that the obj_bencher might report error under some conditions as put by @branch-predictor in the commit message of https://github.com/ceph/ceph/pull/5152/commits . so probably we can merge this pr and keep in mind that we'd better backport #5152 soon. what do you think? |
|
Actually, I made separate issue for backport of #5152 (http://tracker.ceph.com/issues/12947) and the related PR is #5812. |
|
thanks @branch-predictor ! @dachary lgtm then. |
ghost
pushed a commit
that referenced
this pull request
Nov 9, 2015
Heavy memory shuffling in rados bench Reviewed-by: Kefu Chai <kchai@redhat.com>
ghost
merged commit
ghost
pushed a commit
that referenced
this pull request
Nov 17, 2015
Heavy memory shuffling in rados bench Reviewed-by: Kefu Chai <kchai@redhat.com>
ghost
pushed a commit
that referenced
this pull request
Nov 17, 2015
Heavy memory shuffling in rados bench Reviewed-by: Kefu Chai <kchai@redhat.com>
ghost
pushed a commit
that referenced
this pull request
Nov 17, 2015
Heavy memory shuffling in rados bench Reviewed-by: Kefu Chai <kchai@redhat.com>
ghost
pushed a commit
that referenced
this pull request
Nov 17, 2015
Heavy memory shuffling in rados bench Reviewed-by: Kefu Chai <kchai@redhat.com>
ghost
pushed a commit
that referenced
this pull request
Nov 17, 2015
Heavy memory shuffling in rados bench Reviewed-by: Kefu Chai <kchai@redhat.com>
ghost
changed the title
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
http://tracker.ceph.com/issues/12948