jewel: rbd: rbd-mirror: reduce memory footprint during journal replay #10684
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dillaman
force-pushed
the
wip-16904-jewel
branch
from
8c00da1
to
e8a068a
Compare
|
retest this please |
|
(unittest_bluefs seg fault causing test failure) |
|
hey jenkins, test this please (unittest_bluefs seg fault causing test failure) |
ghost
changed the title
ghost
self-assigned this
dillaman
force-pushed
the
wip-16904-jewel
branch
from
e8a068a
to
7017f20
Compare
|
@dachary rebased |
|
@dillaman thanks ! There is an actual compilation problem though: CXX test/rbd_mirror/image_sync/unittest_rbd_mirror-test_mock_SyncPointPruneRequest.o
test/rbd_mirror/image_sync/test_mock_ImageCopyRequest.cc:632:7: error: redefinition of ‘class rbd::mirror::image_sync::TestMockImageSyncImageCopyRequest_EmptySnapSeqs_Test’
TEST_F(TestMockImageSyncImageCopyRequest, EmptySnapSeqs) {
^
test/rbd_mirror/image_sync/test_mock_ImageCopyRequest.cc:610:7: error: previous definition of ‘class
|
Fixes: http://tracker.ceph.com/issues/16489 Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit c97f724)
Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 48f301d)
Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 03c2aec)
Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 1fc2754)
Fixes: http://tracker.ceph.com/issues/16349 Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 2f55aa5)
The watching object name is changed when renaming an old format image, so unregister the watcher before the rename, and register back after, to avoid "Transport endpoint is not connected" error. Fixes: http://tracker.ceph.com/issues/16321 Signed-off-by: Mykola Golub <mgolub@mirantis.com> (cherry picked from commit 1a3973c)
Snapshot rename operations utilize the (cluster) unique snapshot sequence to prevent attempts at replays. When mirroring to a different cluster, these sequences will not align. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 2f4cb26)
Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 77699bf) Conflicts: src/test/librbd/mock/MockOperations.h: no shrink restriction
Remote peers need a key to map snapshot ids between clusters. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit f70b90c)
Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 57cd75e)
Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 270cb74) Conflicts: src/librbd/journal/Replay.cc: no snap limit restriction
Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit fdfca55)
Fixes: http://tracker.ceph.com/issues/16622 Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 4df913d)
Additional runtime configuration settings will be needed. The new class will avoid the need to expand the constructor. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit dad8328)
Support fetching the full object or incremental chunks (with a minimum of at least a single decoded entry if available). Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit f7362e9)
Journal playback will need to read at least a full entry which was currently limited to the maximum object size. In memory constrained environment, this new optional limit will set a fix upper bound on memory usage. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 8c1877b)
Previously it was prefetching up to 2 object sets worth of journal data objects which consumed too much memory. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 2666d36)
Now that it's possible for the ObjectPlayer to only read a partial subset of available entries, the JournalPlayer needs to detect that more entries might be available. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 28d5ca1)
rbd-mirror debugging involved potentially thousands of journals concurrently running. The instance address will correlate log messages between journals. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 11475f4)
If a future flush is requested at the exact same moment that an overflow is detected, the two threads will deadlock since locks are not taken in a consistent order. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 2c65471)
When streaming playback, avoid the unnecessary watch delay when one or more entries have been pruned. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 08a8ee9)
Operation request op finish events should not be fire and forget. Instead, ensure the event is committed to the journal before completing the op. This will avoid several possible split-brain events during mirroring. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 47e0fbf) Conflicts: src/test/librbd/operation/test_mock_ResizeRequest.cc: no shrink restriction
Ensure that, by default, IO journal events are broken up into manageable sizes when factoring in that an rbd-mirror daemon might be replaying events from thousands of images. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 11d7500)
Fixes: http://tracker.ceph.com/issues/16223 Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 24883e0)
When multiple pools are being replicated, start the shut down process concurrently across all pool replayers. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 73cdd08)
Fixed lockdep issue from status update callback and fixed the potential for a stuck status state. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 0275c7c)
The cancel request could race with the actual scheduling of the image sync operation. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit e6cdf95)
librbd will replay these ops when opening an image, so rbd-mirror should also ensure these ops are replayed. Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 862e581)
Signed-off-by: Jason Dillaman <dillaman@redhat.com> (cherry picked from commit 574be74)
Fixes: http://tracker.ceph.com/issues/16539 Signed-off-by: Mykola Golub <mgolub@mirantis.com> (cherry picked from commit 06a333b)
dillaman
force-pushed
the
wip-16904-jewel
branch
from
7017f20
to
92d7882
Compare
ghost
changed the title
ghost
pushed a commit
that referenced
this pull request
Aug 17, 2016
… during journal replay Reviewed-by: Loic Dachary <ldachary@redhat.com>
This was referenced Aug 17, 2016
Closed
|
Testing in progress at http://tracker.ceph.com/issues/16904#note-5 |
|
@dillaman it passed a rbd suite run modulo (I think ;-) known false negatives http://tracker.ceph.com/issues/16904#note-5. Good to merge ? |
|
👍 |
ghost
merged commit
ghost
changed the title
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
http://tracker.ceph.com/issues/16512
http://tracker.ceph.com/issues/16796
http://tracker.ceph.com/issues/16658
http://tracker.ceph.com/issues/16747
http://tracker.ceph.com/issues/17066
http://tracker.ceph.com/issues/16904