Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

jewel: rbd: mirror: improve resiliency of stress test case #11433

Merged
45 commits merged into from Oct 17, 2016
Merged

jewel: rbd: mirror: improve resiliency of stress test case #11433

45 commits merged into from Oct 17, 2016

Conversation

dillaman
Copy link

@dillaman dillaman commented Oct 11, 2016 •
edited by ghost

Mykola Golub and others added 30 commits October 11, 2016 12:40
cls/journal: add async client_update_state method
3aec576 
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 58b8c66)
journal: allow to trim journal for "laggy" clients
4056e36 
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 0b8b1aa)
librbd: optionally flag "laggy" journal clients disconnected
d3ad2ff 
Fixes: http://tracker.ceph.com/issues/14738
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit b8eafef)
rbd: new command to disconnect journal client
7382e1a 
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit fc3ba54)
rbd-mirror: decode_client_meta should return false on error
170476e 
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit cd5eb36)
rbd-mirror: resync was possible only when image replayer start had su…
80aa7e0 
…cceeded

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 4bf6912)
rbd-mirror: stop replay when client is disconnected
0b40239 
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 330dba0)
rbd-mirror: option to automatically resync after journal client disco…
f7ffbfa 
…nnect

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 77fd6a1)

Conflicts:
	src/common/config_opts.h: trivial resolution
qa/workunits/rbd: demote/promote image on same cluster
d775680 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e6aa18e)
rbd-mirror: include tag tid in bootstrap debug log messages
13daaff 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 47d1e62)
librbd: separate journal::TagData predecessor data into new struct
8ccdad6 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 718befd)
librbd: block RPC requests when demoting the image
4baea6a 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8b195e1)
librbd: normalize journal promotion/demotion events
f8f3bbd 
A non-primary image's commit possition won't accurately reflect
the current demotion/promotion chain. Therefore, directly specify
the predecessor for promotion events.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ac590e8)
rbd-mirror: demote/promote in same cluster results in split-brain
7a43484 
Fixes: http://tracker.ceph.com/issues/16855
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a6901ca)
rbd-mirror: prevent syncing to non-primary image after failover
ca94f25 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit beaef37)
test: fixed unused function warnings in unittest_rbd_mirror
89c6618 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3545d9e)
librbd: fix journal create compilation error
3bbd8ba 
The synchronous journal create method no longer exists on the master
branch and the associated change to introduce an asynchronous journal
creation state machine should be delayed to provide more testing time
on the master branch before being backported to jewel.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
qa/workunits/rbd: before removing image make sure it is not bootstrapped
775a999 
If an image is being bootstrapped, it implies that the rbd-mirror
daemon currently has the image open. The removal API will prevent the
removal of any image that is opened by another client.

Works-around: http://tracker.ceph.com/issues/16555
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 9737a8d)
test/rbd: fix possible mock journal race conditions
5d1d898 
Fixes: http://tracker.ceph.com/issues/17317
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4718983)
qa/workunits/rbd: new mirroring forced promotion test
2ec5e93 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a43268a)
qa/workunits/rbd: fix remove mirrored image race conditions
2f9e641 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a0f65b9)
journal: expose ability to retrieve partial tag list
41cf3d9 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 277b6dd)
librbd: unify journal event callbacks into single interface
829ff8c 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit dbbcecf)
librbd: helper class for quiescing in-flight async ops
4f5ce86 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 39d9e5c)
librbd: new journal listener event for force promotion
f9aae06 
Fixes: http://tracker.ceph.com/issues/16974
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fd00549)
journal: send update notification when tag allocated
9ad132e 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f1cd613)
@rjfd
journal: increase concurrency of journal recorder
5edbfe2 
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 5c88edd)
@rjfd
journal: make librados call async in ObjectRecorder
180a86b 
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 7b740f5)
@rjfd
rbd: Fix race between journal flush and append events
6ea4cde 
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit aa959e7)
@rjfd
rbd-mirror: test: Fixed timeout problem in rbd_mirror_stress.sh
8070d6a 
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 5c73703)
Jason Dillaman added 7 commits October 11, 2016 13:57
journal: avoid holding lock while sending journal append
21502d9 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit dc77a62)
journal: use reverse iterator search to find flush record
9f36143 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a960658)
journal: delay moving overflowed buffers until quiesced
310f3f7 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4483531)
journal: clean up object recorder closed/overflow callback
912ce37 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 24faead)
librbd: ignore attempts to request lock if already lock owner
cea000c 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 355c79c)
rbd-mirror: potential race allocating tag during shut down
d51a2f6 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8c2ff9b)
qa/workunits: reduce amount of client IO for rbd-mirror stress
7487332 
Journal IO is faster now, resulting in OSDs quickly filling up
under the stress test case.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a6dd6b5)
@dillaman
Copy link
Author

dillaman commented Oct 11, 2016 •
edited

This PR includes the backports for the following master-branch PRs: #10378, #10703, #10155, #11153, #11090, #10445, #11104, #10484, #11185, #11326, and #11257

Mykola Golub added 2 commits October 11, 2016 14:16
journal: complete action only after notification completed
9bc5e44 
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 55762cc)
journal: ensure in-flight ops are complete destroying journaler
1c76ef4 
Fixes: http://tracker.ceph.com/issues/17446
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a25b1d7)
@dillaman dillaman changed the title [DNM] jewel: rbd-mirror: improve resiliency of stress test case jewel: rbd-mirror: improve resiliency of stress test case Oct 11, 2016
@dillaman
Copy link
Author

retest this please

@ghost ghost self-assigned this Oct 12, 2016
@ghost
Copy link

ghost commented Oct 12, 2016 •
edited by ghost

@dillaman I'll do the backport cross reference when the tracker is back online.

This was referenced Oct 13, 2016
Closed
Closed
ghost pushed a commit that referenced this pull request Oct 13, 2016
Merge pull request #11433: jewel: rbd-mirror: improve resiliency of s…
4bf6a91 
…tress test case

Reviewed-by: Loic Dachary <ldachary@redhat.com>
@ghost
Copy link

ghost commented Oct 13, 2016

pushed this in the jewel-backports branch and will run a rbd suite today

ghost pushed a commit that referenced this pull request Oct 17, 2016
Merge pull request #11433: jewel: rbd-mirror: improve resiliency of s…
7306023 
…tress test case

Reviewed-by: Loic Dachary <ldachary@redhat.com>
@ghost
Copy link

ghost commented Oct 17, 2016

It passed the rbd suite http://tracker.ceph.com/issues/17487#note-10 (except for two failures that are unrelated http://tracker.ceph.com/issues/17590 and http://tracker.ceph.com/issues/17262)

@ghost ghost merged commit 7d0dd1c into ceph:jewel Oct 17, 2016
@theanalyst theanalyst changed the title jewel: rbd-mirror: improve resiliency of stress test case "jewel: rbd: mirror: improve resiliency of stress test case" Nov 17, 2016
@theanalyst theanalyst changed the title "jewel: rbd: mirror: improve resiliency of stress test case" jewel: rbd: mirror: improve resiliency of stress test case Nov 17, 2016
@dillaman dillaman deleted the wip-17416-jewel branch January 25, 2017 16:03
This was referenced Jun 21, 2023
Open
Open
@ekmixon ekmixon mentioned this pull request Nov 29, 2023
Open
@ekmixon ekmixon mentioned this pull request Dec 20, 2023
Open
@ekmixon ekmixon mentioned this pull request Mar 15, 2024
Open
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug-fix rbd tests
Projects
None yet
Milestone
jewel
2 participants
@dillaman @rjfd