Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mimic: rbd-mirror: optionally support active/active replication #22105

Merged
merged 25 commits into from
May 21, 2018
Merged

mimic: rbd-mirror: optionally support active/active replication #22105

merged 25 commits into from
May 21, 2018

Conversation

dillaman
Copy link

No description provided.

Jason Dillaman added 25 commits May 19, 2018 08:16
rbd-mirror: support optionally enabling active/active mirroring
04bd250 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 53b87b9)
rbd-mirror: instance replayer now respects manual stop command
fdcf90e 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 06e1244)
rbd-mirror: instantly remove instances without associated images
7c9b263 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit bce5328)
rbd-mirror: prevent blacklisting of local instance after failover/back
4ec33ea 
If the leader role is manually released, upon failback the instance
will have removed its local instance object, preventing RPC
messaging.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ce97430)
rbd-mirror: policy map shouldn't default to first instance if dead
1f398ad 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 524f08c)
rbd-mirror: cancel image replayer status timer task at shut down
59db540 
Also avoid attempting to send status using an invalid librados::IoCtx
handle due to a deleted pool.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e2a0088)
rbd-mirror: reduce unnecessary logging for periodic start image replayer
dcc67bb 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d7cb5db)
rbd-mirror: avoid sending duplicate re-acquire messages upon leader s…
0e12093 
…tartup

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a9d335d)
rbd-mirror: propagate image delete if leader was offline
c860e04 
In an active/active scenario, if the leader was offline while mirroring for
a remote image was disabled, the assigned replayer instance may not detect
the image removal.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b3acc56)
rbd-mirror: do not propagate image map instance updates when shutting…
4cd7b67 
… down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4525b7a)
rbd-mirror: image create should lookup local parent snapshot id
294c7e0 
A recent code change associated with a librbd cleanup incorrectly started
using the remote parent image's snapshot id.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 02afa2a)
librbd: don't assert if parent pool doesn't exist
51e29a3 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 88436b5)
librbd: journal replay should no-op out-of-bounds IO
8a65b0e 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9a1fcef)
librbd: deep-copy should remove parent metadata if overlap updated
fb8846b 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit af42984)
test/rbd_mirror: fix race condition with image map timer callbacks
9feb3af 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2534e5e)
test/rbd: fsx should utilize the default image features
4357ab5 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c1e111e)
qa/workunits/rbd: append rbd-mirror instance number for all-daemon as…
016607b 
…ok command

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 15a197e)
qa/workunits/rbd: don't override rbd-mirror logs if running under teu…
99b1f29 
…thology

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1bb6d4f)
qa/workunits/rbd: rbd-mirror admin socket commands should retry
97ce17a 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a6a8c8a)
qa/workunits/rbd: skip rbd-mirror start/stop cases when thrashing dae…
1922efc 
…mons

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3086f29)
qa/workunits/rbd: fixed missing error code on trap handler
44cbe94 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 02e32fe)
qa/workunits/rbd: fixed rbd-mirror snapshot comparison loop
17feb4c 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 917f8a0)
qa/suites/rbd: added active/active rbd-mirror tests
fda9036 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e2d5f84)
qa/suites/rbd: reduce runtime of rbd-mirror fsx test case
0116671 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 61e94e6)
cls/rbd: fixed last read key for interrupted mirror_image_map_list
ef0a3fb 
Previously, the image map would only return a maximum of 64 mappings.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3cbe0ce)
@dillaman dillaman added this to the mimic milestone May 20, 2018
@dillaman
Copy link
Author

http://pulpito.ceph.com/jdillaman-2018-05-20_08:17:31-rbd-wip-jd-testing-distro-basic-smithi/

@trociny
Copy link
Contributor

trociny commented May 21, 2018

@dillaman, the teuthology result contains one rbd_mirror_fsx_compare.sh failure. It failed waiting snapshot creation propagated to slave, because the replay was in error state in that time due to this error:

2018-05-20 12:44:45.338 7fd88d7fa700 -1 librbd::SnapshotUnprotectRequest: cannot unprotect: at least 1 child(ren) [108323436314] in pool 'mirror'
2018-05-20 12:44:45.338 7fd88d7fa700 -1 librbd::SnapshotUnprotectRequest: encountered error: (16) Device or resource busy
2018-05-20 12:44:45.338 7fd88d7fa700 -1 librbd::SnapshotUnprotectRequest: 0x7fd7ec1a10b0 should_complete_error: ret_val=-16
2018-05-20 12:44:45.338 7fd88d7fa700 -1 librbd::SnapshotUnprotectRequest: 0x7fd7ec1a10b0 should_complete_error: ret_val=-16
2018-05-20 12:44:45.338 7fd88d7fa700 -1 librbd::deep_copy::SnapshotCopyRequest: 0x7fd7ec13fc20 handle_snap_unprotect: failed to unprotect snapshot 'snap': (16) Device or resource busy
2018-05-20 12:44:45.338 7fd88d7fa700 -1 librbd::DeepCopyRequest: 0x7fd7ec575020 handle_copy_snapshots: failed to copy snapshot metadata: (16) Device or resource busy
2018-05-20 12:44:45.338 7fd88d7fa700 10 rbd::mirror::ImageSync: 0x7fd7b410fb20 handle_copy_image: r=-16
2018-05-20 12:44:45.338 7fd88d7fa700 -1 rbd::mirror::ImageSync: 0x7fd7b410fb20 handle_copy_image: failed to copy image: (16) Device or resource busy

Do you think we still may proceed and merge this to mimic?

@dillaman
Copy link
Author

@trociny Yeah, I think it's fine since it looks like it's just this known issue [1]

[1] http://tracker.ceph.com/issues/24140

@trociny trociny merged commit 063fb45 into ceph:mimic May 21, 2018
@dillaman dillaman deleted the wip-rbd-mirror-policy-mimic branch May 21, 2018 12:45
@luoguanzzz
Copy link

What caused this error?

[root@ceph-mon-node1 lgrbd0]# rbd snap unprotect rbd1/lgrbd2@lgrbd2.snap
2019-11-14 10:31:46.409 7f07f8ff9700 -1 librbd::SnapshotUnprotectRequest: cannot unprotect: at least 1 child(ren) [25a4e6b8b4567] in pool 'rbd1'
2019-11-14 10:31:46.412 7f07f8ff9700 -1 librbd::SnapshotUnprotectRequest: encountered error: (16) Device or resource busy
2019-11-14 10:31:46.412 7f07f8ff9700 -1 librbd::SnapshotUnprotectRequest: 0x563503d4c380 should_complete_error: ret_val=-16
2019-11-14 10:31:46.419 7f07f8ff9700 -1 librbd::SnapshotUnprotectRequest: 0x563503d4c380 should_complete_error: ret_val=-16
rbd: unprotecting snap failed: (16) Device or resource busy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@trociny trociny trociny approved these changes

Assignees
No one assigned
Labels
bug-fix feature rbd
Projects
None yet
Milestone
mimic
Development

Successfully merging this pull request may close these issues.
3 participants
@dillaman @trociny @luoguanzzz