rgw: RGWMetaSyncCR holds refs to stacks instead of crs

[cbodley]

because RGWCoroutine::wakeup() calls RGWCoroutinesStack::wakeup(), the
stack must also stay alive. change the reference from
RGWMetaSyncShardControlCR to RGWCoroutinesStack and wake up the stacks
directly

Fixes: http://tracker.ceph.com/issues/16666

Signed-off-by: Casey Bodley cbodley@redhat.com

[yehudasa]

@cbodley looks fine, should get it through teuthology + test/rgw

[cbodley]

teuthology suite scheduled at http://pulpito.ceph.com/cbodley-2016-07-15_10:12:53-rgw-wip-rgw-meta-stack-wakeup---basic-mira/

[mattbenjamin]

So this is the version that takes a different ref, not 2 refs. Was this the most conservative option?

[cbodley]

So this is the version that takes a different ref, not 2 refs. Was this the most conservative option?

That was the plan we discussed yesterday, yeah. But I was able to goose the sync process to test this case directly yesterday, and verified that we don't need to keep the RGWMetaSyncShardControlCR around in order to call wakeup() on its RGWCoroutineStack.

I did this by modifying RGWMetaSyncShardCR to return an error on startup for all shards > 0, so that shard 0 would keep the parent RGWMetaSyncCR open after the other shards were drained and freed. I then created buckets on another gateway to trigger the mdlog_notify calls, and the ones that mapped onto the finished shards were successful - wakeup() was called on the stack without segfaulting.

@yehudasa does that make sense to you? if not, i'm happy to keep the existing reference on RGWMetaSyncShardControlCR to be safe

[mattbenjamin]

If holding the extra ref is harmless, it seems potentially sensible to do it at least temporarily, unless there seems like greater risk of not returning all refs. I'm not arguing for magic incantations, however :(

[yehudasa]

@cbodley @mattbenjamin yeah, I agree with Matt on this one. Just to be on the safe side, let's hold the extra ref for now, get it backported to Jewel like that and open a ticket to remind us to change it. Once we're happy with stability on master we can remove the second ref and see how things are going.

[cbodley]

updated to hold both refs. passing test_multi.py