radosgw-admin: bucket sync status not 'caught up' during full sync

[cbodley]

the 'radosgw-admin bucket sync status' command will incorrectly report that the "bucket is caught up with source" if some shards are still on full sync

Fixes: http://tracker.ceph.com/issues/40806

[cbodley]

passed qa in http://pulpito.ceph.com/cbodley-2019-08-14_17:32:34-rgw-wip-cbodley-testing-distro-basic-smithi/

[smanjara]

Not sure if I'm reading the logs in [1] right but metadata sync status says full_sync is complete even when test-zone2 is behind master?

"full_sync":{"total":1,"complete":1}}
2019-08-14T18:14:01.964 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1565806425.768022_200.1

[1] http://qa-proxy.ceph.com/teuthology/cbodley-2019-08-14_17:32:34-rgw-wip-cbodley-testing-distro-basic-smithi/4214957/teuthology.log

[cbodley]

"full_sync":{"total":1,"complete":1}}

this part is saying that full sync had a total of 1 entry to sync, and it completed that 1 entry

2019-08-14T18:14:01.964 INFO:teuthology.orchestra.run.smithi203.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"bc5c1e02-bb2a-49fb-98c7-768910f99e7f","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}}

each shard shows state=1, which is rgw_meta_sync_marker::SyncState::IncrementalSync

2019-08-14T18:14:01.964 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1565806425.768022_200.1 target=

this says that master's log goes up to marker 1_1565806425.768022_200.1 for shard 0, but the target sync status has an empty marker for that shard, which is why it shows as behind

[cbodley]

thanks for review!