qa: introduce rbd-mirror thrasher to existing tests

[dillaman]

No description provided.

[dillaman]

http://pulpito.ceph.com/jdillaman-2018-04-19_18:50:18-rbd-master-distro-basic-smithi/
http://pulpito.ceph.com/jdillaman-2018-04-19_19:28:49-rbd-master-distro-basic-smithi/

[tchaikov]

retest this please

[trociny]

@dillaman not sure it is problem with the test or it reveals a real problem: I am observing "stop/start/restart mirror via admin socket" test fails after stopping an image replayer via admin socket.

In [1,2] it stopped the replayer on one instance, but when checking the status via admin socket another instance kept reporting it was running there. In [3] the status check via admin socket passed (no instance reported it running) but the status in mirror pool directory was reporting it 'up+replaying'.

[1] http://qa-proxy.ceph.com/teuthology/trociny-2018-04-21_06:50:46-rbd-wip-mgolub-testing-distro-basic-smithi/2425070/teuthology.log
[2] http://qa-proxy.ceph.com/teuthology/trociny-2018-04-21_18:49:24-rbd-wip-mgolub-testing-distro-basic-smithi/2425388/teuthology.log
[3] http://qa-proxy.ceph.com/teuthology/trociny-2018-04-21_06:50:46-rbd-wip-mgolub-testing-distro-basic-smithi/2425083/teuthology.log

[dillaman]

Tweaked it a little bit into two classes of tests: (1) daemons managed by teuthology for thrashed tests (which will skip anything related to stop/starting services, and (2) daemons managed by the workunits.

http://pulpito.ceph.com/jdillaman-2018-04-23_15:35:34-rbd-master-distro-basic-smithi/
http://pulpito.ceph.com/jdillaman-2018-04-23_16:17:33-rbd-master-distro-basic-smithi/

[trociny]

LGTM

[trociny]

@dillaman It looks like the thrasher was able to trigger a rbd-mirror daemon crash:
http://pulpito.ceph.com/trociny-2018-04-24_08:17:18-rbd-wip-mgolub-testing-distro-basic-smithi/2434398/

[dillaman]

@trociny ... just to close the loop, the crash was caused by a bug in the Objecter: #21649