test/pybind/test_rados.py: tolerate TimedOut in test_ping_monitor

[athanatos]

Fixes: http://tracker.ceph.com/issues/18529
Signed-off-by: Samuel Just sjust@redhat.com

[athanatos]

Tested: http://pulpito.ceph.com/samuelj-2017-01-14_22:21:18-rados-wip-sam-testing---basic-smithi/ failures seem unrelated

[tchaikov]

if self.rados.ping_monitor() times out, an exception would have been raised. but in this case, seems the pinged monitor returned an empty string, hence @None@ is returned. and it cannot be decoded by json.

i pasted the backtrace of the failed test at http://tracker.ceph.com/issues/18529#note-2

[jdurgin]

@tchaikov's analysis is correct - we're getting a 0 return value and empty output from ping_monitor(), not a TimedOut exception - I just reproduced this with ms inject socket failure = 30 on the client side.

I think it's fine to work around this in the test, since ping_monitor is a pretty specialized command - just need to try again if we get an empty response rather than an exception, and same for the bash cephtool test

[athanatos]

Updated to fix the unit test at least

Full run: http://pulpito.ceph.com/samuelj-2017-01-20_21:05:28-rados-wip-sam-testing---basic-smithi/

@jdurgin
Ready to merge.

[tchaikov]

in rados.pyx, if the returning length is 0, None is returned. so probably, we should instead check like:

if output:
   continue
buf = json.loads(output)

or be more specific:

if output is None:
   continue
buf = json.loads(output)

[athanatos]

Updated