test_object_broadcast is flaky since #5226

[simon-mo]

According to the travis tracker, test_object_manager.py::test_object_broadcast has been failing or unknown state since #5226

• Travis log: https://travis-ci.com/ray-project/ray/builds/124165213
• Tracker: https://ray-travis-tracker.herokuapp.com/

(cc @ericl @robertnishihara )

[ericl]

Yeah somehow it's only flaky on Linux.