Fix a race condition in manage runner

[terminalmage]

The runner was not connecting the client event listener to the event bus. This meant that all events between the client.run_job() and when client.get_cli_event_returns() began polling for events would be lost. Before get_cli_event_returns (via LocalClient's get_iter_returns) gets around to polling for events, it first checks to see if the job cache has a record of the jid it's querying. When using a custom returner for the job cache, one which has even a little bit of latency, return events from minions may sneak past before the jid lookup is complete, meaning that get_iter_returns will not return them and the manage runner will assume the minion did not respond.

Connecting to the event bus before we run the test.ping ensures that we do not miss any of these events.

Resolves #44820

NOTE you can test this fix by adding a time.sleep(5) after the client.run_job(). This will introduce enough latency between the run_job and get_cli_iter_returns to trigger the behavior. With the connect_pub line commented out, the minion will fail, even though with debug logging turned on you can clearly see the return come in. With the fix in place, the sleep does not prevent the return event from being processed by get_cli_iter_returns, and the return from salt-run manage.status is as expected.

[terminalmage]

@msteed the fix here (as well as the addition of the sleep to aid in confirmation) should be simple enough for you to try on the master you're using for testing purposes.

[terminalmage]

Actually I noticed that the run_job func has a listen arg that does the same thing I was doing manually in my initial commit. I just pushed a cleaner fix.

[msteed]

@terminalmage : Thanks, this works perfectly

[terminalmage]

👍