fixes #10489 - adding two timeouts for content tasks

[jlsherrill]

The first will fail the task if the client does not pick up the task
this likely means that either the client is not running, goferd is not running,
or goferd is having some sort of communication issue. (Default 20 seconds)

The second will fail if the client has picked up the task but has not completed it
after some time. This could happen if the link is really really slow, or the client dies
in the middle of a content action. (Default to 60 minutes)

[jlsherrill]

requires Dynflow/dynflow#151

[ehelms]

What about making these admin configurable settings as opposed to ones that require server restart?

[jlsherrill]

One way to test the initial 'pickup' timeout is to simply shut down goferd on the client.
To test the larger timeout, decrease the 'content_action_finish_timeout' setting to something smaller (like 30 seconds), and patch /usr/lib/gofer/plugins/katelloplugin.py on a client (restarting goferd aftewards):

--- /usr/lib/gofer/plugins/katelloplugin.py.bak 2015-05-14 15:48:40.013998877 -0400
+++ /usr/lib/gofer/plugins/katelloplugin.py 2015-05-14 15:48:22.973999712 -0400
@@ -355,6 +355,8 @@
         """
         conduit = Conduit()
         dispatcher = Dispatcher()
+        import os
+        os.system("sleep 400")
         report = dispatcher.install(conduit, units, options)
         return report.dict()

for some reason time.sleep() did not work for me, but it may have been a serverside issue at the time.

[ehelms]

[test]

[ehelms]

@iNecas does this failure strike you? http://ci.theforeman.org/job/test_katello_core/13049/testReport/

[iNecas]

@ehelms it fails because https://github.com/Katello/katello/pull/5225/files#diff-3ba349775f9c0dd7cdf95b0fdc340077R19 passes '', instead of some number, that is expected: the settings needs to be set in testing environment to work properly

[ehelms]

@iNecas thanks, I am going to take this PR and attempt to fix the tests and will push a new PR with them if you don't mind doing the final review. I have already tested this change as is in a production environment.