update_json for module intermediate comms - #13620
Conversation
699991d to
1ee4f9e
Compare
bcoca
changed the title
ansibot
added
affects_2.3
ansibot
added
needs_rebase
ansibot
added
module_util
needs_revision
jctanner
removed
needs_rebase
ansibot
added
WIP
needs_rebase
ansibot
added
needs_info
ansibot
removed
the
needs_rebase
|
Changes in this PR appear to be causing integration tests to run significantly longer than usual, with many of them reaching the 1 hour timeout on Shippable or the ~55 minute timeout on AWS. |
|
I hope I'm not stepping on anyones toes here since this is an active PR but I really want to see this merged so I went ahead and (hopefully) debugged it. See https://github.com/tsipinakis/ansible/commits/update_json @bcoca Could you pull my branch into the PR and see if that fixes the CI issues? I also went ahead and did an initial implementation of live updates on the local connection in the process of debugging. |
|
@tsipinakis feel free to push to this PR, im currently dealing with other things but I def want to get back to this .. if someone else beats me too it, GREAT! |
|
@bcoca I'm not a part of the org so you'll need to give me access to your fork for me to be able to update the PR. (Just tried and got access denied) |
|
you dont need to be part of the org, github allows pushing to specific PRs by default .. but this is very old branch and might predate that ... |
|
I think you're referring to the |
|
Whoops, that's what I get for not testing python3, pushed a fixup. Perhaps it'd be better for me to open a new PR for this? |
|
@tsipinakis What is the status of this pull request? Interested because this will fix #29403. |
|
@danieljaouen Paused until I get some more time to see why the tests are (still) failing. If you want to take it over go right ahead. |
|
@tsipinakis Perhaps @bcoca could shed some light on why the tests are failing? |
|
@tsipinakis @danieljaouen this PR needs to be rebased - the branch is 987 commits behind the current |
|
CI failure in unit tests and integration tests, as well as timeouts on many integration tests. |
|
!waffling stale_ci |
|
CI failure in unit tests: https://app.shippable.com/github/ansible/ansible/runs/99004/5/tests CI failure (timeout) in integration tests: https://app.shippable.com/github/ansible/ansible/runs/99004/60/console |
- new update_json method that sends 'intermediate messages'
- new internal flag that can enable this facility from controller
- method to run_command and log, log also requires debug to return data
- 'live' keyword, enables 'live updates' from modules that support it
- common method for connections to use fixes to module sending
- spilt json filtering to return trailing data vs warnings
- customized display
- added 'live' toggles for adhoc/console
- Fix performance degradation when running commands
When all the file descriptors are empty and closed we still have to
enter the if block to remove them from rpipes which means we will have
to call `select()` again on the next iteration and as a result wait for
the timeout unnecessarily which leads to a 1 second slowdown for each
command being run.
- Add live support to local connection
TODO:
- Ouput layer
- factor in async
- only disable buffer if live
- update stdin/stdout only if needed
12 participants
TODO:
implements ansible/proposals#92