New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly wait on returns in saltnado #48193

Merged
merged 2 commits into from Jun 22, 2018

Conversation

Projects
None yet
4 participants
@jacksontj
Contributor

jacksontj commented Jun 18, 2018

This was broken because the behavior was to simply check the ckminions
and wait for only those returns to complete. This works assuming
ckminions is accurate (which there are many cases where it isn't, such
as syndics).

_disbatch_local's waiting on returns needs to match LocalClient's
behavior (namely that in get_iter_returns). This means we are allowed to
return when (1) we have waitged the min_wait_time (0 if not a syndic)
(2) no minions are running the job (3) all minions we saw running it are
done running the job. The only method allowed for earlier termination is
if the gather_job_timeout is exceeded.

Fixes #42659

What does this PR do?

What issues does this PR fix or reference?

Previous Behavior

Remove this section if not relevant

New Behavior

Remove this section if not relevant

Tests written?

Yes/No

Commits signed with GPG?

Yes/No

Please review Salt's Contributing Guide for best practices.

See GitHub's page on GPG signing for more information about signing commits with GPG.

@salt-jenkins salt-jenkins requested a review from saltstack/team-netapi Jun 18, 2018

@rallytime rallytime requested a review from saltstack/team-core Jun 18, 2018

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented Jun 18, 2018

Thanks for the help with this issues @jacksontj

# Map of minion_id -> returned for all minions we think we need to wait on
minions = {}
for m in pub_data['minions']:
minions[m] = False

This comment has been minimized.

@gtmanfred

gtmanfred Jun 18, 2018

Contributor

we can just do

minions = {minion: False for minion in pub_data['minions']}

This comment has been minimized.

@jacksontj

jacksontj Jun 18, 2018

Contributor

@gtmanfred oh, did we drop python 2.6 support? IIRC dict comprehension isn't in 2.6

This comment has been minimized.

@gtmanfred

gtmanfred Jun 18, 2018

Contributor

yup, we dropped 2.6 starting with 2017.7.0

This comment has been minimized.

@jacksontj

jacksontj Jun 18, 2018

Contributor

Updated :)

@rallytime rallytime requested a review from DmitryKuzmenko Jun 19, 2018

@DmitryKuzmenko

Awesome!

@jacksontj jacksontj force-pushed the jacksontj:issue_42659 branch from 9ecc589 to 2dc2cdf Jun 19, 2018

@jacksontj

This comment has been minimized.

Contributor

jacksontj commented Jun 19, 2018

Squashed those commits, and rebased on current develop (in hopes of fixing some of the CI failures ;) )

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented Jun 19, 2018

@jacksontj looks like you definitely fixed the syndic.

First differing element 0:
{'minion': True, 'localhost': True}
{'minion': True, 'localhost': True, 'sub_minion': True}

- [{'localhost': True, 'minion': True}]
+ [{'localhost': True, 'minion': True, 'sub_minion': True}]
?                                ++++++++++++++++++++

We are getting the subminion returning on the saltnado syndic tests, can you add that to the test case?

Thanks,
Daniel

@jacksontj

This comment has been minimized.

Contributor

jacksontj commented Jun 19, 2018

When I run these locally I get a bunch of failures in the API tests with the localhost minion not responding. Actually, they fail with or without this patch, are these tests broken?

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented Jun 20, 2018

The localhost minion won't be there if you don't pass --ssh to start the extra sshd process that is used to test salt-ssh, and in this case testing enable_ssh_minions: True

@gtmanfred

This comment has been minimized.

Contributor

gtmanfred commented Jun 21, 2018

The tests look good to me now.

jacksontj added some commits Jun 18, 2018

Properly wait on returns in saltnado
This was broken because the behavior was to simply check the ckminions
and wait for only those returns to complete. This works assuming
ckminions is accurate (which there are many cases where it isn't, such
as syndics).

_disbatch_local's waiting on returns needs to match LocalClient's
behavior (namely that in get_iter_returns). This means we are allowed to
return when (1) we have waitged the min_wait_time (0 if not a syndic)
(2) no minions are running the job (3) all minions we saw running it are
done running the job. The only method allowed for earlier termination is
if the gather_job_timeout is exceeded.

Fixes #42659
Properly configure syndic in test case
Without this option we aren't really a syndic and it won't wait

@jacksontj jacksontj force-pushed the jacksontj:issue_42659 branch from 9f98575 to 10038f9 Jun 21, 2018

@rallytime rallytime merged commit 08735c9 into saltstack:develop Jun 22, 2018

5 of 10 checks passed

jenkins/PR/salt-pr-linode-ubuntu16-py3 Pull Requests » Salt PR - Linode Ubuntu16.04 - PY3 #10890 — ABORTED
Details
codeclimate 3 issues to fix
Details
default Build finished.
Details
jenkins/PR/salt-pr-linode-cent7-py3 Pull Requests » Salt PR - Linode CentOS 7 - PY3 #5919 — FAILURE
Details
jenkins/PR/salt-pr-rs-cent7-n Pull Requests » Salt PR - RS CentOS 7 #19972 — FAILURE
Details
WIP ready for review
Details
jenkins/PR/salt-pr-clone Pull Requests » Salt PR - Clone #26123 — SUCCESS
Details
jenkins/PR/salt-pr-docs-n Pull Requests » Salt PR - Docs #18175 — SUCCESS
Details
jenkins/PR/salt-pr-linode-ubuntu14-n Pull Requests » Salt PR - Linode Ubuntu14.04 #23848 — SUCCESS
Details
jenkins/PR/salt-pr-lint-n Pull Requests » Salt PR - Code Lint #22810 — SUCCESS
Details

@jacksontj jacksontj deleted the jacksontj:issue_42659 branch Jun 22, 2018

rallytime added a commit that referenced this pull request Jun 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment