Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2018.3] Fixes to scheduler, list of whens plus splay #50216

Merged

Conversation

Projects
None yet
3 participants
@garethgreenaway
Copy link
Member

commented Oct 25, 2018

What does this PR do?

Fixing an issue when a combination of the when parameter as a list plus using the splay parameter would cause the schedule to continuously run jobs in an endless loop, regardless of if their scheduled time had been receached. Also fixing a related issue where scheduled jobs that rely on _next_fire_time were not being run as the corrected splayed time but rather running at the original scheduled time. Adding new tests.

What issues does this PR fix or reference?

#50162

Tests written?

Yes

Commits signed with GPG?

Yes

Please review Salt's Contributing Guide for best practices.

See GitHub's page on GPG signing for more information about signing commits with GPG.

Fixing an issue when a combination of the when parameter as a list pl…
…us using the splay parameter would cause the schedule to continuously run jobs in an endless loop, regardless of if their scheduled time had been receached. Also fixing a related issue where scheduled jobs that rely on _next_fire_time were not being run as the corrected splayed time but rather running at the original scheduled time. Adding new tests.

@garethgreenaway garethgreenaway force-pushed the garethgreenaway:50162_when_plus_splay_endless_loop branch from 562ff98 to 685509f Oct 25, 2018

@rallytime rallytime merged commit 51b3fa4 into saltstack:2018.3 Oct 25, 2018

4 of 9 checks passed

continuous-integration/jenkins/pr-merge This commit cannot be built
Details
jenkins/pr/py2-windows-2016 The py2-windows-2016 job has failed
Details
jenkins/pr/py3-centos-7 The py3-centos-7 job has failed
Details
jenkins/pr/py3-ubuntu-1604 The py3-ubuntu-1604 job has failed
Details
jenkins/pr/py3-windows-2016 The py3-windows-2016 job has failed
Details
jenkins/pr/docs The docs job has passed
Details
jenkins/pr/lint The lint job has passed
Details
jenkins/pr/py2-centos-7 The py2-centos-7 job has passed
Details
jenkins/pr/py2-ubuntu-1604 The py2-ubuntu-1604 job has passed
Details

garethgreenaway added a commit to garethgreenaway/salt that referenced this pull request Oct 26, 2018

@rbthomp

This comment has been minimized.

Copy link
Contributor

commented Oct 29, 2018

@garethgreenaway Ok it looks like when I apply this patch in Fluorine I get two jobs that run. Not sure if this is related to the patch as I just get the endless looping without it.

{"fun_args": [], "jid": "20181029113700444628", "return": true, "retcode": 0, "success": true, "schedule": "test_schedule", "pid": 20297, "fun": "test.ping", "id": "LXOT-SALTTEST.afcucorp.test"}
{"fun_args": [], "jid": "20181029113700706003", "return": true, "retcode": 0, "success": true, "schedule": "test_schedule", "pid": 20304, "fun": "test.ping", "id": "LXOT-SALTTEST.afcucorp.test"}
@garethgreenaway

This comment has been minimized.

Copy link
Member Author

commented Oct 29, 2018

@rbthomp Thanks. Will take a look.

rallytime added a commit that referenced this pull request Oct 30, 2018

@garethgreenaway

This comment has been minimized.

Copy link
Member Author

commented Nov 1, 2018

@rbthomp Can double check fluorine again now that the patch has been merged? I'm not sure seeing double jobs when I'm running against the branch.

@rbthomp

This comment has been minimized.

Copy link
Contributor

commented Nov 1, 2018

@garethgreenaway I'm running the latest Fluorine build and we are back to looping. I've also notice that now when I run salt 'target' schedule.run_job test_job, which I was using to test the jobs running twice throws, this error now appears.

[ERROR   ] Multiple exceptions in yield list
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 656, in callback
    result_list.append(f.result())
  File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
    raise_exc_info(self._exc_info)
  File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 230, in wrapper
    yielded = next(result)
  File "/usr/lib/python2.7/site-packages/salt/minion.py", line 2358, in handle_event
    self.manage_schedule(tag, data)
  File "/usr/lib/python2.7/site-packages/salt/minion.py", line 2224, in manage_schedule
    self.schedule.run_job(name)
  File "/usr/lib/python2.7/site-packages/salt/utils/schedule.py", line 458, in run_job
    datetime.datetime.now())
  File "/usr/lib/python2.7/site-packages/salt/utils/schedule.py", line 207, in _check_max_running
    if not data['run']:
KeyError: u'run'
[ERROR   ] Exception in callback <functools.partial object at 0x7f8b14566100>
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/tornado/ioloop.py", line 591, in _run_callback
    ret = callback()
  File "/usr/lib64/python2.7/site-packages/tornado/stack_context.py", line 274, in null_wrapper
    return fn(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/tornado/ioloop.py", line 597, in <lambda>
    self.add_future(ret, lambda f: f.result())
  File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
    raise_exc_info(self._exc_info)
  File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 876, in run
    yielded = self.gen.throw(*exc_info)
  File "/usr/lib/python2.7/site-packages/salt/minion.py", line 958, in handle_event
    yield [minion.handle_event(package) for minion in self.minions]
  File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 870, in run
    value = future.result()
  File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
    raise_exc_info(self._exc_info)
  File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 656, in callback
    result_list.append(f.result())
  File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
    raise_exc_info(self._exc_info)
  File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 230, in wrapper
    yielded = next(result)
  File "/usr/lib/python2.7/site-packages/salt/minion.py", line 2358, in handle_event
    self.manage_schedule(tag, data)
  File "/usr/lib/python2.7/site-packages/salt/minion.py", line 2224, in manage_schedule
    self.schedule.run_job(name)
  File "/usr/lib/python2.7/site-packages/salt/utils/schedule.py", line 458, in run_job
    datetime.datetime.now())
  File "/usr/lib/python2.7/site-packages/salt/utils/schedule.py", line 207, in _check_max_running
    if not data['run']:
KeyError: u'run'```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.