Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add jobs support #95

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions shakedown/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@
from shakedown.dcos.zookeeper import *
from shakedown.dcos.agent import *
from shakedown.dcos.master import *
from shakedown.dcos.jobs import *

VERSION='1.1.10'
54 changes: 54 additions & 0 deletions shakedown/dcos/jobs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
import spinner
import time
import urllib
from dcos import config, errors, http

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know Python's style rules. Should there be another newline here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added


def post_job(job):
return http.post(_metronome_url('v1/jobs'), json=job)


def get_job(job_id, history=True):
url = 'v1/jobs/{}'.format(job_id)
if history:
url = url + '?embed=history'
return http.get(_metronome_url(url))


def post_run(job_id):
return http.post(_metronome_url('v1/jobs/{}/runs').format(job_id))


def get_run(job_id, run_id):
return http.get(_metronome_url('v1/jobs/{}/runs/{}').format(job_id, run_id))


def run_job(job_spec, wait=True, assertSuccess=True):
job_id = post_job(job_spec).json()['id']
run_id = post_run(job_id).json()['id']
if wait:
wait_for_run(job_id, run_id)
if assertSuccess:
job = get_job(job_id).json()
assert(len(job['history']['successfulFinishedRuns']) > 0)
return job_id


def wait_for_run(job_id, run_id, duration=5):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look like duration is used anywhere.

def run_finished():
try:
response = get_run(job_id, run_id)
except errors.DCOSHTTPException as ex:
if ex.response.status_code == 404:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there somewhere that has a list of these constants? Also, I've seen a bunch of tests fail flakily due to occasional 404s returned lately. Are you sure you want to fail on 404? What if something like 502 is returned forever, does this just run forever?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I believe there's already a generic spin implementation somewhere in shakedown.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generic spinner: spinner.wait_for

return True
else:
raise ex
return False
spinner.wait_for(run_finished, ignore_exceptions=False)


def _metronome_url(url):
dcos_url = config.get_config_val('core.dcos_url')
return urllib.parse.urljoin(
urllib.parse.urljoin(dcos_url, 'service/metronome/'),
url)
7 changes: 3 additions & 4 deletions shakedown/dcos/spinner.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@


def wait_for(predicate, timeout_seconds=120, sleep_seconds=1, ignore_exceptions=True, inverse_predicate=False):
""" waits or spins for a predicate. Predicate is in function that returns a True or False.
An exception in the function will be returned.
A timeout will throw a TimeoutExpired Exception.

""" Waits for predicate to return True.
If predicate raises an exception and ignore_exceptions is true, the exception will be ignored.
A timeout will raise a TimeoutExpired Exception.
"""

timeout = Deadline.create_deadline(timeout_seconds)
Expand Down