New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug 1542390: rewrite bugzilla associations crontabber job #4883
Conversation
This rewrites the bugzilla association crontabber job as a Django command. This also adds some needed features in cronrun--it passes the last_success time now.
@@ -14,7 +14,7 @@ set -eo pipefail | |||
./socorro-cmd crontabber --job=archivescraper | |||
|
|||
# Insert data that's no longer on archive.mozilla.org | |||
./scripts/insert_missing_versions.py | |||
python ./scripts/insert_missing_versions.py |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops.
start_date = timezone.now() - datetime.timedelta(days=1) | ||
else: | ||
# Try to parse it as a datetime | ||
start_date = parse_datetime(start_date) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cronrun passes it in as a datetime.
try: | ||
cmd_kwargs = {} | ||
with self.lock_job(job_spec['cmd']): | ||
with self.lock_job(job_spec['cmd']): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I moved the lock_job
context outside all the exception handling. If exceptions happen, then the job will unlock.
if job_spec.get('backfill', False): | ||
# If "backfill" is in the spec, then we want to pass in | ||
# run_time as an argument | ||
cmd_kwargs['run_time'] = format_datetime(run_time) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These arguments get passed in as strings so the command code can parse them just like any argument.
The big difference here is direct sql code vs. using the |
self-r+ |
This rewrites the bugzilla association crontabber job as a Django command.
This also adds some needed features in cronrun--it passes the last_success
time now.