Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix harvest job status update #2089

Merged

Conversation

@noirbizarre
Copy link
Member

commented Mar 27, 2019

This PR ensures that harvest jobs status updates are applied to the right job and prevent harvest jobs from being stuck in the initialized state.

In order to allow workers to finish processing tasks submitted with the source ID, this PR create new tasks and deprecate the old ones:

New task Deprecated equivalent
harvest_job_item(job_id, item_id) harvest_item(source_id, item_id)
harvest_job_finalize(results, job_id) harvest_finalize(results, source_id)

The old tasks do the following:

  • raise a deprecation warning
  • find the last job for the given source
  • call the new task with the job ID

Fixes this kind of harvest issues:

@noirbizarre noirbizarre requested a review from opendatateam/etalab Mar 27, 2019
@noirbizarre noirbizarre added this to the 1.6.6 milestone Mar 27, 2019
@abulte
abulte approved these changes Mar 27, 2019
CHANGELOG.md Outdated Show resolved Hide resolved
@noirbizarre noirbizarre force-pushed the noirbizarre:fix-harvest-job-status-update branch from 980b557 to 8791e1a Mar 27, 2019
Fix NEXT-DATAGOUVFR-18QF
Fix NEXT-DATAGOUVFR-8W
Fix WWW-DATAGOUVFR-19PN
Fix WWW-DATAGOUVFR-NS6
@noirbizarre noirbizarre force-pushed the noirbizarre:fix-harvest-job-status-update branch from 8791e1a to 53dfa3b Mar 27, 2019
@noirbizarre noirbizarre merged commit fedd78a into opendatateam:master Mar 27, 2019
3 checks passed
3 checks passed
ci/circleci: assets Your tests passed on CircleCI!
Details
ci/circleci: dist Your tests passed on CircleCI!
Details
ci/circleci: python Your tests passed on CircleCI!
Details
@noirbizarre noirbizarre deleted the noirbizarre:fix-harvest-job-status-update branch Mar 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.