Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix removing forced parquet and clear images from piperun #285

Merged
merged 14 commits into from
Aug 26, 2020

Conversation

srggrs
Copy link
Contributor

@srggrs srggrs commented Aug 25, 2020

This fix #211 and #210

@ajstewart can u test this to check the fixes are solved?

@srggrs srggrs marked this pull request as draft August 25, 2020 09:16
@github-actions github-actions bot added this to In progress in Pipeline Backlog Aug 25, 2020
@srggrs srggrs changed the title Fixes re run job Fix high priority tags Aug 25, 2020
@srggrs srggrs changed the title Fix high priority tags Fix high priority tags (WIP) Aug 25, 2020
@srggrs srggrs marked this pull request as ready for review August 25, 2020 10:30
@srggrs srggrs changed the title Fix high priority tags (WIP) Fix removing forced parquet and clear images from piperun Aug 25, 2020
@ajstewart
Copy link
Contributor

Be good to get #283 merged so I can test this with forced fits a bit smoother.

@srggrs
Copy link
Contributor Author

srggrs commented Aug 25, 2020

@ajstewart approved #283 so merge and I'll resolve the conflicts here if any

* master:
  Replace upload with function
  Fix source query
  Fixed upload indent
  removed flux_percentile field
  Update pipeline/models.py
  fix grammar error
  change to simple test case
  change mapping to srcs_df id column dict
  removed fields flux gain and its errors
  fix datatables for multi tables, add related table, minor fixes
  removed comment
  removed not used parameter
  removed unnecessary file
  fix upload function to use refactor bulk upload, minor changes
  remove dropping unque pairs, add dropping same related, small changes
  change to symmetrical False, add unique constraint
  fix related creation and upload, minor fixes
  added model utils with bulk update model function
  locking ipython version to avoid inf loops in dask when debug
  add m2m related source through, remove un-used fields

# Conflicts:
#	pipeline/pipeline/forced_extraction.py
@ajstewart
Copy link
Contributor

I can confirm that that this fixes #211.

However it does not fix #210 - the forced parquets still remain in the job directory when monitor is turned off on a re-run.

@srggrs
Copy link
Contributor Author

srggrs commented Aug 26, 2020

I can confirm that that this fixes #211.

However it does not fix #210 - the forced parquets still remain in the job directory when monitor is turned off on a re-run.

Thanks! I fix that, you can re-test now

ajstewart
ajstewart previously approved these changes Aug 26, 2020
Copy link
Contributor

@ajstewart ajstewart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now fixes #210 as well 👍.

DEVELOPING.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
Pipeline Backlog automation moved this from In progress to Reviewer approved Aug 26, 2020
Co-authored-by: Adam Stewart <ajstewart@users.noreply.github.com>
Pipeline Backlog automation moved this from Reviewer approved to Review in progress Aug 26, 2020
@srggrs srggrs requested a review from ajstewart August 26, 2020 02:15
Pipeline Backlog automation moved this from Review in progress to Reviewer approved Aug 26, 2020
@srggrs srggrs merged commit c107327 into master Aug 26, 2020
Pipeline Backlog automation moved this from Reviewer approved to Done Aug 26, 2020
@srggrs srggrs deleted the fixes-re-run-job branch August 26, 2020 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

Forced measurements and new sources is performed on all images if re-run with a subset
2 participants