Added image add mode run restore command

[ajstewart]

#443 should be merged first.

Adds a new CLI option to restore a pipeline run that has been run through add mode to the previous version as saved in the .bak files from add mode.

I thought this at least gives an admin power to 'undo' a pipeline run that someone might have added the wrong image or if something goes wrong during add mode the job can be saved and restored without much trouble. It waits for user confirmation before proceeding with a restore.

For now I decided this option shouldn't be available through the UI, only admins via CLI, but this could be added in the future.

Fixes #435.

Example:

./manage.py restorepiperun -v 3 new-test-data                                            ✔  7s  vast-pipeline-dev   00:53:14
2021-01-15 13:54:23,263 restorepiperun INFO Will restore the run to the following config:
PIPE_RUN_PATH...................................../Users/adam/GitHub/vast-pipeline/pipeline-runs/new-test-data
IMAGE_FILES.......................................['/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_0127-73A.EPOCH01.I.cutout.fits', '/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_2118+00A.EPOCH01.I.cutout.fits', '/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_2118-06A.EPOCH01.I.cutout.fits', '/Users/adam/askap/pipeline-test-data/EPOCH03x/VAST_2118-06A.EPOCH03x.I.cutout.fits', '/Users/adam/askap/pipeline-test-data/EPOCH03x/VAST_2118+00A.EPOCH03x.I.cutout.fits', '/Users/adam/askap/pipeline-test-data/EPOCH02/VAST_2118-06A.EPOCH02.I.cutout.fits', '/Users/adam/askap/pipeline-test-data/EPOCH02/VAST_2118+00A.EPOCH02.I.cutout.fits']
SELAVY_FILES......................................['/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_0127-73A.EPOCH01.I.cutout.components.txt', '/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_2118+00A.EPOCH01.I.cutout.components.txt', '/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_2118-06A.EPOCH01.I.cutout.components.txt', '/Users/adam/askap/pipeline-test-data/EPOCH03x/VAST_2118-06A.EPOCH03x.I.cutout.components.txt', '/Users/adam/askap/pipeline-test-data/EPOCH03x/VAST_2118+00A.EPOCH03x.I.cutout.components.txt', '/Users/adam/askap/pipeline-test-data/EPOCH02/VAST_2118-06A.EPOCH02.I.cutout.components.txt', '/Users/adam/askap/pipeline-test-data/EPOCH02/VAST_2118+00A.EPOCH02.I.cutout.components.txt']
BACKGROUND_FILES..................................['/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_0127-73A.EPOCH01.I.cutout_bkg.fits', '/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_2118+00A.EPOCH01.I.cutout_bkg.fits', '/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_2118-06A.EPOCH01.I.cutout_bkg.fits', '/Users/adam/askap/pipeline-test-data/EPOCH03x/VAST_2118-06A.EPOCH03x.I.cutout_bkg.fits', '/Users/adam/askap/pipeline-test-data/EPOCH03x/VAST_2118+00A.EPOCH03x.I.cutout_bkg.fits', '/Users/adam/askap/pipeline-test-data/EPOCH02/VAST_2118-06A.EPOCH02.I.cutout_bkg.fits', '/Users/adam/askap/pipeline-test-data/EPOCH02/VAST_2118+00A.EPOCH02.I.cutout_bkg.fits']
NOISE_FILES.......................................['/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_0127-73A.EPOCH01.I.cutout_rms.fits', '/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_2118+00A.EPOCH01.I.cutout_rms.fits', '/Users/adam/askap/pipeline-test-data/EPOCH01/VAST_2118-06A.EPOCH01.I.cutout_rms.fits', '/Users/adam/askap/pipeline-test-data/EPOCH03x/VAST_2118-06A.EPOCH03x.I.cutout_rms.fits', '/Users/adam/askap/pipeline-test-data/EPOCH03x/VAST_2118+00A.EPOCH03x.I.cutout_rms.fits', '/Users/adam/askap/pipeline-test-data/EPOCH02/VAST_2118-06A.EPOCH02.I.cutout_rms.fits', '/Users/adam/askap/pipeline-test-data/EPOCH02/VAST_2118+00A.EPOCH02.I.cutout_rms.fits']
SOURCE_FINDER.....................................selavy
MONITOR...........................................True
MONITOR_MIN_SIGMA.................................3.0
MONITOR_EDGE_BUFFER_SCALE.........................1.2
MONITOR_CLUSTER_THRESHOLD.........................3.0
MONITOR_ALLOW_NAN.................................False
ASTROMETRIC_UNCERTAINTY_RA........................1
ASTROMETRIC_UNCERTAINTY_DEC.......................1
ASSOCIATION_PARALLEL..............................True
ASSOCIATION_EPOCH_DUPLICATE_RADIUS................2.5
ASSOCIATION_METHOD................................basic
ASSOCIATION_RADIUS................................15.0
ASSOCIATION_DE_RUITER_RADIUS......................5.68
ASSOCIATION_BEAMWIDTH_LIMIT.......................1.5
NEW_SOURCE_MIN_SIGMA..............................5.0
DEFAULT_SURVEY....................................None
FLUX_PERC_ERROR...................................0
USE_CONDON_ERRORS.................................True
SELAVY_LOCAL_RMS_ZERO_FILL_VALUE..................0.2
CREATE_MEASUREMENTS_ARROW_FILES...................False
SUPPRESS_ASTROPY_WARNINGS.........................True
SOURCE_AGGREGATE_PAIR_METRICS_MIN_ABS_VS..........4.3
Would you like to restore the run ? (y/n): y
2021-01-15 13:54:26,246 restorepiperun INFO Restoring 'new-test-data' from backup parquet files.
2021-01-15 13:54:26,792 restorepiperun INFO Deleting new sources and associated objects to restore run Total objects deleted: 2475
2021-01-15 13:54:26,793 restorepiperun DEBUG (type, #deleted): {'vast_pipeline.RelatedSource': 15, 'vast_pipeline.Association': 553, 'vast_pipeline.MeasurementPair': 1809, 'vast_pipeline.Source': 98}
2021-01-15 13:54:27,090 restorepiperun INFO Deleting forced measurement and associated objects to restore run. Total objects deleted: 3769
2021-01-15 13:54:27,091 restorepiperun DEBUG (type, #deleted): {'vast_pipeline.Association': 453, 'vast_pipeline.MeasurementPair': 2459, 'vast_pipeline.Measurement': 857}
2021-01-15 13:54:27,275 restorepiperun INFO Deleting associations to restore run. Total objects deleted: 1127
2021-01-15 13:54:27,275 restorepiperun DEBUG (type, #deleted): {'vast_pipeline.Association': 1127}
2021-01-15 13:54:27,537 restorepiperun INFO Deleting measurement pairs to restore run. Total objects deleted: 5233
2021-01-15 13:54:27,537 restorepiperun DEBUG (type, #deleted): {'vast_pipeline.MeasurementPair': 5233}
2021-01-15 13:54:27,553 restorepiperun INFO Restore complete.

[ajstewart]

@srggrs see what you think of this. I wasn't sure if the functions should be placed somewhere else. See the original PR comment for details on how it works.

[srggrs]

Minor changes otherwise LGTM

[ajstewart]

Thanks, I've applied those changes. I will document this feature once #433 is in.

[srggrs]

all good