calculate throughput corrections in a separate function and save the them to disk. change default to use sky correction

[akremin]

Moved calculation of throughput corrections into a separate function and save the factors to skymodel files.

This includes the creation of a new function in sky.py called calculate_throuhput_corrections() that is run at the end of compute_sky().

subtract_sky() uses the values, which are saved in a 1D array in skymodel. The index corresponds to the fiber number. Its value is 1 by default. The array is stored in skymodel.throughput_corrections

The array is saved in a new HDU within the skymodel fits file labeled THRPUTCORR. If that exists in a loaded file, it is assigned to skymodel.throughput_corrections otherwise None is assigned.

This PR also changes the default of the pipeline to use sky corrections in fiber throughput. It changes the input flag of procexp (desi_process_exposure) to --no-sky-throughput-correction and removes the use of the previous flag from desi_proc.

Checked with:
srun -N 1 -n 2 -c 32 /global/homes/k/kremin/workspace/default_skycorrect/desispec/bin/desi_proc --cameras 1 -n 20200314 -e 55444 --mpi --traceshift

It runs to completion.

The results are in:
/global/cfs/cdirs/desi/spectro/redux/kremin/exposures/20200314/00055444

[julienguy]

Good for me.

[akremin]

This addresses issue #938

[sbailey]

I haven't found the cause of the problem, but the throughput corrections don't look right:


I suspect that it is something to do with assumptions about fiber ordering, or maybe not applying correct order of calibs and masking prior to calculating throughput correction. Please investigate. It might be useful to go back to master and make a minimal change to save the original throughput correction in the sframe file, for comparison to this refactored throughput correction that is stored with the sky model itself (which in principle is better, but both the magnitude and structure are off here).

Other comments inline too.

[akremin]

If the above is completely due to the wavelength miscalibration seen in test reductions, then this PR is independent of that. I've discovered that the wavelength miscalibration is already present in master, which I've reported here: #941 .

A glance at the r1 throughput values show what seem to be more reasonable results:

New test exposures where that issue has been negated can be found in the same data directory (note that it's a different exposure than the original post):

/global/cfs/cdirs/desi/spectro/redux/kremin/preproc/20200314/0005545
/global/cfs/cdirs/desi/spectro/redux/kremin/exposures/20200314/0005545

All issues resolved except the wavelength issue that is already in master. Closing this and opening a new PR for that.