S/N refactor and QA speed-up

[profxj]

Several offline QA related changes:

• Refactor the old QL QA on fitting S/N vs. mag
• Speed up QA in a few other ways
• Edits to for desi_qa_frame and desi_qa_exposure

Tested on simulated data (mini test 19.9) which has proper fibermaps.

QA is now generated in ~8min on one core for one night of real data (without the individual figures)
using this command on NERSC:

desi_qa_prod --make_frameqa 1 --specprod_dir /global/projecta/projectdirs/desi/spectro/redux/daily --night 20200224 --qaprod_dir /global/projecta/projectdirs/desi/spectro/redux/xavier/daily/QA

I expect exposures with more successful data will increase this to a bit more than 10min per core.

[sbailey]

Thanks for the speedup. This may be user-error, but I wasn't successful in using this as the starting point for then running desi_qa_night:

time desi_qa_prod --make_frameqa 3     --specprod_dir /global/cfs/cdirs/desi/spectro/redux/minisv2     --night 20200219 --qaprod_dir $SCRATCH/desi/temp/qa

desi_qa_night --expid_series  --specprod_dir /global/cfs/cdirs/desi/spectro/redux/minisv2     --night 20200219 --qaprod_dir $SCRATCH/desi/temp/qa

...

INFO:qa_night.py:42:main: starting
INFO:qa.py:228:load_qa_multiexp: Loading QA prod file: /global/cscratch1/sd/sjbailey/desi/temp/qa/20200219_qa.json
INFO:qa.py:231:load_qa_multiexp: QA prod file /global/cscratch1/sd/sjbailey/desi/temp/qa/20200219_qa.json does not exist!
ERROR:qa.py:232:load_qa_multiexp: You probably need to generate it with desi_qa_prod --make_frameqa=3 --slurp
Traceback (most recent call last):
  File "/global/common/software/desi/users/sjbailey/desispec/bin/desi_qa_night", line 16, in <module>
    qa_night.main(args)
  File "/global/common/software/desi/users/sjbailey/desispec/py/desispec/scripts/qa_night.py", line 58, in main
    qa_prod.load_data()
  File "/global/common/software/desi/users/sjbailey/desispec/py/desispec/qa/qa_multiexp.py", line 174, in load_data
    self.data = load_qa_multiexp(inroot)
  File "/global/common/software/desi/users/sjbailey/desispec/py/desispec/io/qa.py", line 233, in load_qa_multiexp
    with open(infile, 'rt') as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/global/cscratch1/sd/sjbailey/desi/temp/qa/20200219_qa.json'

It is unclear how 20200219_qa.json was supposed to be made.

User error, or new bug?

[profxj]

Two user issues in the first line:

1. Use --make_frameqa 1 or you will generate all the sub-plots (not needed)
2. Add --slurp. This is why the 2nd command failed.

I just ran these on cori and the first command took real 8min and an additional real 1min
for the --slurp. The second command took ~10s. Here are the two recommended
commands again:

desi_qa_prod --make_frameqa 1     --specprod_dir /global/cfs/cdirs/desi/spectro/redux/minisv2     --night 20200219 --qaprod_dir $SCRATCH/desi/temp/qa --slurp

desi_qa_night --expid_series  --specprod_dir /global/cfs/cdirs/desi/spectro/redux/minisv2     --night 20200219 --qaprod_dir $SCRATCH/desi/temp/qa

Updated the QA doc to --make_frameqa 1 instead of 3.

[profxj]

FWIW, here is an example of exposure level S/N QA from the minisv2

https://portal.nersc.gov/cfs/desi/users/xavier/TMP2_QA/exposure_s2n.png

using this command:

desi_qa_exposure --expid 51031 --specprod_dir /global/cfs/cdirs/desi/spectro/redux/minisv2 --qatype s2n --qaprod_dir $SCRATCH/desi/temp/qa --qamulti_root 20200219_qa

Just a test of the S/N refactor in this PR.