More modifications to the exposurs and tile completeness table

[julienguy]

• FAPRGM instead of FAFLAVOR, ZDONE=false/true instead of ZSTATUS
• new ordering of columns and rows
• check for zbest and coadd file and sets ZDONE=true if a least 8 petals.

tsnr-exposures.csv

NIGHT,EXPID,TILEID,SURVEY,FAPRGM,EXPTIME,EFFTIME_SPEC,GOALTIME,GOALTYPE,MINTFRAC,SEEING_ETC,EFFTIME_ETC,TSNR2_ELG,TSNR2_QSO,TSNR2_LRG,TSNR2_LYA,TSNR2_BGS,ELG_EFFTIME_DARK,BGS_EFFTIME_BRIGHT,LYA_EFFTIME_DARK
...
20210324,81857,81062,sv2,bright,835.3,45.5,150.0,bright,0.9,1.8,32.7,5.3,1.2,3.0,2.9,259.1,45.5,36.3,26.1
20210324,81859,81062,sv2,bright,835.3,70.5,150.0,bright,0.9,0.0,0.0,8.2,1.9,4.7,5.2,419.6,70.5,58.7,47.3
20210324,81860,81060,sv2,bright,1064.7,218.8,150.0,bright,0.9,0.0,0.0,25.4,5.6,14.4,13.8,1253.3,218.8,175.5,125.4
20210324,81861,81060,sv2,bright,1064.7,203.7,150.0,bright,0.9,0.0,0.0,23.7,5.3,13.5,14.2,1190.0,203.7,166.6,129.2
20210324,81897,80734,sv1,ssv,922.1,438.3,0.0,unknown,0.9,0.0,0.0,51.0,12.0,31.2,39.3,2893.2,438.3,405.1,357.0
20210324,81898,80734,sv1,ssv,922.0,598.0,0.0,unknown,0.9,0.0,0.0,69.5,16.9,44.8,59.0,4253.2,598.0,595.4,535.9
20210324,81899,80861,sv1,lrgqso,692.1,742.4,0.0,dark,0.9,0.0,0.0,86.3,20.6,55.6,56.8,5143.2,742.4,720.0,515.4
20210324,81900,80861,sv1,lrgqso,692.0,796.4,0.0,dark,0.9,0.0,0.0,92.6,22.0,59.9,58.1,5528.2,796.4,773.9,527.4
...

tiles.csv

TILEID,SURVEY,FAPRGM,NEXP,EXPTIME,EFFTIME_ETC,EFFTIME_SPEC,GOALTIME,OBSSTATUS,ZDONE,ELG_EFFTIME_DARK,BGS_EFFTIME_BRIGHT,LYA_EFFTIME_DARK,GOALTY
PE,MINTFRAC
...

80895,sv1,lrgqso2,5,3056.5,0.0,4351.6,4000.0,obsend,false,4351.6,4617.2,3798.9,dark,0.9
80896,sv1,elg,1,979.5,0.0,1936.1,4000.0,obsstart,false,1936.1,2076.3,1980.6,dark,0.9
80899,sv1,lrgqso2,5,2818.4,0.0,3032.3,4000.0,obsstart,false,3032.3,3186.4,2491.8,dark,0.9
80902,sv1,elg,5,2352.5,0.0,2234.2,4000.0,obsstart,false,2234.2,2318.7,1656.8,dark,0.9
80916,unknown,dithfocus,3,540.2,0.0,62.6,1000.0,obsstart,false,62.6,65.7,52.9,unknown,0.9
80917,unknown,unknown,3,540.2,0.0,0.0,1000.0,unknown,false,0.0,0.0,0.0,unknown,0.9
80918,unknown,unknown,2,360.1,0.0,0.0,1000.0,unknown,false,0.0,0.0,0.0,unknown,0.9
80919,unknown,unknown,2,360.1,0.0,0.0,1000.0,unknown,false,0.0,0.0,0.0,unknown,0.9
80920,unknown,unknown,1,180.1,0.0,0.0,1000.0,unknown,false,0.0,0.0,0.0,unknown,0.9
80921,unknown,unknown,1,180.1,0.0,0.0,1000.0,unknown,false,0.0,0.0,0.0,unknown,0.9
80922,unknown,unknown,1,34.7,0.0,0.0,1000.0,unknown,false,0.0,0.0,0.0,unknown,0.9
81056,sv2,bright,2,1313.8,317.7,350.2,150.0,obsend,true,405.8,350.2,248.2,bright,0.9
81057,sv2,bright,2,1336.5,0.0,288.5,150.0,obsend,true,345.2,288.5,187.4,bright,0.9
81058,sv2,bright,2,1405.7,0.0,365.2,150.0,obsend,true,441.8,365.2,238.0,bright,0.9
81059,sv2,bright,2,1337.7,222.7,273.5,150.0,obsend,true,329.2,273.5,180.9,bright,0.9
81060,sv2,bright,2,2129.5,0.0,342.1,150.0,obsend,true,422.6,342.1,254.6,bright,0.9
81061,sv2,bright,3,2466.8,0.0,421.6,150.0,obsend,true,528.3,421.6,276.1,bright,0.9
81062,sv2,bright,2,1670.6,0.0,95.0,150.0,obsstart,false,116.0,95.0,73.4,bright,0.9
81063,sv2,bright,2,1226.8,0.0,56.0,150.0,obsstart,false,62.8,56.0,53.5,bright,0.9

[coveralls]

Coverage decreased (-0.04%) to 28.911% when pulling 2945124 on tile-completeness-check-z into 6858bc3 on master.

[geordie666]

Two comments, but otherwise this looks good:

1. I thought we decided on FAPRGRM rather than FAPRGM?
2. My understanding was the FAPRGRM would correspond to PROGRAM from @schlafly's tiles file (and would therefore have values of BRIGHT, DARK, BACKUP). So, I guess this would be instead of what you have as GOALTYPE? Then we would retain FAFLAVOR to allow for some exotic "dedicated" secondary programs.

@schlafly should comment as well, though. If there is a perfect mapping between PROGRAM in @schlafly's tiles file and GOALTYPE in this tile completeness table, then I guess I could just use GOALTYPE.

[julienguy]

I will change FAPRGRM. GOALTYPE serves a different purpose (it is to make sure we agree on how we compute the effective) so I will leave it. Shall I add back FAFLAVOR?

[schlafly]

Yes, please keep GOALTYPE to future proof against potential other program types that want to use the same effective time computation but have a different name. Since FAFLAVOR == SURVEY + PROGRAM, there's not a need for both, though my vote is to track it.

[geordie666]

I think we do want FAFLAVOR to record the inevitable exotic tile types (dedicated secondary observations, Targets of Opportunity, etc.).

So, I think, to summarize:

• Add back FAFLAVOR.
• Change the column name FAPRGM to FAPRGRM.
• FAPRGRM should be the equivalent of PROGRAM from Eddie's tiles file so should be one of DARK, BRIGHT, BACKUP (my preference is upper-case as DARK, BRIGHT, BACKUP correspond to bit-names, but feel free to choose either of upper-case or lower-case and I'll make sure to always explicitly convert those strings to .upper())

[julienguy]

Thanks. I applied the modifications. I have also updated the files in /global/cfs/cdirs/desi/spectro/redux/daily/