Update KPNO transfer script #758
Conversation
|
It looks like this update is replacing the syncing of KPNO:/data/dts/exposures/raw -> NERSC:${DESI_SPECTRO_DATA} with a sync of KPNO:/exposures/desi/sps -> ${DESI_ROOT}/engineering/spectrograph/sps instead. Is the intension that at some point in the future we would switch back? There might be a time period when we are taking multi-spectrograph data into KPNO:/data/dts/exposures/raw while still finishing functional verification of the final spectrographs in KPNO:/exposures/desi/sps. i.e. it seems like for now we need to support syncing multiple locations on multiple timescales rather than replacing one sync with another. Caveat: I haven't understood why the spectrograph FV tests aren't being written to the final standard KPNO:/data/dts/exposures/raw location, or whether the reasons are technical or philosophical. |
|
@sbailey, there is no replacement going on. This script is for the non-essential transfers. The desi_dts.sh script is unchanged, and in fact that script has been continuously active for several weeks. This really seems to be a point of confusion. Did everyone except me forget that we have two different data transfer scripts? |
|
OK, that sounds fine. I was confused by It can be a separate update, but perhaps we need different names for these. From their names, |
|
@sbailey, thanx. I'm going to let the script run for another day or so before merging. I'll see if I can think of another name. |
|
Punting the script renaming issue to #761. Will merge as soon as tests pass. |
This PR updates the non-essential KPNO->NERSC transfer script. This now supports the spectrograph functional verification tests. In addition, this script is currently triggered directly from cron (once a day at noon MST) rather than running as a daemon, although that capability still exists.