-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #58 from desihub/spacewatch-etc
Spacewatch downloads and other updates
- Loading branch information
Showing
29 changed files
with
886 additions
and
576 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,2 @@ | ||
| include LICENSE.rst | ||
| include README.rst | ||
| include requirements.txt | ||
|
|
||
| graft bin | ||
| graft doc | ||
| graft etc | ||
|
|
||
| prune build | ||
| prune dist | ||
| prune htmlcov | ||
| prune doc/_build | ||
| prune .github | ||
| global-exclude .gitignore .readthedocs.yml |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| #!/usr/bin/env python | ||
| from sys import exit | ||
| from desitransfer.nightlog import main | ||
| from desitransfer.spacewatch import main | ||
| exit(main()) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| #!/bin/bash | ||
| # | ||
| # Parallel copy DESI mirror data, to catch up after outages. | ||
| # | ||
| if [[ -z "${DESISYNC_HOSTNAME}" ]]; then | ||
| echo "ERROR: DESISYNC_HOSTNAME is undefined!" | ||
| exit 1 | ||
| fi | ||
| if [[ -z "${DESI_ROOT}" ]]; then | ||
| echo "ERROR: DESI_ROOT is undefined!" | ||
| exit 1 | ||
| fi | ||
| syn="/usr/bin/rsync --archive --verbose --delete --delete-after --no-motd --password-file ${HOME}/.desi" | ||
| src=rsync://${DESISYNC_HOSTNAME}/desi | ||
| dst=${DESI_ROOT} | ||
| log_root=${HOME}/Documents/Logfiles | ||
|
|
||
| for d in engineering/focalplane engineering/focalplane/hwtables \ | ||
| spectro/data \ | ||
| spectro/redux/daily spectro/redux/daily/exposures spectro/redux/daily/preproc spectro/redux/daily/tiles \ | ||
| spectro/nightwatch/kpno spectro/staging/lost+found; do | ||
| case ${d} in | ||
| engineering/focalplane) priority='nice'; exclude='--exclude archive --exclude hwtables --exclude *.ipynb --exclude .ipynb_checkpoints' ;; | ||
| engineering/focalplane/hwtables) priority='nice'; exclude='--include *.csv --exclude *' ;; | ||
| spectro/data) priority=''; exclude='--exclude 2018* --exclude 2019* --exclude 2020* --exclude 2021* --exclude 2022*' ;; | ||
| spectro/redux/daily) priority=''; exclude='--exclude *.tmp --exclude attic --exclude exposures --exclude preproc --exclude temp --exclude tiles' ;; | ||
| spectro/redux/daily/exposures) priority=''; exclude='--exclude *.tmp' ;; | ||
| spectro/redux/daily/preproc) priority=''; exclude='--exclude *.tmp --exclude preproc-*.fits --exclude preproc-*.fits.gz' ;; | ||
| spectro/redux/daily/tiles) priority=''; exclude='--exclude *.tmp --exclude temp' ;; | ||
| *) priority='nice'; exclude='' ;; | ||
| esac | ||
| log=${log_root}/catchup_$(tr '/' '_' <<<${d}).log | ||
| [[ -f ${log} ]] || touch ${log} | ||
| echo "${priority} ${syn} ${exclude} ${src}/${d}/ ${dst}/${d}/ &>> ${log} &" | ||
| ${priority} ${syn} ${exclude} ${src}/${d}/ ${dst}/${d}/ &>> ${log} & | ||
| done | ||
|
|
||
| # log=${log_root}/catchup_engineering_focalplane.log | ||
| # [[ -f ${log} ]] || touch ${log} | ||
| # nice ${syn} --exclude archive --exclude hwtables --exclude \*.ipynb --exclude .ipynb_checkpoints \ | ||
| # ${src}/engineering/focalplane/ ${dst}/engineering/focalplane/ &>> ${log} & | ||
|
|
||
| # log=${log_root}/catchup_engineering_focalplane_hwtables.log | ||
| # [[ -f ${log} ]] || touch ${log} | ||
| # nice ${syn} --include \*.csv --exclude \* \ | ||
| # ${src}/engineering/focalplane/hwtables/ ${dst}/engineering/focalplane/hwtables/ &>> ${log} & | ||
|
|
||
| # log=${log_root}/catchup_spectro_data.log | ||
| # [[ -f ${log} ]] || touch ${log} | ||
| # ${syn} --exclude 2018\* --exclude 2019\* --exclude 2020\* --exclude 2021\* --exclude 2022\* \ | ||
| # ${src}/spectro/data/ ${dst}/spectro/data/ &>> ${log} & | ||
|
|
||
| # log=${log_root}/catchup_spectro_redux_daily.log | ||
| # [[ -f ${log} ]] || touch ${log} | ||
| # ${syn} --exclude \*.tmp --exclude attic --exclude exposures --exclude preproc --exclude temp --exclude tiles \ | ||
| # ${src}/spectro/redux/daily/ ${dst}/spectro/redux/daily/ &>> ${log} & | ||
|
|
||
| # log=${log_root}/catchup_spectro_redux_daily_exposures.log | ||
| # [[ -f ${log} ]] || touch ${log} | ||
| # ${syn} --exclude \*.tmp \ | ||
| # ${src}/spectro/redux/daily/exposures/ ${dst}/spectro/redux/daily/exposures/ &>> ${log} & | ||
|
|
||
| # log=${log_root}/catchup_spectro_redux_daily_preproc.log | ||
| # [[ -f ${log} ]] || touch ${log} | ||
| # ${syn} --exclude \*.tmp --exclude preproc-\*.fits --exclude preproc-\*.fits.gz \ | ||
| # ${src}/spectro/redux/daily/preproc/ ${dst}/spectro/redux/daily/preproc/ &>> ${log} & | ||
|
|
||
| # log=${log_root}/catchup_spectro_redux_daily_tiles.log | ||
| # [[ -f ${log} ]] || touch ${log} | ||
| # ${syn} --exclude \*.tmp --exclude temp \ | ||
| # ${src}/spectro/redux/daily/tiles/ ${dst}/spectro/redux/daily/tiles/ &>> ${log} & | ||
|
|
||
| # log=${log_root}/catchup_spectro_nightwatch_kpno.log | ||
| # [[ -f ${log} ]] || touch ${log} | ||
| # nice ${syn} \ | ||
| # ${src}/spectro/nightwatch/kpno/ ${dst}/spectro/nightwatch/kpno/ &>> ${log} & | ||
|
|
||
| # log=${log_root}/catchup_spectro_staging_lost+found.log | ||
| # [[ -f ${log} ]] || touch ${log} | ||
| # nice ${syn} \ | ||
| # ${src}/spectro/staging/lost+found/ ${dst}/spectro/staging/lost+found/ &>> ${log} & |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.