Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notebook autodeploy wipe all existing deployed notebooks if github down #43

Closed
tlvu opened this issue Apr 23, 2020 · 0 comments · Fixed by #44
Closed

Notebook autodeploy wipe all existing deployed notebooks if github down #43

tlvu opened this issue Apr 23, 2020 · 0 comments · Fixed by #44
Assignees

Comments

@tlvu
Copy link
Collaborator

tlvu commented Apr 23, 2020

Should check if source exist before deleting dest.

Should even fail earlier when github download failed.

notebookdeploy START_TIME=2020-04-23T10:01:01-0400
++ mktemp -d -t notebookdeploy.XXXXXXXXXXXX
+ TMPDIR=/tmp/notebookdeploy.ICk70Vto2LaE
+ cd /tmp/notebookdeploy.ICk70Vto2LaE
+ mkdir tutorial-notebooks
+ cd tutorial-notebooks
+ wget --quiet https://raw.githubusercontent.com/Ouranosinc/PAVICS-e2e-workflow-tests/master/downloadrepos
+ chmod a+x downloadrepos
chmod: cannot access ‘downloadrepos’: No such file or directory
+ wget --quiet https://raw.githubusercontent.com/Ouranosinc/PAVICS-e2e-workflow-tests/master/default_build_params
+ wget --quiet https://raw.githubusercontent.com/Ouranosinc/PAVICS-e2e-workflow-tests/master/binder/reorg-notebooks
+ chmod a+x reorg-notebooks
chmod: cannot access ‘reorg-notebooks’: No such file or directory
+ wget --quiet --output-document - https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/archive/master.tar.gz
+ tar xz

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
+ ./downloadrepos
/etc/cron.hourly/PAVICS-deploy-notebooks: line 63: ./downloadrepos: No such file or directory
+ ./reorg-notebooks
/etc/cron.hourly/PAVICS-deploy-notebooks: line 64: ./reorg-notebooks: No such file or directory
+ mv -v 'PAVICS-e2e-workflow-tests-master/notebooks/*.ipynb' ./
mv: cannot stat ‘PAVICS-e2e-workflow-tests-master/notebooks/*.ipynb’: No such file or directory
+ rm -rfv PAVICS-e2e-workflow-tests-master
+ rm -rfv downloadrepos default_build_params reorg-notebooks
+ TMP_SCRIPT=/tmp/notebookdeploy.ICk70Vto2LaE/deploy-notebook
+ cat
+ chmod a+x /tmp/notebookdeploy.ICk70Vto2LaE/deploy-notebook
+ docker pull bash
Using default tag: latest
latest: Pulling from library/bash
Digest: sha256:febb3d74f41f2405fe21b7c7b47ca1aee0eda0a3ffb5483ebe3423639d30d631
Status: Image is up to date for bash:latest
+ docker run --rm --name deploy_tutorial_notebooks -u root -v /tmp/notebookdeploy.ICk70Vto2LaE/deploy-notebook:/deploy-notebook:ro -v /tmp/notebookdeploy.ICk70Vto2LaE/tutorial-notebooks:/tutorial-notebooks:ro -v /data/jupyterhub_user_data:/notebook_dir:rw --entrypoint /deploy-notebook bash
+ cd /notebook_dir
+ rm -rf tutorial-notebooks/WCS_example.ipynb tutorial-notebooks/WFS_example.ipynb tutorial-notebooks/WMS_example.ipynb tutorial-notebooks/WPS_example.ipynb tutorial-notebooks/catalog_search.ipynb tutorial-notebooks/dap_subset.ipynb tutorial-notebooks/esgf-compute-api-examples-devel tutorial-notebooks/esgf-dap.ipynb tutorial-notebooks/finch-usage.ipynb tutorial-notebooks/hummingbird.ipynb tutorial-notebooks/opendap.ipynb tutorial-notebooks/pavics_thredds.ipynb tutorial-notebooks/raven-master tutorial-notebooks/rendering.ipynb tutorial-notebooks/subsetting.ipynb
+ cp -rv '/tutorial-notebooks/*' tutorial-notebooks
cp: can't stat '/tutorial-notebooks/*': No such file or directory
+ chown -R root:root tutorial-notebooks
+ set +x
removed directory: ‘/tmp/notebookdeploy.ICk70Vto2LaE/tutorial-notebooks’
removed ‘/tmp/notebookdeploy.ICk70Vto2LaE/deploy-notebook’
removed directory: ‘/tmp/notebookdeploy.ICk70Vto2LaE’

notebookdeploy finished START_TIME=2020-04-23T10:01:01-0400
notebookdeploy finished   END_TIME=2020-04-23T10:02:12-0400
@tlvu tlvu changed the title notebook autodeploy wipe all existing deployed notebooks if github down Notebook autodeploy wipe all existing deployed notebooks if github down Apr 23, 2020
@tlvu tlvu self-assigned this Apr 23, 2020
@tlvu tlvu closed this as completed in #44 Apr 24, 2020
tlvu added a commit that referenced this issue Apr 24, 2020
…lready-deployed-nb-when-github-down


Fix notebook autodeploy wipe already deployed notebook when GitHub down.

Fixes #43

Fail early with any unexpected error to not wipe already deployed notebooks.

Check source dir not empty before wiping dest dir containing already deployed notebooks.

Reduce cleaning verbosity for more concise logging.

To fix this error found in production logs when Github is down today:
```
notebookdeploy START_TIME=2020-04-23T10:01:01-0400
++ mktemp -d -t notebookdeploy.XXXXXXXXXXXX
+ TMPDIR=/tmp/notebookdeploy.ICk70Vto2LaE
+ cd /tmp/notebookdeploy.ICk70Vto2LaE
+ mkdir tutorial-notebooks
+ cd tutorial-notebooks
+ wget --quiet https://raw.githubusercontent.com/Ouranosinc/PAVICS-e2e-workflow-tests/master/downloadrepos
+ chmod a+x downloadrepos
chmod: cannot access ‘downloadrepos’: No such file or directory
+ wget --quiet https://raw.githubusercontent.com/Ouranosinc/PAVICS-e2e-workflow-tests/master/default_build_params
+ wget --quiet https://raw.githubusercontent.com/Ouranosinc/PAVICS-e2e-workflow-tests/master/binder/reorg-notebooks
+ chmod a+x reorg-notebooks
chmod: cannot access ‘reorg-notebooks’: No such file or directory
+ wget --quiet --output-document - https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/archive/master.tar.gz
+ tar xz

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
+ ./downloadrepos
/etc/cron.hourly/PAVICS-deploy-notebooks: line 63: ./downloadrepos: No such file or directory
+ ./reorg-notebooks
/etc/cron.hourly/PAVICS-deploy-notebooks: line 64: ./reorg-notebooks: No such file or directory
+ mv -v 'PAVICS-e2e-workflow-tests-master/notebooks/*.ipynb' ./
mv: cannot stat ‘PAVICS-e2e-workflow-tests-master/notebooks/*.ipynb’: No such file or directory
+ rm -rfv PAVICS-e2e-workflow-tests-master
+ rm -rfv downloadrepos default_build_params reorg-notebooks
+ TMP_SCRIPT=/tmp/notebookdeploy.ICk70Vto2LaE/deploy-notebook
+ cat
+ chmod a+x /tmp/notebookdeploy.ICk70Vto2LaE/deploy-notebook
+ docker pull bash
Using default tag: latest
latest: Pulling from library/bash
Digest: sha256:febb3d74f41f2405fe21b7c7b47ca1aee0eda0a3ffb5483ebe3423639d30d631
Status: Image is up to date for bash:latest
+ docker run --rm --name deploy_tutorial_notebooks -u root -v /tmp/notebookdeploy.ICk70Vto2LaE/deploy-notebook:/deploy-notebook:ro -v /tmp/notebookdeploy.ICk70Vto2LaE/tutorial-notebooks:/tutorial-notebooks:ro -v /data/jupyterhub_user_data:/notebook_dir:rw --entrypoint /deploy-notebook bash
+ cd /notebook_dir
+ rm -rf tutorial-notebooks/WCS_example.ipynb tutorial-notebooks/WFS_example.ipynb tutorial-notebooks/WMS_example.ipynb tutorial-notebooks/WPS_example.ipynb tutorial-notebooks/catalog_search.ipynb tutorial-notebooks/dap_subset.ipynb tutorial-notebooks/esgf-compute-api-examples-devel tutorial-notebooks/esgf-dap.ipynb tutorial-notebooks/finch-usage.ipynb tutorial-notebooks/hummingbird.ipynb tutorial-notebooks/opendap.ipynb tutorial-notebooks/pavics_thredds.ipynb tutorial-notebooks/raven-master tutorial-notebooks/rendering.ipynb tutorial-notebooks/subsetting.ipynb
+ cp -rv '/tutorial-notebooks/*' tutorial-notebooks
cp: can't stat '/tutorial-notebooks/*': No such file or directory
+ chown -R root:root tutorial-notebooks
+ set +x
removed directory: ‘/tmp/notebookdeploy.ICk70Vto2LaE/tutorial-notebooks’
removed ‘/tmp/notebookdeploy.ICk70Vto2LaE/deploy-notebook’
removed directory: ‘/tmp/notebookdeploy.ICk70Vto2LaE’

notebookdeploy finished START_TIME=2020-04-23T10:01:01-0400
notebookdeploy finished   END_TIME=2020-04-23T10:02:12-0400
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant