New docker build for updated clisops, ravenpy, xclim and add xkcd font for matplotlib #68
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ch binaries Follow same recipe as in Ouranosinc/raven#369 To fix `pip install ravenpy --install-option="--with-binaries"` error `error: option --with-binaries not recognized`.
New build from `Dockerfile.update` instead of `Dockerfile`.
Include equivalent of 2 previous commits:
* docker: add fonts-humor-sans for XKCD fonts for matplotlib plotting package
See Ouranosinc/pavics-sdi#204
* docker: install ravenpy using conda to have pre-built raven and ostrich binaries
Follow same recipe as in Ouranosinc/raven#369
To fix `pip install ravenpy --install-option="--with-binaries"` error
`error: option --with-binaries not recognized`.
Dependency problems from regular `Dockerfile`. A bunch of those, this is just the end:
```
Package distributed conflicts for:
scikit-image -> dask[version='>=0.5'] -> distributed[version='>=1.11.0|>=1.15.0|>=1.16.0|>=1.19.0|>=1.20.0|>=1.21.0|>=1.22.0|>=1.23.0|>=1.23.1|>=1.23.2|>=1.23.3|>=1.24.2|>=1.25.0|>=1.25.2|>=1.25.3|>=1.26.0|>=1.26.1|>=1.27.0|>=1.28.0|>=2|>=2.10.0|>=2.11.0|>=2.12.0|>=2.13.0|>=2.14.0|>=2.15.0|>=2.16.0|>=2.17.0|>=2.18.0|>=2.19.0|>=2.20.0|>=2.21.0|>=2.22.0|>=2.23.0|>=2.24.0|>=2.25.0|>=2.26.0|>=2.27.0|>=2.28.0|>=2.29.0|>=2.30.0|>=2020.12.0|>=2021.1.0|>
=2021.1.1|>=2021.2.0|>=2021.3.0|>=2021.3.1|>=2.9.2|>=2.9.1|>=2.9.0|>=2.8.1|>=2.8.0|>=2.7.0|>=2.6.0|>=2.5.2|>=2.5.0|>=2.4.0|>=2.3.0|>=2.2.0|>=1.21.1']
xclim -> dask[version='>=2.6'] -> distributed[version='>=1.27.0|>=1.28.0|>=2|>=2.10.0|>=2.11.0|>=2.12.0|>=2.13.0|>=2.14.0|>=2.15.0|>=2.16.0|>=2.17.0|>=2.18.0|>=2.19.0|>=2.20.0|>=2.21.0|>=2.22.0|>=2.23.0|>=2.24.0|>=2.25.0|>=2.26.0|>=2.27.0|>=2.28.0|>=2.29.0|>=2.30.0|>=2020.12.0|>=2021.1.0|>=2021.1.1|>=2021.2.0|>=2021.3.0|>=2021.3.1|>=2.9.2|>=2.9.1|>=2.9.0|>=2.8.1|>=2.8.0|>=2.7.0|>=2.6.0|>=2.5.2|>=2.5.0|>=2.4.0|>=2.3.0|>=2.2.0']
xarray -> dask[version='>=0.9'] -> distributed[version='>=1.11.0|>=1.15.0|>=1.16.0|>=1.19.0|>=1.20.0|>=1.21.0|>=1.22.0|>=1.23.0|>=1.23.1|>=1.23.2|>=1.23.3|>=1.24.2|>=1.25.0|>=1.25.2|>=1.25.3|>=1.26.0|>=1.26.1|>=1.27.0|>=1.28.0|>=2|>=2.10.0|>=2.11.0|>=2.12.0|>=2.13.0|>=2.14.0|>=2.15.0|>=2.16.0|>=2.17.0|>=2.18.0|>=2.19.0|>=2.20.0|>=2.21.0|>=2.22.0|>=2.23.0|>=2.24.0|>=2.25.0|>=2.26.0|>=2.27.0|>=2.28.0|>=2.29.0|>=2.30.0|>=2020.12.0|>=2021.1.0|>=2021.
1.1|>=2021.2.0|>=2021.3.0|>=2021.3.1|>=2.9.2|>=2.9.1|>=2.9.0|>=2.8.1|>=2.8.0|>=2.7.0|>=2.6.0|>=2.5.2|>=2.5.0|>=2.4.0|>=2.3.0|>=2.2.0|>=1.21.1']
Package libedit conflicts for:
netcdf4 -> openssh -> libedit[version='>=3.1.20191231,<3.2.0a0']
curl -> krb5[version='>=1.17.1,<1.18.0a0'] -> libedit[version='>=3.1.20170329,<3.2.0a0|>=3.1.20170329,<4.0a0|>=3.1.20191231,<3.2.0a0|>=3.1.20191231,<4.0a0|>=3.1.20181209,<3.2.0a0|>=3.1.20181209,<4.0a0']
Package cryptography conflicts for:
cdms2 -> pyopenssl -> cryptography[version='>=1.3|>=1.3.4|>=1.9|>=2.1.4|>=2.2.1|>=2.8|>=3.2']
jupyterhub -> oauthlib[version='>=3.0'] -> cryptography
esgf-compute-api -> oauthlib -> cryptography
Package libsodium conflicts for:
bokeh -> pyzmq -> libsodium[version='>=1.0.16,<1.0.17.0a0|>=1.0.17,<1.0.18.0a0|>=1.0.18,<1.0.19.0a0']
ipyleaflet -> pyzmq[version='>=15.2.0'] -> libsodium[version='>=1.0.16,<1.0.17.0a0|>=1.0.17,<1.0.18.0a0|>=1.0.18,<1.0.19.0a0']
xeus-python -> zeromq[version='>=4.3.3,<4.4.0a0'] -> libsodium[version='>=1.0.16,<1.0.17.0a0|>=1.0.17,<1.0.18.0a0|>=1.0.18,<1.0.19.0a0']
notebook -> pyzmq[version='>=17'] -> libsodium[version='>=1.0.16,<1.0.17.0a0|>=1.0.17,<1.0.18.0a0|>=1.0.18,<1.0.19.0a0']
Package mesalib conflicts for:
mesalib
vcs -> vtk-cdat -> mesalib[version='>17']
Package numba conflicts for:
xclim -> numba
geoviews -> datashader -> numba[version='>=0.35.0|>=0.37.0|>=0.37.0,!=0.49.*,!=0.50.*|>=0.37.0,<0.49']
ravenpy -> xclim[version='>=0.23'] -> numba[version='>=0.52']
Package soupsieve conflicts for:
siphon -> beautifulsoup4 -> soupsieve[version='>=1.2']
threddsclient -> beautifulsoup4 -> soupsieve[version='>=1.2']
pydap -> beautifulsoup4 -> soupsieve[version='>=1.2']
Package twisted conflicts for:
jupyterhub -> prometheus_client[version='>=0.0.21'] -> twisted
notebook -> prometheus_client -> twisted
Package ptyprocess conflicts for:
jupyterlab-git -> pexpect -> ptyprocess[version='>=0.5']
notebook -> terminado[version='>=0.8.3'] -> ptyprocess
Package entrypoints conflicts for:
jupyter -> nbconvert -> entrypoints[version='>=0.2.2']
jupyterhub -> entrypoints
notebook -> jupyter_client[version='>=5.3.4'] -> entrypoints[version='>=0.2.2']
nbval -> jupyter_client -> entrypoints
voila -> nbconvert[version='>=6.0.0,<7'] -> entrypoints[version='>=0.2.2']
Package networkx conflicts for:
geopandas -> mapclassify -> networkx
scikit-image -> networkx[version='>=1.8,<2.0|>=1.8|>=2.0']
Package nbdime conflicts for:
jupyterlab-git -> nbdime[version='>=1.1.0|>=2.0.0|>=2.0.0,<3.0.0']
nbdime
Package cdms2 conflicts for:
vcs -> genutil -> cdms2[version='>=3.1.2|>=3.1.3']
cdms2
vcs -> cdms2[version='>=3.1.4']
Package scikit-image conflicts for:
geoviews -> datashader -> scikit-image
scikit-image
Package lancet conflicts for:
geoviews -> holoviews[version='>=1.9.0'] -> lancet
holoviews -> lancet
Package greenlet conflicts for:
jupyterhub -> sqlalchemy[version='>=1.1'] -> greenlet!=0.4.17
geopandas -> sqlalchemy -> greenlet!=0.4.17
Package nomkl conflicts for:
cdms2 -> openblas -> nomkl==3.0[build='0|h06a4308_0']
numpy -> openblas[version='>=0.3.3,<0.3.4.0a0'] -> nomkl==3.0[build='0|h06a4308_0']The following specifications were found to be incompatible with your system:
- feature:/linux-64::__glibc==2.28=0
- feature:|@/linux-64::__glibc==2.28=0
- jupyter-archive -> nodejs -> __glibc[version='>=2.17,<3.0.a0']
- jupyterhub -> nodejs[version='>=12'] -> __glibc[version='>=2.17,<3.0.a0']
- jupyterlab-git -> nodejs -> __glibc[version='>=2.17,<3.0.a0']
Your installed version is: 2.28
The command '/bin/sh -c umask 0000 && conda env create -f /environment.yml' returned a non-zero code: 1
```
…there Error with conda ravenpy: ``` UnsatisfiableError: The following specifications were found to be incompatible with the existing python installation in your environment: Specifications: - ravenpy -> python[version='>=3.8,<3.9.0a0'] Your python: python=3.7 ```
…t dependency"
Without using 'update' only these 2 packages where updated.
```
The following packages will be downloaded:
package | build
---------------------------|-----------------
openssl-1.1.1k | h7f98852_0 2.1 MB conda-forge
xclim-0.25.0 | pyhd8ed1ab_0 223 KB conda-forge
------------------------------------------------------------
Total: 2.3 MB
The following packages will be UPDATED:
openssl 1.1.1i-h7f98852_0 --> 1.1.1k-h7f98852_0
xclim 0.23.0-pyhd8ed1ab_0 --> 0.25.0-pyhd8ed1ab_0
```
With 'update' we have clisops and also a whole bunch of other packages. My
hypothesis is 'install' only update packages specified in `environment.yml`
while 'update' will update all indirect dependent packages.
```
clisops 0.5.1-pyhd3deb0d_0 --> 0.6.3-pyh44b312d_0
```
…0 exists ravenpy has to be in a different `conda install` command since it has not previously been installed yet. `conda update` is for existing already installed packages only. Not working yet, getting this error: ``` ERROR conda.core.link:_execute(698): An error occurred while installing package 'zeitsperre::ravenpy-0.3.1-py37_0'. Rolling back transaction: ...working... done [Errno 13] Permission denied: '/opt/conda/envs/birdy/lib/python3.7/site-packages/ravenpy/cli' () The command '/bin/sh -c conda update -c zeitsperre -c conda-forge -c cdat -c bokeh -c plotly -c defaults -n birdy clisops xclim && conda install -c zeitsperre -c conda-forge -c cdat -c bokeh -c plotly -c defaults -n birdy ravenpy' returned a non-zero code: 1 ```
To fix this error: ``` ERROR conda.core.link:_execute(698): An error occurred while installing package 'zeitsperre::ravenpy-0.3.1-py37_0'. Rolling back transaction: ...working... done [Errno 13] Permission denied: '/opt/conda/envs/birdy/lib/python3.7/site-packages/ravenpy/cli' () The command '/bin/sh -c conda update -c zeitsperre -c conda-forge -c cdat -c bokeh -c plotly -c defaults -n birdy clisops xclim && conda install -c zeitsperre -c conda-forge -c cdat -c bokeh -c plotly -c defaults -n birdy ravenpy' returned a non-zero code: 1 ```
…r-clisops-and-add-xkcd-font-for-matplotlib
|
@Zeitsperre a little bit odd that That new Jupyter env is deployed on https://medus.ouranos.ca/jupyter, could you do a quick spot check if it is really there or let me know how to spot check for new functionality in the new ravenpy? This is not a clean fresh build unfortunately. We |
|
@Zeitsperre I |
It works, new ravenpy now shows up on |
tlvu
added a commit
to bird-house/birdhouse-deploy
that referenced
this pull request
Apr 8, 2021
See PR Ouranosinc/PAVICS-e2e-workflow-tests#68 for more info. Relevant changes: ```diff < - clisops=0.5.1=pyhd3deb0d_0 > - clisops=0.6.3=pyh44b312d_0 < - xclim=0.23.0=pyhd8ed1ab_0 > - xclim=0.25.0=pyhd8ed1ab_0 > - ostrich=0.1.2=h2bc3f7f_0 > - raven=0.1.1=h2bc3f7f_0 < - ravenpy==0.2.3 # from pip > - ravenpy=0.3.1=py37_0 # from conda > - aiohttp=3.7.4=py37h5e8e339_0 < - roocs-utils=0.1.5=pyhd3deb0d_1 > - roocs-utils=0.3.0=pyh6c4a22f_0 < - cf_xarray=0.4.0=pyh44b312d_0 > - cf_xarray=0.5.1=pyh44b312d_0 < - geopandas=0.8.2=pyhd8ed1ab_0 > - geopandas=0.9.0=pyhd8ed1ab_0 < - python=3.7.9=hffdb5ce_100_cpython > - python=3.7.10=hffdb5ce_100_cpython ```
tlogan2000
approved these changes
Apr 8, 2021
tlvu
added a commit
to Ouranosinc/pavics-sdi
that referenced
this pull request
Apr 8, 2021
…er env New Jupyter env Ouranosinc/PAVICS-e2e-workflow-tests#68. Fix following error: ``` _______ pavics-sdi-master/docs/source/notebooks/regridding.ipynb::Cell 5 _______ Notebook cell execution failed Cell 5: Cell outputs differ Input: ds_in.cf.describe() Traceback: mismatch 'stdout' assert reference_output == test_output failed: "Axes:\n\tX: ...'siconc']\n\n" == "Coordinates:...'siconc']\n\n" + Axes: + X: [] + Y: [] + Z: [] + T: ['time'] + Coordinates: - - CF Axes: * T: ['time'] - X, Y, Z: n/a + longitude: ['longitude'] + latitude: ['latitude'] + vertical: [] + time: ['time'] + Cell Measures: + area: ['areacello'] + volume: [] - - CF Coordinates: longitude: ['longitude'] - latitude: ['latitude'] - * time: ['time'] - vertical: n/a - - Cell Measures: area, volume: n/a + Standard Names: + area_type: ['type'] + cell_area: ['areacello'] + sea_ice_area_fraction: ['siconc'] - - Standard Names: area_type: ['type'] - latitude: ['latitude'] - longitude: ['longitude'] - * time: ['time'] - - Data Variables: - - Cell Measures: area: ['areacello'] - volume: n/a - - - Standard Names: cell_area: ['areacello'] - sea_ice_area_fraction: ['siconc'] - ```
This was referenced Apr 8, 2021
Merged
Even weirder is the full build also works and also on DockerHub. Note DockerHub build have not been working since Sept 2020, see #51 Got pavics/workflow-tests:210408 built by DockerHub.
tlvu
added a commit
to bird-house/birdhouse-deploy
that referenced
this pull request
Apr 8, 2021
This update is from a full clean build from scratch. The previous one was an update build from the previous release. See PR Ouranosinc/PAVICS-e2e-workflow-tests#68 for more info. Relevant changes: ```diff < - clisops=0.5.1=pyhd3deb0d_0 > - clisops=0.6.3=pyh44b312d_0 < - xclim=0.23.0=pyhd8ed1ab_0 > - xclim=0.25.0=pyhd8ed1ab_0 > - ostrich=0.1.2=h2bc3f7f_0 > - raven=0.1.1=h2bc3f7f_0 < - ravenpy==0.2.3 # from pip > - ravenpy=0.3.1=py37_0 # from conda > - aiohttp=3.7.4=py37h5e8e339_0 < - roocs-utils=0.1.5=pyhd3deb0d_1 > - roocs-utils=0.3.0=pyh6c4a22f_0 < - cf_xarray=0.4.0=pyh44b312d_0 > - cf_xarray=0.5.1=pyh44b312d_0 < - rioxarray=0.2.0=pyhd8ed1ab_0 > - rioxarray=0.3.1=pyhd8ed1ab_0 < - xarray=0.16.2=pyhd8ed1ab_0 > - xarray=0.17.0=pyhd8ed1ab_0 < - geopandas=0.8.2=pyhd8ed1ab_0 > - geopandas=0.9.0=pyhd8ed1ab_0 < - gdal=3.1.4=py37h2ec2946_5 > - gdal=3.2.1=py37hc5bc4e4_7 < - jupyter_conda=4.1.0=hd8ed1ab_1 > - jupyter_conda=5.0.0=hd8ed1ab_0 < - python=3.7.9=hffdb5ce_100_cpython > - python=3.7.10=hffdb5ce_100_cpython ```
tlvu
added a commit
to Ouranosinc/pavics-sdi
that referenced
this pull request
Apr 8, 2021
New Jupyter env from Ouranosinc/PAVICS-e2e-workflow-tests#68 Switch to use xarray html display at the same time together with NBVAL_IGNORE_OUTPUT. Fix the following error: ``` _ pavics-sdi-update-regridding-ipynb-for-new-jupyter-env/docs/source/notebooks/pavics_thredds.ipynb::Cell 2 _ Notebook cell execution failed Cell 2: Cell outputs differ Input: import requests from requests_magpie import MagpieAuth secured_url = "https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/testdata/secure/tasmax_Amon_MPI-ESM-MR_rcp45_r2i1p1_200601-200612.nc" auth = MagpieAuth("https://pavics.ouranos.ca/magpie", "authtest", "authtest1234") # Open session session = requests.Session() session.auth = auth # Open a Pydap data store and pass it to xarray store = xr.backends.PydapDataStore.open(secured_url, session=session) ds = xr.open_dataset(store, decode_cf=False) # Attributes are problematic with this file. ds Traceback: mismatch 'text/html' assert reference_output == test_output failed: '<pre><xar...: time</pre>' == '<pre><xar...: time</pre>' Skipping 625 identical leading characters in diff, use -v to show - ttributes: (12/28) + ttributes: institution: Max Planck Institute for Meteorology institute_id: MPI-M experiment_id: rcp45 source: MPI-ESM-MR 2011; URL: http://svn.zmaw.de... model_id: MPI-ESM-MR forcing: GHG,Oz,SD,Sl,Vl,LU - ... ... + parent_experiment_id: historical + parent_experiment_rip: r1i1p1 + branch_time: 56978.0 + contact: cmip5-mpi-esm@dkrz.de + history: Model raw output postprocessing with mod... + references: ECHAM6: n/a; JSBACH: Raddatz et al., 200... + initialization_method: 1 + physics_version: 1 + tracking_id: e7e78b40-458b-48bd-963a-aa377a89f90b + product: output + experiment: RCP4.5 + frequency: mon + creation_date: DATE_TIME_TZ + Conventions: CF-1.4 + project_id: CMIP5 + table_id: Table Amon (27 April 2011) a5a1c518f52ae... title: MPI-ESM-MR model output prepared for CMI... parent_experiment: historical modeling_realm: atmos realization: 1 cmor_version: 2.6.0 DODS_EXTRA.Unlimited_Dimension: time</pre> ```
tlvu
added a commit
to Ouranosinc/pavics-sdi
that referenced
this pull request
Apr 8, 2021
New Jupyter env from Ouranosinc/PAVICS-e2e-workflow-tests#68 Fix the following error and switch to xarray html output and NBVAL_IGNORE_OUTPUT at the same time: ``` _ pavics-sdi-update-regridding-ipynb-for-new-jupyter-env/docs/source/notebooks/subsetting.ipynb::Cell 4 _ Notebook cell execution failed Cell 4: Cell outputs differ Input: res = resp.get() print("URL: ", res.output) res = resp.get(asobj=True) res.output Traceback: mismatch 'text/html' assert reference_output == test_output failed: '<pre><xar... 2.6.0</pre>' == '<pre><xar... 2.6.0</pre>' Skipping 587 identical leading characters in diff, use -v to show - ttributes: (12/27) + ttributes: institution: Max Planck Institute for Meteorology institute_id: MPI-M experiment_id: rcp45 source: MPI-ESM-MR 2011; URL: http://svn.zmaw.de/svn/cosm... model_id: MPI-ESM-MR forcing: GHG,Oz,SD,Sl,Vl,LU - ... ... + parent_experiment_id: historical + parent_experiment_rip: r1i1p1 + branch_time: 56978.0 + contact: cmip5-mpi-esm@dkrz.de + history: Model raw output postprocessing with modelling en... + references: ECHAM6: n/a; JSBACH: Raddatz et al., 2007. Will t... + initialization_method: 1 + physics_version: 1 + tracking_id: e7e78b40-458b-48bd-963a-aa377a89f90b + product: output + experiment: RCP4.5 + frequency: mon + creation_date: DATE_TIME_TZ + Conventions: CF-1.4 + project_id: CMIP5 table_id: Table Amon (27 April 2011) a5a1c518f52ae340313ba0... title: MPI-ESM-MR model output prepared for CMIP5 RCP4.5 parent_experiment: historical modeling_realm: atmos realization: 1 cmor_version: 2.6.0</pre> ```
tlvu
added a commit
to Ouranosinc/pavics-sdi
that referenced
this pull request
Apr 8, 2021
New Jupyter env from Ouranosinc/PAVICS-e2e-workflow-tests#68 Fix this error: ``` _ pavics-sdi-update-regridding-ipynb-for-new-jupyter-env/docs/source/notebooks/regridding.ipynb::Cell 28 _ Notebook cell execution failed Cell 28: Cell outputs differ Input: # Now we can plot easily the results as a choropleth map! ax = shapes_data.plot('tasmin', legend=True, legend_kwds={'label': 'Minimal temperature 1993-05-20 [K]'}) ax.set_ylabel('Latitude') ax.set_xlabel('Longitude') Traceback: mismatch 'text/plain' assert reference_output == test_output failed: "Text(0.5, 79... 'Longitude')" == "Text(0.5, 79... 'Longitude')" - Text(0.5, 79.15273424321427, 'Longitude') ? ^^ + Text(0.5, 79.1527342432141, 'Longitude') ? ^ ```
tlvu
added a commit
to bird-house/birdhouse-deploy
that referenced
this pull request
Apr 9, 2021
jupyter: update for new clisops, xclim, ravenpy Matching PR to deploy the new Jupyter env to PAVICS. See PR Ouranosinc/PAVICS-e2e-workflow-tests#68 for more info. Relevant changes: ```diff < - clisops=0.5.1=pyhd3deb0d_0 > - clisops=0.6.3=pyh44b312d_0 < - xclim=0.23.0=pyhd8ed1ab_0 > - xclim=0.25.0=pyhd8ed1ab_0 > - ostrich=0.1.2=h2bc3f7f_0 > - raven=0.1.1=h2bc3f7f_0 < - ravenpy==0.2.3 # from pip > - ravenpy=0.3.1=py37_0 # from conda > - aiohttp=3.7.4=py37h5e8e339_0 < - roocs-utils=0.1.5=pyhd3deb0d_1 > - roocs-utils=0.3.0=pyh6c4a22f_0 < - cf_xarray=0.4.0=pyh44b312d_0 > - cf_xarray=0.5.1=pyh44b312d_0 < - rioxarray=0.2.0=pyhd8ed1ab_0 > - rioxarray=0.3.1=pyhd8ed1ab_0 < - xarray=0.16.2=pyhd8ed1ab_0 > - xarray=0.17.0=pyhd8ed1ab_0 < - geopandas=0.8.2=pyhd8ed1ab_0 > - geopandas=0.9.0=pyhd8ed1ab_0 < - gdal=3.1.4=py37h2ec2946_5 > - gdal=3.2.1=py37hc5bc4e4_7 < - jupyter_conda=4.1.0=hd8ed1ab_1 > - jupyter_conda=5.0.0=hd8ed1ab_0 < - python=3.7.9=hffdb5ce_100_cpython > - python=3.7.10=hffdb5ce_100_cpython ```
tlvu
added a commit
to Ouranosinc/pavics-sdi
that referenced
this pull request
Apr 9, 2021
…ew-jupyter-env notebooks: fix output change breaking Jenkins due to new Jupyter env New Jupyter env Ouranosinc/PAVICS-e2e-workflow-tests#68. Passing Jenkins build with this fix http://jenkins.ouranos.ca/job/PAVICS-e2e-workflow-tests/job/new-docker-build-for-clisops-and-add-xkcd-font-for-matplotlib/16/console See each commit for the fixed output detected by Jenkins. Migrate to Xarray html output with NBVAL_IGNORE_OUTPUT to avoid breaking Jenkins (html output always detected as different by nbval).
tlvu
added a commit
to bird-house/birdhouse-deploy-ouranos
that referenced
this pull request
Apr 9, 2021
Deploy PAVICS-landing notebooks to Jupyter env For issue Ouranosinc/PAVICS-landing#19. Need updated Jupyter env Ouranosinc/PAVICS-e2e-workflow-tests#68. Given the PAVICS-landing is a private repo, I was not able to hook into the existing notebooks deployment (that assume all notebook repos are public) that controls the `tutorial-notebooks` folder so the new folder `pavics-homepage` has to be on its own and not under `tutorial-notebooks`. If you absolutely want under `tutorial-notebooks`, I will have to migrate the existing notebooks deployment to use this new generic way to deploy anything, this will take much more time. So the current config will check for changes and autodeploy every hour (`DEPLOY_DATA_JOB_SCHEDULE`). I also added the `.geojson` data files. If in the future there are new data file type, you can simply add the new extension to the list in `rsync_extra_opts`. I tried to run all the notebooks and the `PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb` have this error `OSError: [Errno 30] Read-only file system: 'output'` at the last cell. Have not had time to investigate but on the production Jupyter env, the `/pavics-homepage` dir is read-only so if you need to write some temporary file, you have to select another location, like under `/tmp` for example. All other notebooks seem to work fine. It's live on our staging https://medus.ouranos.ca/ 
tlvu
deleted the
new-docker-build-for-clisops-and-add-xkcd-font-for-matplotlib
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Jenkins build passing http://jenkins.ouranos.ca/job/PAVICS-e2e-workflow-tests/job/new-docker-build-for-clisops-and-add-xkcd-font-for-matplotlib/16/console with notebook fix Ouranosinc/pavics-sdi#212
Matching PR bird-house/birdhouse-deploy#144 to deploy to PAVICS.
This new Jupyter env needed to run the Pavics-homepage tutorial notebooks here bird-house/birdhouse-deploy-ouranos#8
Fixes Ouranosinc/pavics-sdi#204
Relevant changes:
Full diff of
conda env export:210216-210216-update210406-1-conda-env-export.diff.txt
210216-210216-update210408-conda-env-export.diff.txt
210216-210408-conda-env-export.diff.txt
Full new
conda env export:210216-update210406-1-conda-env-export.yml.txt
210216-update210408-conda-env-export.yml.txt
210408-conda-env-export.yml.txt