Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release 0.8.3: proper pin of dask and distributed for the Finch docker image #233

Merged
merged 5 commits into from Apr 21, 2022

Conversation

tlvu
Copy link
Collaborator

@tlvu tlvu commented Apr 19, 2022

Pin dask and distributed for the performance problem observed in Ouranosinc/PAVICS-e2e-workflow-tests#100.

This is the same pin as commit 7bf4a7e but in file environment.yml for Docker build.

Our nightly Jenkins has also been consistently falling at the notebook subset-user-input.ipynb with the following error and leave scrap in the WPS queue:

21:01:21  ServiceException: <?xml version="1.0" encoding="utf-8"?>
21:01:21  <ExceptionReport version="1.0.0"
21:01:21      xmlns="http://www.opengis.net/ows/1.1"
21:01:21      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
21:01:21      xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd">
21:01:21      <Exception exceptionCode="NoApplicableCode" locator="NotAcceptable">
21:01:21          <ExceptionText>Request failed: (&#x27;Connection aborted.&#x27;, RemoteDisconnected(&#x27;Remote end closed connection without response&#x27;))</ExceptionText>
21:01:21      </Exception>
21:01:21  </ExceptionReport>

Will perform release procedure at https://github.com/bird-house/finch/blob/master/docs/source/dev_guide.rst#bump-a-new-version once this PR approved.

Got this error on DockerHub:

```
10 [5/6] RUN conda config --add channels conda-forge && conda env create -n finch -f environment.yml && conda install -c conda-forge -n finch gunicorn psycopg2 && rm -rf /opt/conda/pkgs/*
10 sha256:300f07b5b9c74a71d82689738dfbba263b52421627f1100c2b35f97fa0d3957d
10 1.391 Collecting package metadata (repodata.json): ...working... done
10 98.50 Solving environment: ...working... *** picosat: out of memory in 'resize'
10 1262.5 Aborted
10 ERROR: executor failed running [/bin/sh -c conda config --add channels conda-forge && conda env create -n finch -f environment.yml && conda install -c conda-forge -n finch gunicorn psycopg2 && rm -rf /opt/conda/pkgs/*]: exit code: 134
------
> [5/6] RUN conda config --add channels conda-forge && conda env create -n finch -f environment.yml && conda install -c conda-forge -n finch gunicorn psycopg2 && rm -rf /opt/conda/pkgs/*:
------
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c conda config --add channels conda-forge && conda env create -n finch -f environment.yml && conda install -c conda-forge -n finch gunicorn psycopg2 && rm -rf /opt/conda/pkgs/*]: exit code: 134
Build failed using Buildkit
```

This issue conda/conda#9067 hints that maybe a conda update might fix it.
@tlvu
Copy link
Collaborator Author

tlvu commented Apr 20, 2022

Having problems building on DockerHub with error picosat: out of memory in 'resize' so while investigating, I notice xclim is pinned at 0.31. Given current xclim is at 0.35, is it normal that xclim is pinned so old? The previous Finch 0.8.2 is also using this old xclim, while probably xclim 0.34 exists, is that expected?

- xclim == 0.31

@aulemahal
Copy link
Collaborator

Updating xclim within finch is usually more complex than only bumping the number, so we pin a specific version each time. And in the last weeks none of us had the time and the need to update finch.

tlvu and others added 3 commits April 20, 2022 15:43
Trying to fix this error on DockerHub, hope this works:

```
10 [5/6] RUN conda config --add channels conda-forge && conda env create -n finch -f environment.yml && conda install -c conda-forge -n finch gunicorn psycopg2 && rm -rf /opt/conda/pkgs/*
10 sha256:32f6b0b62771f2317c26270dfe736f18955f5461cf54e4fcdcbf6df1327eb4ab
10 1.904 Collecting package metadata (repodata.json): ...working... done
10 126.7 Solving environment: ...working... Killed
```
@tlvu tlvu merged commit 168fa52 into master Apr 21, 2022
@tlvu tlvu deleted the new-release-to-pin-dask-distributed branch April 21, 2022 15:11
tlvu added a commit to bird-house/birdhouse-deploy that referenced this pull request Apr 21, 2022
PR to deploy new Finch releases in
bird-house/finch#233 on PAVICS.

See the Finch PR for more info.

Finch release notes:

0.8.3 (2022-04-21)
------------------
* Preserve RCP dimension in ensemble processes, even when only RCP
is selected.

* Pin ``dask`` and ``distributed`` at ``2022.1.0``, see
Ouranosinc/PAVICS-e2e-workflow-tests#100
tlvu added a commit to bird-house/birdhouse-deploy that referenced this pull request Apr 21, 2022
…e-fix

finch: new release for dask performance problem

PR to deploy new Finch releases in bird-house/finch#233 on PAVICS.

See the Finch PR for more info.

Finch release notes:

0.8.3 (2022-04-21)
------------------
* Preserve RCP dimension in ensemble processes, even when only RCP is selected.

* Pin ``dask`` and ``distributed`` at ``2022.1.0``, see Ouranosinc/PAVICS-e2e-workflow-tests#100
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants