Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upBuild Fails to Create Conda Environment #2651
Comments
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
fmaussion
Feb 21, 2017
Hi, we have the same problem since about 1 week:
xarray: https://readthedocs.org/projects/xray/builds/5051404/
fmaussion
commented
Feb 21, 2017
|
Hi, we have the same problem since about 1 week: xarray: https://readthedocs.org/projects/xray/builds/5051404/ |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
swainn
Feb 23, 2017
It seems this issue can be remedied by upgrading to conda 4.2.13. See: conda/conda#3448 (comment)
swainn
commented
Feb 23, 2017
|
It seems this issue can be remedied by upgrading to conda 4.2.13. See: conda/conda#3448 (comment) |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
fmaussion
commented
Feb 23, 2017
|
Thanks. Do we have any control on the conda version installed on RTD? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
fmaussion
Feb 23, 2017
What is striking me is that we are the only ones reporting this. I'd expect such an issue to affect much more people?
fmaussion
commented
Feb 23, 2017
|
What is striking me is that we are the only ones reporting this. I'd expect such an issue to affect much more people? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
woodmd
commented
Feb 24, 2017
|
I have the same error: |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
fmaussion
Feb 28, 2017
Our docs aren't built since two weeks now... :-(
Did anyone come up with a solution?
fmaussion
commented
Feb 28, 2017
•
|
Our docs aren't built since two weeks now... :-( Did anyone come up with a solution? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
swainn
Feb 28, 2017
Same here. We've had no docs builds succeed for nearly two weeks and no response from RTD...
swainn
commented
Feb 28, 2017
|
Same here. We've had no docs builds succeed for nearly two weeks and no response from RTD... |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
spencerahill
Mar 2, 2017
Same problem for us too: https://readthedocs.org/projects/aospy/builds/5097720/
spencerahill
commented
Mar 2, 2017
|
Same problem for us too: https://readthedocs.org/projects/aospy/builds/5097720/ |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
snowman2
Mar 3, 2017
Similar issue here: http://readthedocs.org/projects/gsshapy/builds/5091350. Builds having issues for over 2 weeks.
snowman2
commented
Mar 3, 2017
|
Similar issue here: http://readthedocs.org/projects/gsshapy/builds/5091350. Builds having issues for over 2 weeks. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
dcolligan
Mar 3, 2017
Same issue on our builds, for the same period of time as others are reporting:
https://readthedocs.org/projects/ga4gh-reference-implementation/builds/5097461/
https://readthedocs.org/projects/ga4gh-client/builds/5097407/
https://readthedocs.org/projects/ga4gh-schemas/builds/5051360/
dcolligan
commented
Mar 3, 2017
|
Same issue on our builds, for the same period of time as others are reporting: https://readthedocs.org/projects/ga4gh-reference-implementation/builds/5097461/ |
cdeil
referenced this issue
Mar 5, 2017
Closed
Broken conda install for docs build on Readthedocs #934
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
david4096
Mar 7, 2017
We're considering dropping RTD because of this. Is there something we can do to help the update of conda happen on RTD?
david4096
commented
Mar 7, 2017
|
We're considering dropping RTD because of this. Is there something we can do to help the update of conda happen on RTD? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
humitos
Mar 7, 2017
Member
@agjohnson now that we are talking about upgrading the packages from conda-forge in #2378, is it possible to upgrade the conda version also? Since it's a bugfix version I think there is no problem to upgrade from 4.2.12 to 4.2.13 (people says this problem is fixed there). What do you think?
Considering how RTD internally works with conda (https://github.com/rtfd/readthedocs-docker-images/blob/master/Dockerfile#L40), it seems that this upgrade won't be available until the next docker image is built and deployed to production.
Other (some of them, quick-and-dirty) solutions I have on the top of my head:
- run
conda update condabefore each of the builds (kind of dangerous since RTD can't guarantee a specific version) - add a config in the
ymlto pin the conda version before running the build - upgrade the conda version just once inside the docker container used to build the documentation (hotfix)
The latest version of conda seems to be 4.3.14-py36_0 in my pc when running conda update conda
|
@agjohnson now that we are talking about upgrading the packages from Considering how RTD internally works with conda (https://github.com/rtfd/readthedocs-docker-images/blob/master/Dockerfile#L40), it seems that this upgrade won't be available until the next docker image is built and deployed to production. Other (some of them, quick-and-dirty) solutions I have on the top of my head:
The latest version of conda seems to be |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
I don't fully understand conda, and we need people on our core team who can actually track and update the usage. What is the correct fix for this? Is it something that we broke, or did conda break?
|
I don't fully understand conda, and we need people on our core team who can actually track and update the usage. What is the correct fix for this? Is it something that we broke, or did conda break? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
I'm happy to upgrade conda if that is the correct fix -- but I'd like to know the root cause of what broke, so that we can prevent it from happening again.
|
I'm happy to upgrade conda if that is the correct fix -- but I'd like to know the root cause of what broke, so that we can prevent it from happening again. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
snowman2
commented
Mar 7, 2017
|
Here is a discussion on the issue on conda: conda/conda#3448 (comment) |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
So this was an issue with the version of conda from when we created the last set of docker images? Is there a way to pin an install of conda to a specific verison? The official install docs only mention a way to install the "latest".
|
So this was an issue with the version of conda from when we created the last set of docker images? Is there a way to pin an install of conda to a specific verison? The official install docs only mention a way to install the "latest". |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
The current plan is to rebuild the 2.0 docker image, which I believe will cause it to install the latest version of conda. I'd like to figure out a way to pin the version to prevent this issue going forward. We've had a lot of breakage in the conda tooling specifically, and I'd like to knowingly upgrade it, instead of it happening randomly on image rebuilds.
|
The current plan is to rebuild the 2.0 docker image, which I believe will cause it to install the latest version of conda. I'd like to figure out a way to pin the version to prevent this issue going forward. We've had a lot of breakage in the conda tooling specifically, and I'd like to knowingly upgrade it, instead of it happening randomly on image rebuilds. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
Image rebuild is going here: https://hub.docker.com/r/readthedocs/build/builds/bgpgd7axkbaihm6wkexya6h/ -- it will probably take ~30 mins
|
Image rebuild is going here: https://hub.docker.com/r/readthedocs/build/builds/bgpgd7axkbaihm6wkexya6h/ -- it will probably take ~30 mins |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
humitos
Mar 7, 2017
Member
I'd like to figure out a way to pin the version to prevent this issue going forward.
There is a blog post about that: http://www.damian.oquanta.info/posts/how-to-pin-conda.html
Anyway, it says that conda will upgrade automatically on each install (I just tried that at my pc and it's true) and if you want to not do so, you have to pin conda itself by using the pinned file (https://conda.io/docs/faq.html#pinning-packages)
[12:44] humitos: ~> conda install scipy
Fetching package metadata .........
Solving package specifications: .
Package plan for installation in environment /home/humitos/.miniconda3:
The following NEW packages will be INSTALLED:
libgfortran: 3.0.0-1
mkl: 2017.0.1-0
numpy: 1.12.0-py36_0
scipy: 0.18.1-np112py36_1
The following packages will be UPDATED:
conda: 4.3.11-py36_0 --> 4.3.14-py36_0
Proceed ([y]/n)?
On the other hand, I didn't find a way to install for the first time an specific version of conda. It seems that you have to install the latest one and then you can do conda update conda=x.x.x if you want to pin it.
There is a blog post about that: http://www.damian.oquanta.info/posts/how-to-pin-conda.html Anyway, it says that conda will upgrade automatically on each install (I just tried that at my pc and it's true) and if you want to not do so, you have to pin conda itself by using the On the other hand, I didn't find a way to install for the first time an specific version of conda. It seems that you have to install the latest one and then you can do |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
Sounds like a way forward at least. So we can do the network install, and then do the conda update conda=x.x.x after that in the Dockerfile.
|
Sounds like a way forward at least. So we can do the network install, and then do the |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
I'm not convinced that won't break at some point in the future, but at least it makes the builds more deterministic :)
|
I'm not convinced that won't break at some point in the future, but at least it makes the builds more deterministic :) |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
snowman2
Mar 7, 2017
Please don't deprecate conda support. Is makes using/distributing Python packages soo much easier.
snowman2
commented
Mar 7, 2017
|
Please don't deprecate conda support. Is makes using/distributing Python packages soo much easier. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
agjohnson
Mar 7, 2017
Contributor
Moving conversation around the Docker image to rtfd/readthedocs-docker-images#22
A new build image likely won't hit our servers for a few hours, the build process takes a while.
Also, I'll say that for anyone wondering why their project is still hitting this issue, consider contributing your time to resolve this. We are community run service and need both your input and your contributions.
|
Moving conversation around the Docker image to rtfd/readthedocs-docker-images#22 A new build image likely won't hit our servers for a few hours, the build process takes a while. Also, I'll say that for anyone wondering why their project is still hitting this issue, consider contributing your time to resolve this. We are community run service and need both your input and your contributions. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
It seems that 4.3.11 is the last released version of miniconda? So the latest released version is broken?
|
It seems that 4.3.11 is the last released version of miniconda? So the latest released version is broken? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
It seems that 4.2.12 is the last released verison, and the above bug says that 4.2.13 is the one that needs to be used, but isn't released here: https://repo.continuum.io/miniconda/ ?
|
It seems that 4.2.12 is the last released verison, and the above bug says that 4.2.13 is the one that needs to be used, but isn't released here: https://repo.continuum.io/miniconda/ ? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
snowman2
commented
Mar 7, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
humitos
Mar 7, 2017
Member
wait, is the latest version of conda failing to create an environment and install some packages? Instead of us keep upgrading the RTD server to a random version of conda, can you (people with failing builds) test this locally by creating the proper conda environment with the different versions and confirm which version of conda is not having troubles in your pc?
If conda is broken there is nothing that we can do at RTD but just wait for a fix on them :)
|
wait, is the latest version of conda failing to create an environment and install some packages? Instead of us keep upgrading the RTD server to a random version of conda, can you (people with failing builds) test this locally by creating the proper conda environment with the different versions and confirm which version of conda is not having troubles in your pc? If conda is broken there is nothing that we can do at RTD but just wait for a fix on them :) |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 7, 2017
Member
Agreed -- I don't really know how to fix this, other than shooting in the dark at random released or unreleased versions of miniconda. I'd love to know a known good version, and be able to go with it moving forward.
|
Agreed -- I don't really know how to fix this, other than shooting in the dark at random released or unreleased versions of miniconda. I'd love to know a known good version, and be able to go with it moving forward. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
humitos
Mar 7, 2017
Member
This build (https://readthedocs.org/projects/aospy/builds/5117255/) has a different error and I'm guessing the Permission denied problem could be related because the miniconda is installed as root and then it's executed as docs:
-
Installed as root: https://github.com/rtfd/readthedocs-docker-images/blob/releases/2.x/Dockerfile#L37
-
User changed to
docs: https://github.com/rtfd/readthedocs-docker-images/blob/releases/2.x/Dockerfile#L44
(the docker hub image build related: https://hub.docker.com/r/readthedocs/build/builds/bgpgd7axkbaihm6wkexya6h/)
|
This build (https://readthedocs.org/projects/aospy/builds/5117255/) has a different error and I'm guessing the
(the docker hub image build related: https://hub.docker.com/r/readthedocs/build/builds/bgpgd7axkbaihm6wkexya6h/) |
humitos
added
Bug
Operations
labels
Mar 7, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
agjohnson
Mar 7, 2017
Contributor
If this is a permission issue: rtfd/readthedocs-docker-images#24
Testing this shouldn't be difficult.
|
If this is a permission issue: rtfd/readthedocs-docker-images#24 Testing this shouldn't be difficult. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
|
That doesn't explain why this recently started failing though? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
agjohnson
Mar 7, 2017
Contributor
There were two major changes in this timeframe: we rerolled the build images (upgrading conda), and the new image is based on ubuntu 16.04 now. I have no clue what the internals of the shell script do, but if it was doing something different in 14.04 vs 16.04, this could be the cause.
|
There were two major changes in this timeframe: we rerolled the build images (upgrading conda), and the new image is based on ubuntu 16.04 now. I have no clue what the internals of the shell script do, but if it was doing something different in 14.04 vs 16.04, this could be the cause. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
humitos
Mar 7, 2017
Member
Just to add more context here. I tried to build one of the failing project outside RTD (env, docker img, etc) and it worked:
[14:59] humitos: repositories> git clone https://github.com/spencerahill/aospy.git
Clonar en «aospy»...
remote: Counting objects: 2156, done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 2156 (delta 7), reused 1 (delta 1), pack-reused 2133
Receiving objects: 100% (2156/2156), 3.62 MiB | 484.00 KiB/s, done.
Resolving deltas: 100% (1468/1468), done.
[14:59] humitos: repositories> cd aospy/
/home/humitos/readthedocs/repositories/aospy
[14:59] humitos: aospy> conda env create --name aospy_latest --file docs/environment.yml
Fetching package metadata ...........
Solving package specifications: .
ca-certificate 100% |##############################################################################################################################################| Time: 0:00:03 53.90 kB/s
[... snipped ...]
matplotlib-1.5 100% |##############################################################################################################################################| Time: 0:00:14 543.61 kB/s
#
# To activate this environment, use:
# > source activate aospy_latest
#
# To deactivate this environment, use:
# > source deactivate aospy_latest
#
[15:13] humitos: aospy> conda info
Current conda install:
platform : linux-64
conda version : 4.3.11
conda is private : False
conda-env version : 4.3.11
conda-build version : not installed
python version : 3.6.0.final.0
requests version : 2.12.4
root environment : /home/humitos/.miniconda3 (writable)
default environment : /home/humitos/.miniconda3
envs directories : /home/humitos/.miniconda3/envs
/home/humitos/.conda/envs
package cache : /home/humitos/.miniconda3/pkgs
/home/humitos/.conda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/linux-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
user-agent : conda/4.3.11 requests/2.12.4 CPython/3.6.0 Linux/4.4.0-64-generic debian/stretch/sid glibc/2.23
UID:GID : 1000:1000
[15:48] humitos: aospy>
After this, I booted the readthedocs/build:2.0 and ran into this permission problem again:
docs@40764e8adc4f:/$ conda info
Current conda install:
platform : linux-64
conda version : 4.2.12
conda is private : False
conda-env version : 4.2.12
conda-build version : not installed
python version : 2.7.12.final.0
requests version : 2.11.1
root environment : /usr/local/miniconda (read only)
default environment : /usr/local/miniconda
envs directories : /home/docs/.conda/envs
/usr/local/miniconda/envs
package cache : /home/docs/.conda/envs/.pkgs
/usr/local/miniconda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
docs@40764e8adc4f:/$ git clone https://github.com/spencerahill/aospy.git
fatal: could not create work tree dir 'aospy': Permission denied
docs@40764e8adc4f:/$ cd
docs@40764e8adc4f:~$ git clone https://github.com/spencerahill/aospy.git
Cloning into 'aospy'...
remote: Counting objects: 2156, done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 2156 (delta 7), reused 1 (delta 1), pack-reused 2133
Receiving objects: 100% (2156/2156), 3.62 MiB | 502.00 KiB/s, done.
Resolving deltas: 100% (1468/1468), done.
Checking connectivity... done.
docs@40764e8adc4f:~$ cd aospy/
docs@40764e8adc4f:~/aospy$ conda env create --name latest --file docs/environment.yml
Fetching package metadata .........
Solving package specifications: ..........
Fetching packages ...
ca-certificate 100% |##############################################################################################################################################| Time: 0:00:00 240.00 kB/s
[ ... snipped ... ]
xz-5.2.2-0.tar 100% |##############################################################################################################################################| Time: 0:00:02 356.06 kB/s
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:
https://github.com/conda/conda/issues
Current conda install:
platform : linux-64
conda version : 4.2.12
conda is private : False
conda-env version : 4.2.12
conda-build version : not installed
python version : 2.7.12.final.0
requests version : 2.11.1
root environment : /usr/local/miniconda (read only)
default environment : /usr/local/miniconda
envs directories : /home/docs/.conda/envs
/usr/local/miniconda/envs
package cache : /home/docs/.conda/envs/.pkgs
/usr/local/miniconda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
`$ /usr/local/miniconda/bin/conda-env create --name latest --file docs/environment.yml`
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python2.7/site-packages/conda/exceptions.py", line 479, in conda_exception_handler
return_value = func(*args, **kwargs)
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main_create.py", line 111, in execute
installer.install(prefix, pkg_specs, args, env)
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/installers/conda.py", line 38, in install
raise CondaRuntimeError('RuntimeError: %s' % e)
CondaRuntimeError: Runtime error: RuntimeError: Runtime error: Could not open u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part' for writing ([Errno 13] Permission denied: u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part').
docs@40764e8adc4f:~/aospy$ ls -la /usr/local/miniconda/pkgs/
total 112
drwxr-xr-x 20 root root 4096 Feb 5 07:43 .
drwxr-xr-x 11 root root 4096 Feb 5 07:43 ..
-rw-rw-r-- 1 root root 2097 Nov 3 19:04 .cio-config.py
-rw-rw-r-- 1 root root 17834 Nov 3 19:04 .install.py
drwxr-xr-x 2 root root 4096 Feb 5 07:43 cache
drwxr-xr-x 6 root root 4096 Feb 5 07:43 conda-4.2.12-py27_0
drwxr-xr-x 3 root root 4096 Feb 5 07:43 conda-env-2.6.0-0
drwxr-xr-x 4 root root 4096 Feb 5 07:43 enum34-1.1.6-py27_0
drwxr-xr-x 7 root root 4096 Feb 5 07:43 openssl-1.0.2j-0
drwxr-xr-x 5 root root 4096 Feb 5 07:43 pip-8.1.2-py27_0
drwxr-xr-x 4 root root 4096 Feb 5 07:43 pycosat-0.6.1-py27_1
drwxr-xr-x 4 root root 4096 Feb 5 07:43 pycrypto-2.6.1-py27_4
drwxr-xr-x 7 root root 4096 Feb 5 07:43 python-2.7.12-1
drwxr-xr-x 6 root root 4096 Feb 5 07:43 readline-6.2-2
drwxr-xr-x 4 root root 4096 Feb 5 07:43 requests-2.11.1-py27_0
drwxr-xr-x 4 root root 4096 Feb 5 07:43 ruamel_yaml-0.11.14-py27_0
drwxr-xr-x 5 root root 4096 Feb 5 07:43 setuptools-27.2.0-py27_0
drwxr-xr-x 6 root root 4096 Feb 5 07:43 sqlite-3.13.0-0
drwxr-xr-x 6 root root 4096 Feb 5 07:43 tk-8.5.18-0
-rw-rw-r-- 1 root root 1777 Nov 3 19:04 urls
-rw-rw-r-- 1 root root 1216 Nov 3 19:04 urls.txt
drwxr-xr-x 5 root root 4096 Feb 5 07:43 wheel-0.29.0-py27_0
drwxr-xr-x 5 root root 4096 Feb 5 07:43 yaml-0.1.6-0
drwxr-xr-x 5 root root 4096 Feb 5 07:43 zlib-1.2.8-3
docs@40764e8adc4f:~/aospy$ ls -la /home/docs/.conda/envs/.pkgs
total 46044
drwxr-xr-x 3 docs docs 4096 Mar 7 20:55 .
drwxr-xr-x 4 docs docs 4096 Mar 7 20:53 ..
-rw-r--r-- 1 docs docs 169592 Mar 7 20:53 ca-certificates-2017.1.23-0.tar.bz2
drwxr-xr-x 2 docs docs 4096 Mar 7 20:53 cache
-rw-r--r-- 1 docs docs 5892278 Mar 7 20:54 gettext-0.19.7-1.tar.bz2
-rw-r--r-- 1 docs docs 120285 Mar 7 20:54 graphite2-1.3.9-0.tar.bz2
-rw-r--r-- 1 docs docs 23833004 Mar 7 20:54 icu-58.1-1.tar.bz2
-rw-r--r-- 1 docs docs 939817 Mar 7 20:55 jpeg-9b-0.tar.bz2
-rw-r--r-- 1 docs docs 47736 Mar 7 20:55 libffi-3.2.1-3.tar.bz2
-rw-r--r-- 1 docs docs 287481 Mar 7 20:55 libgfortran-3.0.0-1.tar.bz2
-rw-r--r-- 1 docs docs 2077443 Mar 7 20:55 libiconv-1.14-4.tar.bz2
-rw-r--r-- 1 docs docs 1121442 Mar 7 20:55 ncurses-5.9-10.tar.bz2
-rw-r--r-- 1 docs docs 742096 Mar 7 20:55 pcre-8.39-0.tar.bz2
-rw-r--r-- 1 docs docs 3799933 Mar 7 20:55 pixman-0.34.0-0.tar.bz2
-rw-r--r-- 1 docs docs 5160488 Mar 7 20:55 sqlite-3.13.0-1.tar.bz2
-rw-r--r-- 1 docs docs 2038478 Mar 7 20:55 tk-8.5.19-1.tar.bz2
-rw-r--r-- 1 docs docs 998 Mar 7 20:55 urls.txt
-rw-r--r-- 1 docs docs 873657 Mar 7 20:55 xz-5.2.2-0.tar.bz2
docs@40764e8adc4f:~/aospy$
So, the issue is that we have two directories set as package cache, one that is accessible by the docs user and another one that only root has permission. For some reason, some packages goes to the /home/docs/.conda/envs/.pkgs (where the docs user has permission) and for some reason there are packages that goes to /usr/local/miniconda/pkgs and it fails because we are executing conda as docs user.
Another test was trying to update the conda itself inside the docker image and it also says we don't have permissions to do it:
docs@40764e8adc4f:~$ conda info
Current conda install:
platform : linux-64
conda version : 4.2.12
conda is private : False
conda-env version : 4.2.12
conda-build version : not installed
python version : 2.7.12.final.0
requests version : 2.11.1
root environment : /usr/local/miniconda (read only)
default environment : /usr/local/miniconda
envs directories : /home/docs/.conda/envs
/usr/local/miniconda/envs
package cache : /home/docs/.conda/envs/.pkgs
/usr/local/miniconda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
docs@40764e8adc4f:~$ conda update conda
Fetching package metadata .......
Solving package specifications: ..........
Package plan for installation in environment /usr/local/miniconda:
The following packages will be downloaded:
package | build
---------------------------|-----------------
libffi-3.2.1 | 1 38 KB
idna-2.2 | py27_0 122 KB
ipaddress-1.0.18 | py27_0 31 KB
pyasn1-0.1.9 | py27_0 54 KB
pycparser-2.17 | py27_0 153 KB
requests-2.13.0 | py27_0 776 KB
six-1.10.0 | py27_0 16 KB
cffi-1.9.1 | py27_0 325 KB
cryptography-1.7.1 | py27_0 848 KB
pyopenssl-16.2.0 | py27_0 68 KB
conda-4.3.14 | py27_0 481 KB
------------------------------------------------------------
Total: 2.8 MB
The following NEW packages will be INSTALLED:
cffi: 1.9.1-py27_0 (soft-link)
cryptography: 1.7.1-py27_0 (soft-link)
idna: 2.2-py27_0 (soft-link)
ipaddress: 1.0.18-py27_0 (soft-link)
libffi: 3.2.1-1 (soft-link)
pyasn1: 0.1.9-py27_0 (soft-link)
pycparser: 2.17-py27_0 (soft-link)
pyopenssl: 16.2.0-py27_0 (soft-link)
six: 1.10.0-py27_0 (soft-link)
The following packages will be UPDATED:
conda: 4.2.12-py27_0 --> 4.3.14-py27_0 (soft-link)
requests: 2.11.1-py27_0 --> 2.13.0-py27_0 (soft-link)
CondaIOError: IO error: Missing write permissions in: /usr/local/miniconda
#
# You don't appear to have the necessary permissions to update packages
# into the install area '/usr/local/miniconda'.
# However you can clone this environment into your home directory and
# then make changes to it.
# This may be done using the command:
#
# $ conda create -n my_root --clone=/usr/local/miniconda
docs@40764e8adc4f:~$
Following the suggestion from the previous command is not a good idea
docs@40764e8adc4f:~$ conda create -n my_root --clone=/usr/local/miniconda
Source: /usr/local/miniconda
Destination: /home/docs/.conda/envs/my_root
The following packages cannot be cloned out of the root environment:
- conda-4.2.12-py27_0
Packages: 16
Files: 0
Linking packages ...
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
[tk ]|######################################## WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13] | 18%
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
[pycosat ]|########################################################################################## WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
[pip WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]############################################# | 87%
[ COMPLETE ]|#################################################################################################################################################################| 100%
#
# To activate this environment, use:
# > source activate my_root
#
# To deactivate this environment, use:
# > source deactivate my_root
#
docs@40764e8adc4f:~$
I installed miniconda with the docs user and it was successful:
docs@40764e8adc4f:~$ curl -O https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 27.4M 100 27.4M 0 0 162k 0 0:02:53 0:02:53 --:--:-- 141k
docs@40764e8adc4f:~$ bash Miniconda2-latest-Linux-x86_64.sh -b -p /home/docs/miniconda2
PREFIX=/home/docs/miniconda2
installing: python-2.7.13-0 ...
installing: cffi-1.9.1-py27_0 ...
[ ... snipped ... ]
docs@40764e8adc4f:~$ /home/docs/miniconda2/bin/conda info
Current conda install:
platform : linux-64
conda version : 4.3.11
conda is private : False
conda-env version : 4.3.11
conda-build version : not installed
python version : 2.7.13.final.0
requests version : 2.12.4
root environment : /home/docs/miniconda2 (writable)
default environment : /home/docs/miniconda2
envs directories : /home/docs/miniconda2/envs
/home/docs/.conda/envs
package cache : /home/docs/miniconda2/pkgs
/home/docs/.conda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/linux-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
user-agent : conda/4.3.11 requests/2.12.4 CPython/2.7.13 Linux/4.4.0-64-generic debian/stretch/sid glibc/2.23
UID:GID : 1005:205
docs@40764e8adc4f:~$ /home/docs/miniconda2/bin/conda env create --name latest --file aospy/docs/environment.yml
Fetching package metadata ...........
Solving package specifications: .
[ ... snipped ... ]
python-3.5.3-1 100% |##############################################################################################################################################| Time: 0:01:38 196.49 kB/s
[ ... snipped ... ]
matplotlib-1.5 100% |##############################################################################################################################################| Time: 0:00:16 477.15 kB/s
#
# To activate this environment, use:
# > source activate latest
#
# To deactivate this environment, use:
# > source deactivate latest
#
docs@40764e8adc4f:~$
So, my suggestion is to install miniconda after the docs user is created (and switched to) in the docker image as in this PR: rtfd/readthedocs-docker-images#25
Plus, if we really want to pin a version as in rtfd/readthedocs-docker-images#23 we need to use a setting to avoid the automatic upgrade (conda config --set auto_update_conda False -- https://conda.io/docs/config.html#update-conda-automatically-auto-update-conda) but I'm not sure if that is something good to do.
|
Just to add more context here. I tried to build one of the failing project outside RTD (env, docker img, etc) and it worked: After this, I booted the So, the issue is that we have two directories set as Another test was trying to update the conda itself inside the docker image and it also says we don't have permissions to do it: Following the suggestion from the previous command is not a good idea I installed So, my suggestion is to install miniconda after the Plus, if we really want to pin a version as in rtfd/readthedocs-docker-images#23 we need to use a setting to avoid the automatic upgrade ( |
This was referenced Mar 7, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 8, 2017
Member
I've tried downgrading conda, and now we get a YAML error from the conda env command. Hopefully the user permission fix will work, and we can hopefully get this to a working state.
|
I've tried downgrading conda, and now we get a YAML error from the |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 8, 2017
Member
It appears to be this issue: conda/conda#3738
How do we install a version of conda that isn't listed over at https://repo.continuum.io/miniconda/? Do we have to install a lower version of conda, and then use conda upgrade conda==X?
|
It appears to be this issue: conda/conda#3738 How do we install a version of conda that isn't listed over at https://repo.continuum.io/miniconda/? Do we have to install a lower version of conda, and then use |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ericholscher
Mar 8, 2017
Member
I've got this failing locally, and will patch it and deploy a fix in the next couple hours, hopefully.
|
I've got this failing locally, and will patch it and deploy a fix in the next couple hours, hopefully. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
david4096
Mar 8, 2017
My latest https://readthedocs.org/projects/schemas/builds/5118842/
Traceback (most recent call last):
File "/usr/local/miniconda/bin/conda-env", line 4, in <module>
import conda_env.cli.main
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main.py", line 35, in <module>
from . import main_attach
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main_attach.py", line 4, in <module>
from ..env import from_environment
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/env.py", line 17, in <module>
from . import yaml
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/yaml.py", line 8, in <module>
import yaml
ImportError: No module named yaml
david4096
commented
Mar 8, 2017
|
My latest https://readthedocs.org/projects/schemas/builds/5118842/ |
ericholscher
referenced this issue
Mar 8, 2017
Closed
Conda build of `environment.yml` broke without changing yml file #2704
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
|
I believe this should finally be fixed. Knock on wood. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
david4096
Mar 8, 2017
Thanks, looking good so far! https://readthedocs.org/projects/schemas/builds/5118911/
david4096
commented
Mar 8, 2017
|
Thanks, looking good so far! https://readthedocs.org/projects/schemas/builds/5118911/ |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
david4096
commented
Mar 8, 2017
|
Great success, thanks! |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
spencerahill
Mar 8, 2017
Yes, success for us: https://readthedocs.org/projects/aospy/builds/5118901/
Thank you very much!
spencerahill
commented
Mar 8, 2017
|
Yes, success for us: https://readthedocs.org/projects/aospy/builds/5118901/ Thank you very much! |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
agjohnson
Mar 8, 2017
Contributor
Glad to hear this is working for folks. I'll close this for now, feel free to reopen here if it isn't resolved.
|
Glad to hear this is working for folks. I'll close this for now, feel free to reopen here if it isn't resolved. |
agjohnson
closed this
Mar 8, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
humitos
Mar 8, 2017
Member
I think it starts failing recently because a new version of yaml which was installed at root level, so when it tries to upgrade that package with de docs users it fails (check my log and when I said that I don't know why some packages goes to the root cache and some others to the docs cache). It's because of that...
|
I think it starts failing recently because a new version of yaml which was installed at root level, so when it tries to upgrade that package with de docs users it fails (check my log and when I said that I don't know why some packages goes to the root cache and some others to the docs cache). It's because of that... |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
Juanlu001
commented
Mar 8, 2017
•
|
Congratulations to the RTD devs, this wasn't an easy one :) |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
swainn
Mar 8, 2017
This is fixed for me:
https://readthedocs.org/projects/tethys-platform/builds/5122358/
I completely understand the challenges with community projects. Thank you for your time.
swainn
commented
Mar 8, 2017
|
This is fixed for me: |
swainn commentedFeb 18, 2017
Details
Expected Result
I expect the docs for the dev branch of my project to build.
Actual Result
I encounter a failure during the creation of the conda environment (see full output below).
Things I have tried:
(1) Found this suggestion on the Conda issues page, so I attempted wiping the environment several times.
(2) Reverted the back to commit with last successful docs build, but still failed with same error.
(3) Created this Conda environment my dev machine successfully (using miniconda).
Here is the error:
conda env create --name dev --file /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/checkouts/dev/docs-conda_env.yml
Fetching package metadata .........
Solving package specifications: ..........
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:
Current conda install:
$ /usr/local/miniconda/bin/conda-env create --name dev --file /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/checkouts/dev/docs-conda_env.ymlFetching packages ...
yaml-0.1.6-0.t 0% | | ETA: --:--:-- 0.00 B/s