Conda does not fall back to using soft links #4253

nehaljwani opened this Issue Jan 10, 2017 · 0 comments


None yet

1 participant


Let me describe a scenario.

There is a company, called marvelcomics and inside that, is a team called avengers. Two members of this team are:

[shield@earth ~]$ id thor
uid=1004(thor) gid=1002(avengers) groups=1002(avengers)
[shield@earth ~]$ id loki
uid=1003(loki) gid=1002(avengers) groups=1002(avengers)

Now, there is a playground called asgard where these two guys can play:

[shield@earth ~]$ ls -ld /asgard
drwxrwxrwt. 4 root root 4096 Jan 10 11:24 /asgard/
[shield@earth ~]$ getfacl /asgard
getfacl: Removing leading '/' from absolute path names
# file: asgard/
# owner: root
# group: root
# flags: --t
[shield@earth ~]$ mount | grep /asgard
/dev/sdb1 on /asgard type ext4 (rw,relatime,seclabel,data=ordered)

The user thor creates a conda environment on earth, inside the playground using conda and installs his favourite tool: curl.

[thor@earth ~]$ wget
[thor@earth ~]$ chmod +x 
[thor@earth ~]$ ./ -b -p /asgard/kingdom
[thor@earth ~]$ source /asgard/kingdom/bin/activate
(root) [thor@earth ~]$ conda install curl

The user loki tries to steal this kingdom:

[loki@earth ~]$ source /asgard/kingdom/bin/activate 
(root) [loki@earth ~]$ conda config --add envs_dirs /asgard/loki
(root) [loki@earth ~]$ conda create -n stealth curl
Fetching package metadata .......
Solving package specifications: ..........

Package plan for installation in environment /asgard/loki/stealth:

The following NEW packages will be INSTALLED:

    curl:    7.49.0-1
    openssl: 1.0.2j-0 (soft-link)
    zlib:    1.2.8-3  (soft-link)

Proceed ([y]/n)? y

Linking packages ...
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]

CondaOSError: OS error: failed to link (src=u'/asgard/kingdom/pkgs/curl-7.49.0-1/bin/curl', dst=u'/asgard/loki/stealth/bin/curl', type=1, error=OSError(1, 'Operation not permitted'))

Why did loki think that he could steal that easily? Because, he had read in conda's docs:

When allow_softlinks is True, conda uses hard-links when possible, and soft-links (symlinks) when hard-links are not possible, such as when installing on a different filesystem than the one that the package cache is on.

... and he had double checked:

(root) [loki@earth ~]$ conda config --show | grep allow_softlinks
allow_softlinks: True

That's weird. Let's see if shield has added any protection mechanisms:

[shield@earth ~]$ sudo cat /proc/sys/fs/protected_hardlinks 

Of course there is one, so, unless loki has ownership of a file or at least read+write permissions on it, he can't steal it. Let's check:

[shield@earth ~]$ ls -l /asgard/kingdom/pkgs/curl-7.49.0-1/info/index.json 
-rw-rw-r--. 1 thor avengers 274 Jul 21 18:56 /shared/thor/pkgs/curl-7.49.0-1/info/index.json

Well, loki is an avenger, so he has read+write permissions on this file and he indeed can create a link to this file. And conda is correct till now in assuming that he can... However, the reason why he fails to steal:

[shield@earth ~]$ ls -l /asgard/kingdom/pkgs/curl-7.49.0-1/bin/curl
-rwxr-xr-x. 2 thor avengers 182238 Jul 21 18:56 /asgard/kingdom/pkgs/curl-7.49.0-1/bin/curl

There is no write permission on this inode for loki. And hence, he cannot steal blindly.

This scenario makes me raise some queries:

  1. Why does conda check only ${CONDA_PREFIX}/info/index.json for determining whether or not it can use LINK_HARD for the entire package? In other words, how/why is this file the source of truth?
  2. Is there an option in conda using which I can specify that I want to prefer soft-links over hard-links, while creating new environments (using conda create -n or conda env create ...), regardless of whether creating hard-links is possible? Why/Why not?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment