conda install --offline still performs online operations

[ijstokes]

Even conda seems to notice that it shouldn't be doing an online operation when the --offline flag has been specified:

$ conda install --offline statsmodels
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment C:\Users\d53542\AppData\Local\Continuum\Anaconda2:

The following packages will be UPDATED:

    statsmodels: 0.6.1-np111py27_1 --> 0.8.0-np111py27_0

Proceed ([y]/n)? y



CondaError: RuntimeError(u'EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/win-64/statsmodels-0.8.0-np111py27_0.tar.bz2\nThis command is using a remote connection in offline m
ode.\n',)

CondaError: RuntimeError(u'EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/win-64/statsmodels-0.8.0-np111py27_0.tar.bz2\nThis command is using a remote connection in offline m
ode.\n',)

CondaError: RuntimeError(u'EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/win-64/statsmodels-0.8.0-np111py27_0.tar.bz2\nThis command is using a remote connection in offline m
ode.\n',)

C:\Users\d53542\Documents\GIT\health_score>conda -V
conda 4.3.17

I'm at PyCon and am seeing this on the computer of someone I'm talking to (an Anaconda user)

[kalefranz]

Can you add a -vv so we can get the full stack trace?

[kalefranz]

Oh. Wait. Try 4.3.18.

[kalefranz]

There were two PRs in 4.3.18 that hopefully killed at least a couple of these code paths.

[kalefranz]

@ijstokes Can you confirm that you're still seeing this with conda 4.3.18?

[ijstokes]

Unfortunately the problem is still there:

$ conda install --offline statsmodels
Fetching package metadata .........
Solving package specifications: .

Package plan for installation in environment /Users/ijstokes/anaconda/envs/oracletest:

The following NEW packages will be INSTALLED:

    mkl:             2017.0.1-0        
    numpy:           1.12.1-py36_0     
    openssl:         1.0.2k-2          
    pandas:          0.20.1-np112py36_0
    patsy:           0.4.1-py36_0      
    pip:             9.0.1-py36_1      
    python:          3.6.1-2           
    python-dateutil: 2.6.0-py36_0      
    pytz:            2017.2-py36_0     
    readline:        6.2-2             
    scipy:           0.19.0-np112py36_0
    setuptools:      27.2.0-py36_0     
    six:             1.10.0-py36_0     
    sqlite:          3.13.0-0          
    statsmodels:     0.8.0-np112py36_0 
    tk:              8.5.18-0          
    wheel:           0.29.0-py36_0     
    xz:              5.2.2-1           
    zlib:            1.2.8-3           

Proceed ([y]/n)? y



CondaError: RuntimeError('EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/osx-64/patsy-0.4.1-py36_0.tar.bz2\nThis command is using a remote connection in offline mode.\n',)
CondaError: RuntimeError('EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/osx-64/patsy-0.4.1-py36_0.tar.bz2\nThis command is using a remote connection in offline mode.\n',)
CondaError: RuntimeError('EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/osx-64/patsy-0.4.1-py36_0.tar.bz2\nThis command is using a remote connection in offline mode.\n',)

$ conda -V
conda 4.3.18

Here is the debug version:

$ conda install -vv --offline statsmodels

Output is here:

log.txt
log.txt

[kalefranz]

Thanks Ian. This is a new code path for triggering the EnforceUnusedAdapter that I don't think we've seen before. This is helpful.

    Traceback (most recent call last):
      File "/Users/ijstokes/anaconda/lib/python3.6/site-packages/conda/exceptions.py", line 632, in conda_exception_handler
        return_value = func(*args, **kwargs)
      File "/Users/ijstokes/anaconda/lib/python3.6/site-packages/conda/cli/main.py", line 134, in _main
        exit_code = args.func(args, p)
      File "/Users/ijstokes/anaconda/lib/python3.6/site-packages/conda/cli/main_install.py", line 80, in execute
        install(args, parser, 'install')
      File "/Users/ijstokes/anaconda/lib/python3.6/site-packages/conda/cli/install.py", line 357, in install
        execute_actions(actions, index, verbose=not context.quiet)
      File "/Users/ijstokes/anaconda/lib/python3.6/site-packages/conda/plan.py", line 830, in execute_actions
        execute_instructions(plan, index, verbose)
      File "/Users/ijstokes/anaconda/lib/python3.6/site-packages/conda/instructions.py", line 247, in execute_instructions
        cmd(state, arg)
      File "/Users/ijstokes/anaconda/lib/python3.6/site-packages/conda/instructions.py", line 100, in PROGRESSIVEFETCHEXTRACT_CMD
        progressive_fetch_extract.execute()
      File "/Users/ijstokes/anaconda/lib/python3.6/site-packages/conda/core/package_cache.py", line 494, in execute
        self._execute_action(action)
      File "/Users/ijstokes/anaconda/lib/python3.6/site-packages/conda/core/package_cache.py", line 516, in _execute_action
        raise CondaMultiError(exceptions)
    conda.CondaMultiError: RuntimeError('EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/osx-64/patsy-0.4.1-py36_0.tar.bz2\nThis command is using a remote connection in offline mode.\n',)

[ijstokes]

@kalefranz this is killing me. I'm doing a tonne of work this week offline and I can barely use conda because of this. Planes, buses, rooftops, etc. No internet, and --offline is continuing to give me the 3x EnforceUnusedAdapter error message before borking out. I'm amazed other people aren't also suffering from this. Anyway, this is just to say that I don't think this is triggered in --offline edge-cases, it feels to me like it still happens "regularly" when that option is used. Of if I have some weird config somewhere please let me know so I can at least reduce the frequency of the problem until the issue is resolved.

I am using conda 4.3.21

[ijstokes]

@kalefranz: I'm now on conda v4.4.0rc1 (on Linux x86_64) and I continue to get online operations when using --offline:

$ conda create -p /nfs/shared/commonenvs/ana44py36 anaconda=4.4 python=3.6 --offline
Solving environment: done

## Package Plan ##

  environment location: /nfs/shared/commonenvs/ana44py36

  added / updated specs: 
    - anaconda=4.4
    - python=3.6


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    anaconda-navigator-1.6.2   |           py36_0         3.8 MB
    asn1crypto-0.22.0          |           py36_0         148 KB
    anaconda-client-1.6.3      |           py36_0         129 KB
    astroid-1.4.9              |           py36_0         265 KB
    anaconda-project-0.6.0     |           py36_0         209 KB
    _license-1.1               |           py36_1         356 KB
    alabaster-0.7.10           |           py36_0          14 KB
    babel-2.4.0                |           py36_0         4.9 MB
    astropy-1.3.2              |      np112py36_0         9.1 MB
    ------------------------------------------------------------
                                           Total:        18.9 MB

And the obligatory conda info -a:

$ conda info -a

     active environment : None
       user config file : /home/anaconda/.condarc
 populated config files : 
          conda version : 4.4.0rc1
    conda-build version : not installed
         python version : 3.6.1.final.0
       base environment : /nfs/shared/anaconda  (writable)
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
          package cache : /nfs/shared/anaconda/pkgs
                          /home/anaconda/.conda/pkgs
       envs directories : /nfs/shared/anaconda/envs
                          /home/anaconda/.conda/envs
               platform : linux-64
             user-agent : conda/4.4.0rc1 requests/2.14.2 CPython/3.6.1 Linux/3.10.0-514.26.2.el7.x86_64 centos/7 glibc/2.17
                UID:GID : 2112:2112
             netrc file : None
           offline mode : False

# conda environments:
#
base                  *  /nfs/shared/anaconda

sys.version: 3.6.1 |Anaconda 4.4.0 (64-bit)| (default...
sys.prefix: /nfs/shared/anaconda
sys.executable: /nfs/shared/anaconda/bin/python
conda location: /nfs/shared/anaconda/lib/python3.6/site-packages/conda
conda-build: None
conda-env: /nfs/shared/anaconda/bin/conda-env
conda-server: /nfs/shared/anaconda/bin/conda-server
user site dirs: 

CONDA_DEFAULT_ENV: <not set>
CONDA_ENVS_PATH: <not set>
LD_LIBRARY_PATH: <not set>
PATH: /nfs/shared/anaconda/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/anaconda/.local/bin:/home/anaconda/bin
PYTHONHOME: <not set>
PYTHONPATH: <not set>

License directories:
    /home/anaconda/.continuum
    /nfs/shared/anaconda/licenses
License files (license*.txt):
Package/feature end dates:

[ijstokes]

OK, interesting: with the example above I selected "N" to not proceed, but then on the same system I tried a variation and had a repeat of the RuntimeError already reported above in #5370 (comment) above

$ conda create -p /nfs/shared/commonenvs/py36 python=3.6 --offline
Solving environment: done

## Package Plan ##

  environment location: /nfs/shared/commonenvs/py36

  added / updated specs: 
    - python=3.6


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    python-3.6.2               |                0        16.5 MB
    xz-5.2.3                   |                0         667 KB
    zlib-1.2.11                |                0         109 KB
    ------------------------------------------------------------
                                           Total:        17.2 MB

The following NEW packages will be INSTALLED:

    openssl:    1.0.2l-0     
    pip:        9.0.1-py36_1 
    python:     3.6.2-0      
    readline:   6.2-2        
    setuptools: 27.2.0-py36_0
    sqlite:     3.13.0-0     
    tk:         8.5.18-0     
    wheel:      0.29.0-py36_0
    xz:         5.2.3-0      
    zlib:       1.2.11-0     

Proceed ([y]/n)? y


Downloading and Extracting Packages
python 3.6.2:                                                                                                                           |   0% 
xz 5.2.3:                                                                                                                               |   0% 
zlib 1.2.11:                                                                                                                            |   0% 

RuntimeError('EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/linux-64/python-3.6.2-0.tar.bz2\nThis command is using a remote connection in offline mode.\n',)
RuntimeError('EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/linux-64/xz-5.2.3-0.tar.bz2\nThis command is using a remote connection in offline mode.\n',)
RuntimeError('EnforceUnusedAdapter called with url https://repo.continuum.io/pkgs/free/linux-64/zlib-1.2.11-0.tar.bz2\nThis command is using a remote connection in offline mode.\n',)

[robgics]

I believe I am experiencing this same issue, or at least the same symptoms. I am trying to install a module into a conda created environment with --offline, and am getting the runtime error. I am using Conda 4.3.29.

I have the standard install of Anaconda 5.0, and I added the theano package.

If I create a new environment, and use "conda install --offline theano", then I get a dozen or so packages it says it will install, along with theano, and then it gives the RunTimeError on the numpy package.

If I create a new environment, and use "conda install --offline anaconda", that succeeds. If I then do the offline theano install, that ALSO succeeds.

Should I post the debug output, or start a new thread about this instead of posting here?

Thanks.

[kalefranz]

@robgics I believe you're seeing expected behavior. The packages that the solver decides to install need to be in the package cache, otherwise conda will go out and try to get them. The solver has no information about which packages are in the package cache and which don't.

However, perhaps with the --offline flag, we should only be exposing to the solver packages that are in the package cache and packages that are already in the target environment. We could probably make that happen.

[kalefranz]

To clarify, what I meant was that --offline flag should mean that we never add full repodata to the index handed to the solver.

[ijstokes]

@kalefranz to this point:

However, perhaps with the --offline flag, we should only be exposing to the solver packages that are in the package cache and packages that are already in the target environment.

What other interpretation than this would make sense in the presence of the --offline flag?

[kalefranz]

The distinction is that we currently use the cached, although full, repodata.json file to construct index. I'm agreeing that the logic is flawed there. In this case, we should only be constructing the index from cached packages on disk, any packages installed in the target prefix, and any channels that are located at file:// urls.

[robgics]

I agree that --offline should mean that it does not try to go online.

What I don't understand is how it doesn't work with the packages that are already installed, namely numpy. The files are all there, because if I install anaconda first and then theano, it all works (all while being disconnected physically from the internet). But if I try to install theano without anaconda first, it dies on numpy.

[kalefranz]

The solver is probably just selecting a version of numpy that is not available locally, and so there's no choice but to try going out and getting it. The solution I outlined above will prevent the solver from even having an unavailable package as an option to choose.

[robgics]

Sounds good, I look forward to it!

[robgics]

Is there a relatively simple workaround in the meantime? It would help my project move forward. Thanks.

[github-actions]

Hi there, thank you for your contribution to Conda!

This issue has been automatically locked since it has not had recent activity after it was closed.

Please open a new issue if needed.