Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Strange Http error when using Conda 4.2.x with Python 2.7.12 #4010

Closed
mingwandroid opened this issue Dec 6, 2016 · 18 comments
Closed

Strange Http error when using Conda 4.2.x with Python 2.7.12 #4010

mingwandroid opened this issue Dec 6, 2016 · 18 comments
Labels
locked [bot] locked due to inactivity source::anaconda created by members of Anaconda, Inc.

Comments

@mingwandroid
Copy link
Contributor

mingwandroid commented Dec 6, 2016 •
edited

conda search libffi (though any package and any network request at all seems to trigger it) on linux-64 is giving me trouble at present:

Current conda install:

               platform : linux-64
          conda version : 4.2.13.dev8+3484810
       conda is private : False
      conda-env version : 4.2.13.dev8+3484810
    conda-build version : 2.0.11+6.g43b927b
         python version : 2.7.12.final.0
       requests version : 2.11.1
       root environment : /home/ray/mc-x64-2.7  (writable)
    default environment : /home/ray/mc-x64-2.7
       envs directories : /home/ray/mc-x64-2.7/envs
          package cache : /home/ray/mc-x64-2.7/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
                          file:///home/ray/mc-x64-2.7/conda-bld/linux-64
                          file:///home/ray/mc-x64-2.7/conda-bld/noarch
            config file : /home/ray/.condarc
           offline mode : False

.DEBUG conda.fetch:fetch_repodata(87): Opening repodata cache for https://repo.continuum.io/pkgs/pro/noarch at /home/ray/mc-x64-2.7/pkgs/cache/XXXXXXXX.json
DEBUG requests.packages.urllib3.connectionpool:_make_request(401): "GET /pkgs/pro/noarch/repodata.json.bz2 HTTP/1.1" 304 0
DEBUG conda.fetch:fetch_repodata(119): 
> GET /pkgs/pro/noarch/repodata.json.bz2 HTTPS
> User-Agent: conda/4.2.13.dev8+3484810 requests/2.11.1 CPython/2.7.12 Linux/2.6.18-412.el5 CentOS/5.11 glibc/2.5
> Accept: */*
> Accept-Encoding: identity
> Connection: keep-alive
> Cookie: __cfduid=XXXXXXXXXXXXXXXXXXXXXXXXXXXX
> If-Modified-Since: Fri, 12 Feb 2016 07:52:34 GMT
> If-None-Match: "56bd8f42-46"

< HTTPS 304 Not Modified
> Cache-Control: public, max-age=300
> CF-Cache-Status: HIT
> CF-RAY: XXXXXXXXXXXXXXXX-XXX
> Date: Tue, 06 Dec 2016 01:08:34 GMT
> ETag: "XXXXXXXX-XX"
> Expires: Tue, 06 Dec 2016 01:13:34 GMT
> Last-Modified: Fri, 12 Feb 2016 07:52:34 GMT
> Server: cloudflare-nginx
> Vary: Accept-Encoding
> Connection: keep-alive

.DEBUG conda.fetch:fetch_repodata(87): Opening repodata cache for file:///home/ray/mc-x64-2.7/conda-bld/linux-64 at /home/ray/mc-x64-2.7/pkgs/cache/XXXXXXXX.json
ERROR conda._vendor.auxlib.logz:stringify(140): 'NoneType' object has no attribute 'method'
Traceback (most recent call last):
  File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/_vendor/auxlib/logz.py", line 133, in stringify
    requests_models_PreparedRequest_builder(builder, obj.request)
  File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/_vendor/auxlib/logz.py", line 102, in requests_models_PreparedRequest_builder
    builder.append("> {0} {1} {2}".format(request_object.method, request_object.path_url,
AttributeError: 'NoneType' object has no attribute 'method'
DEBUG conda.fetch:fetch_repodata(119): None
An unexpected error has occurred.


...


    Traceback (most recent call last):
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/exceptions.py", line 479, in conda_exception_handler
        return_value = func(*args, **kwargs)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/cli/main.py", line 145, in _main
        exit_code = args.func(args, p)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/cli/main_search.py", line 126, in execute
        execute_search(args, parser)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/cli/main_search.py", line 174, in execute_search
        unknown=args.unknown)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/api.py", line 24, in get_index
        index = fetch_index(channel_urls, use_cache=use_cache, unknown=unknown)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/fetch.py", line 295, in fetch_index
        for url in urls]
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/fetch.py", line 75, in func
        res = f(*args, **kwargs)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/fetch.py", line 223, in fetch_repodata
        e.response.reason if e.response else None)
    CondaHTTPError: HTTP 404 None
    for url <None>
    
    The remote server could not find the channel you requested.
    
    You will need to adjust your conda configuration to proceed.
    Use `conda config --show` to view your configuration's current state.
    Further configuration help can be found at <http://conda.pydata.org/docs/config.html>.

I changed checksums to X's here.

On Python 3.5, I do get `ERROR conda._vendor.auxlib.logz:stringify(140): 'NoneType' object has no attribute 'method' but not the final:

CondaHTTPError: HTTP 404 None
    for url <None>

and on 3.5 I get the list of packages found.
@mingwandroid mingwandroid added the source::anaconda created by members of Anaconda, Inc. label Dec 6, 2016
@kalefranz
Copy link
Contributor

kalefranz commented Dec 6, 2016 via email

@mingwandroid mingwandroid reopened this Dec 6, 2016
@mingwandroid
Copy link
Contributor Author

No stack trace without debug, but I go get the http error. Any thoughts on debugging further?

@kalefranz
Copy link
Contributor

kalefranz commented Dec 6, 2016 via email

@mingwandroid
Copy link
Contributor Author

OK, I tried master branch and that gives me more useful information:

I have, in my ~/.condarc

    - local

.. which AFAIR, I needed to add for the Anaconda build system to work / interact correctly with conda-build.

With this channel, I get the following backtrace:

    Traceback (most recent call last):
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/exceptions.py", line 515, in conda_exception_handler
        return_value = func(*args, **kwargs)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/cli/main.py", line 145, in _main
        exit_code = args.func(args, p)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/cli/main_search.py", line 125, in execute
        execute_search(args, parser)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/cli/main_search.py", line 174, in execute_search
        unknown=args.unknown)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/core/index.py", line 52, in get_index
        index = fetch_index(channel_urls, use_cache=use_cache, unknown=unknown)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/core/index.py", line 303, in fetch_index
        repodatas = _collect_repodatas(use_cache, urls)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/core/index.py", line 289, in _collect_repodatas
        repodatas = _collect_repodatas_serial(use_cache, urls)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/core/index.py", line 257, in _collect_repodatas_serial
        for url in urls]
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/core/index.py", line 91, in func
        res = f(*args, **kwargs)
      File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda/core/index.py", line 242, in fetch_repodata
        getattr(e.response, 'elapsed', None))
    CondaHTTPError: HTTP 404 None for url <file:///home/ray/mc-x64-2.7/conda-bld/linux-64/repodata.json.bz2>
    Elapsed: 00:00.000654

@kalefranz
Copy link
Contributor

kalefranz commented Dec 6, 2016 via email

@mingwandroid
Copy link
Contributor Author

I'm running 2.11.1, but IMHO "file://" should never even call into requests?

@mingwandroid
Copy link
Contributor Author

mingwandroid commented Dec 6, 2016 •
edited

I tried to see the state of the different branches wrt whether I can use them at present:

master branch doesn't work due to shell changes
4.3.x branch also doesn't work due to incompat. with conda-build:

  File "/home/ray/mc-x64-2.7/lib/python2.7/site-packages/conda_build/conda_interface.py", line 13, in <module>
    from conda.compat import (PY3, StringIO, configparser, input, iteritems, lchmod, string_types,  # NOQA
ImportError: cannot import name PY3

Can I ask why conda_interface.py is in conda-build and not in conda?

@kalefranz
Copy link
Contributor

kalefranz commented Dec 6, 2016 via email

@kalefranz
Copy link
Contributor

kalefranz commented Dec 6, 2016 via email

@kalefranz
Copy link
Contributor

kalefranz commented Dec 6, 2016 via email

@mingwandroid
Copy link
Contributor Author

OK, I'll stick to 4.2.x for now.

@kalefranz
Copy link
Contributor

Ok so in the 4.3.x have test runs for conda-build versions 1.21.14, 2.0.1, and 2.0.10. 4 failures on 1.21.14, which can all be explained I think. 1 failure on 2.0.1, which can also be explained. I had Sophia look into all of these.

2.0.10 has 12 failures, and that's new I think. conda-build is complaining about not being able to import pkginfo, which is part of setuptools. I probably just need to add that as a dependency before running the conda-build tests.

Go ahead and review the conda-build tests here if you want.
https://travis-ci.org/conda/conda/builds/181582304

I'm not seeing anything obvious that's a 4.3.x import problem though.

@mingwandroid
Copy link
Contributor Author

I think that conda calls requests for the channel local is the issue I've been facing here. I'm working around it, but it's pretty ugly. I have to serialize my builds manually; using - local in condarc seemed to be working around some issue or other but I'm not sure where the problem really lies (Anaconda build system, conda-build, conda, or just a bad interaction between them => maybe because both Anaconda build system and conda-build determine their own build graphs?).

That file:// URLs are getting passed to requests seems to be a bug in and of itself though (esp. since they're making a lot of noise lately about rejecting everything that isn't http:// or https://.

I need to get on with building compilers ASAP though so I'll be using crosstool-ng rather than conda to build things for the next while.

Hopefully by the time I'm back to using conda-build we'll have a shiny new build system.

@kalefranz
Copy link
Contributor

Conda's used file:// with requests for a LONG time I think. Showed up somewhere around 3.5.0 circa May 2014.

https://github.com/conda/conda/blob/3.5.0/conda/connection.py#L69

@kalefranz
Copy link
Contributor

Ahh, the import problems you're running into. Not sure why the tests didn't show it. But I'll fix it.

@mingwandroid
Copy link
Contributor Author

I get that conda's used requests with file:// for a very long time but unless I am misunderstanding something (very possible) you should probably look at:

kennethreitz/requests@34af72c

I've made a pull request to illustrate the problem: #4044, although the callstack is slightly different than 7 days ago, IMHO the root cause is the same.

@jakirkham jakirkham mentioned this issue Feb 25, 2017
Merged
@kalefranz
Copy link
Contributor

I think all the issues here have been resolved now. Let me know if that's not true.

@github-actions
Copy link

Hi there, thank you for your contribution to Conda!

This issue has been automatically locked since it has not had recent activity after it was closed.

Please open a new issue if needed.

@github-actions github-actions bot added the locked [bot] locked due to inactivity label Oct 22, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 22, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
locked [bot] locked due to inactivity source::anaconda created by members of Anaconda, Inc.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mingwandroid @kalefranz