Issue with docker-pycreds dependency

[smysnk]

Installed /usr/local/lib/python2.7/dist-packages/docker_py-1.10.1-py2.7.egg
Processing dependencies for docker-py==1.10.1
Searching for docker-pycreds>=0.2.0
Reading http://pypi.python.org/simple/docker-pycreds/
Best match: docker-pycreds 0.2.0
Downloading https://pypi.python.org/packages/67/74/e73c8f8f2e84859938c30193bf12ffaae6805f37f8d8171082df76a951a8/docker-pycreds-0.2.0.tar.gz#md5=a39268529154a53f0635a613ded3b4d5
Processing docker-pycreds-0.2.0.tar.gz
Running docker-pycreds-0.2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-njP7iD/docker-pycreds-0.2.0/egg-dist-tmp-YKAYfW
error: ./test-requirements.txt: No such file or directory

[netoneko]

It gets even worse, because in current version of docker-pycreds if you install it manually it will update six to required version, but if you install it as part as pip install docker-compose==1.7.0 it fails to update six and therefore docker-compose command stops working.

[netoneko]

Also, if you run pip install --upgrade docker-compose==1.8.0 it works, if there was no --upgrade, it does not work.

[netoneko]

Seems like the issue comes from this commit (by @shin-): shin-/compose@3f08efb

The main problem is that now clean installation trough pip that used to work for half a year is broken and that might impact production systems for many people around the world.

[netoneko]

Here is the log of Amazon CloudFormation attempting to install docker-compose:

2016-09-11 05:49:40,236 P2570 [INFO] ============================================================
2016-09-11 05:49:40,236 P2570 [INFO] Command 01_install_docker_compose
2016-09-11 05:49:44,219 P2570 [INFO] -----------------------Command Output-----------------------
2016-09-11 05:49:44,219 P2570 [INFO]    You are using pip version 6.1.1, however version 8.1.2 is available.
2016-09-11 05:49:44,219 P2570 [INFO]    You should consider upgrading via the 'pip install --upgrade pip' command.
2016-09-11 05:49:44,219 P2570 [INFO]    Collecting docker-compose==1.7.0
2016-09-11 05:49:44,219 P2570 [INFO]      Downloading docker-compose-1.7.0.tar.gz (141kB)
2016-09-11 05:49:44,219 P2570 [INFO]    Collecting cached-property<2,>=1.2.0 (from docker-compose==1.7.0)
2016-09-11 05:49:44,219 P2570 [INFO]      Downloading cached_property-1.3.0-py2.py3-none-any.whl
2016-09-11 05:49:44,219 P2570 [INFO]    Collecting docopt<0.7,>=0.6.1 (from docker-compose==1.7.0)
2016-09-11 05:49:44,219 P2570 [INFO]      Downloading docopt-0.6.2.tar.gz
2016-09-11 05:49:44,219 P2570 [INFO]    Requirement already satisfied (use --upgrade to upgrade): PyYAML<4,>=3.10 in /usr/lib64/python2.7/dist-packages (from docker-compose==1.7.0)
2016-09-11 05:49:44,220 P2570 [INFO]    Collecting requests<2.8,>=2.6.1 (from docker-compose==1.7.0)
2016-09-11 05:49:44,220 P2570 [INFO]      Downloading requests-2.7.0-py2.py3-none-any.whl (470kB)
2016-09-11 05:49:44,220 P2570 [INFO]    Collecting texttable<0.9,>=0.8.1 (from docker-compose==1.7.0)
2016-09-11 05:49:44,220 P2570 [INFO]      Downloading texttable-0.8.4.tar.gz
2016-09-11 05:49:44,220 P2570 [INFO]    Collecting websocket-client<1.0,>=0.32.0 (from docker-compose==1.7.0)
2016-09-11 05:49:44,220 P2570 [INFO]      Downloading websocket_client-0.37.0.tar.gz (194kB)
2016-09-11 05:49:44,220 P2570 [INFO]    Collecting docker-py<2,>1.7.2 (from docker-compose==1.7.0)
2016-09-11 05:49:44,220 P2570 [INFO]      Downloading docker_py-1.10.1-py2.py3-none-any.whl (48kB)
2016-09-11 05:49:44,220 P2570 [INFO]    Collecting dockerpty<0.5,>=0.4.1 (from docker-compose==1.7.0)
2016-09-11 05:49:44,220 P2570 [INFO]      Downloading dockerpty-0.4.1.tar.gz
2016-09-11 05:49:44,220 P2570 [INFO]    Requirement already satisfied (use --upgrade to upgrade): six<2,>=1.3.0 in /usr/lib/python2.7/dist-packages (from docker-compose==1.7.0)
2016-09-11 05:49:44,220 P2570 [INFO]    Collecting jsonschema<3,>=2.5.1 (from docker-compose==1.7.0)
2016-09-11 05:49:44,220 P2570 [INFO]      Downloading jsonschema-2.5.1-py2.py3-none-any.whl
2016-09-11 05:49:44,220 P2570 [INFO]    Collecting enum34<2,>=1.0.4 (from docker-compose==1.7.0)
2016-09-11 05:49:44,220 P2570 [INFO]      Downloading enum34-1.1.6-py2-none-any.whl
2016-09-11 05:49:44,221 P2570 [INFO]    Collecting ipaddress>=1.0.16 (from docker-py<2,>1.7.2->docker-compose==1.7.0)
2016-09-11 05:49:44,221 P2570 [INFO]      Downloading ipaddress-1.0.17-py2-none-any.whl
2016-09-11 05:49:44,221 P2570 [INFO]    Collecting docker-pycreds>=0.2.0 (from docker-py<2,>1.7.2->docker-compose==1.7.0)
2016-09-11 05:49:44,221 P2570 [INFO]      Downloading docker_pycreds-0.2.0-py2.py3-none-any.whl
2016-09-11 05:49:44,221 P2570 [INFO]    Collecting backports.ssl-match-hostname>=3.5 (from docker-py<2,>1.7.2->docker-compose==1.7.0)
2016-09-11 05:49:44,221 P2570 [INFO]      Downloading backports.ssl_match_hostname-3.5.0.1.tar.gz
2016-09-11 05:49:44,221 P2570 [INFO]    Collecting functools32 (from jsonschema<3,>=2.5.1->docker-compose==1.7.0)
2016-09-11 05:49:44,221 P2570 [INFO]      Downloading functools32-3.2.3-2.zip
2016-09-11 05:49:44,221 P2570 [INFO]    Installing collected packages: cached-property, docopt, requests, texttable, websocket-client, ipaddress, docker-pycreds, backports.ssl-match-hostname, docker-py, dockerpty, functools32, jsonschema, enum34, docker-compose
2016-09-11 05:49:44,221 P2570 [INFO]      Running setup.py install for docopt
2016-09-11 05:49:44,221 P2570 [INFO]      Found existing installation: requests 1.2.3
2016-09-11 05:49:44,221 P2570 [INFO]        Uninstalling requests-1.2.3:
2016-09-11 05:49:44,221 P2570 [INFO]          Successfully uninstalled requests-1.2.3
2016-09-11 05:49:44,221 P2570 [INFO]      Running setup.py install for texttable
2016-09-11 05:49:44,222 P2570 [INFO]      Running setup.py install for websocket-client
2016-09-11 05:49:44,222 P2570 [INFO]      Found existing installation: backports.ssl-match-hostname 3.4.0.2
2016-09-11 05:49:44,222 P2570 [INFO]        Uninstalling backports.ssl-match-hostname-3.4.0.2:
2016-09-11 05:49:44,222 P2570 [INFO]          Successfully uninstalled backports.ssl-match-hostname-3.4.0.2
2016-09-11 05:49:44,222 P2570 [INFO]      Running setup.py install for backports.ssl-match-hostname
2016-09-11 05:49:44,222 P2570 [INFO]      Running setup.py install for dockerpty
2016-09-11 05:49:44,222 P2570 [INFO]      Running setup.py install for functools32
2016-09-11 05:49:44,222 P2570 [INFO]      Running setup.py install for docker-compose
2016-09-11 05:49:44,222 P2570 [INFO]    Successfully installed backports.ssl-match-hostname-3.5.0.1 cached-property-1.3.0 docker-compose-1.7.0 docker-py-1.10.1 docker-pycreds-0.2.0 dockerpty-0.4.1 docopt-0.6.2 enum34-1.1.6 functools32-3.2.3.post2 ipaddress-1.0.17 jsonschema-2.5.1 requests-2.7.0 texttable-0.8.4 websocket-client-0.37.0
2016-09-11 05:49:44,222 P2570 [INFO] ------------------------------------------------------------
2016-09-11 05:49:44,222 P2570 [INFO] Completed successfully.
2016-09-11 05:49:44,223 P2570 [INFO] ============================================================
2016-09-11 05:49:44,223 P2570 [INFO] Command 02_run_docker_pull
2016-09-11 05:49:44,320 P2570 [INFO] -----------------------Command Output-----------------------
2016-09-11 05:49:44,320 P2570 [INFO]    Traceback (most recent call last):
2016-09-11 05:49:44,320 P2570 [INFO]      File "/usr/local/bin/docker-compose", line 5, in <module>
2016-09-11 05:49:44,321 P2570 [INFO]        from pkg_resources import load_entry_point
2016-09-11 05:49:44,321 P2570 [INFO]      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3020, in <module>
2016-09-11 05:49:44,321 P2570 [INFO]        working_set = WorkingSet._build_master()
2016-09-11 05:49:44,321 P2570 [INFO]      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 616, in _build_master
2016-09-11 05:49:44,321 P2570 [INFO]        return cls._build_from_requirements(__requires__)
2016-09-11 05:49:44,321 P2570 [INFO]      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 629, in _build_from_requirements
2016-09-11 05:49:44,321 P2570 [INFO]        dists = ws.resolve(reqs, Environment())
2016-09-11 05:49:44,321 P2570 [INFO]      File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 812, in resolve
2016-09-11 05:49:44,321 P2570 [INFO]        raise VersionConflict(dist, req).with_context(dependent_req)
2016-09-11 05:49:44,321 P2570 [INFO]    pkg_resources.ContextualVersionConflict: (six 1.8.0 (/usr/lib/python2.7/dist-packages), Requirement.parse('six>=1.10.0'), set(['docker-pycreds']))
2016-09-11 05:49:44,321 P2570 [INFO] ------------------------------------------------------------
2016-09-11 05:49:44,321 P2570 [ERROR] Exited with error code 1

If we check docker-compose manually, we'll get:

$ /usr/local/bin/docker-compose -v
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3020, in <module>
    working_set = WorkingSet._build_master()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 616, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 629, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 812, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (six 1.8.0 (/usr/lib/python2.7/dist-packages), Requirement.parse('six>=1.10.0'), set(['docker-pycreds']))

Here is docker-pycreds issue in action:

pip install docker-pycreds==0.2.0
You are using pip version 6.1.1, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Requirement already satisfied (use --upgrade to upgrade): docker-pycreds==0.2.0 in /usr/local/lib/python2.7/site-packages
Collecting six>=1.10.0 (from docker-pycreds==0.2.0)
  Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
  Found existing installation: six 1.8.0
    Uninstalling six-1.8.0:
      Successfully uninstalled six-1.8.0
Successfully installed six-1.10.0

Checking again:

$ /usr/local/bin/docker-compose -v
docker-compose version 1.7.0, build 0d7bf73

[alunduil]

Piggybacking on this issue, the URL for docker-pycreds on PyPI points at (I assume) a private repository so it's not possible to get a complete tarball as an alternative either.

[shin-]

Hey folks, sorry about the difficulties over the past couple days. I just submitted #1202 which should resolve most of the issues with the new docker-pycreds dependency. Thank you for your patience, I'll update again once 1.10.2 is on pypi.

[shin-]

1.10.2 has been pushed to pypi. That version should solve

1. The six dependency being too strict in docker-pycreds
2. The error message about the missing test-requirements.txt
3. The GitHub URL for the docker-pycreds package.

Please let me know if you're still having issues!

[netoneko]

@shin- pleasure doing business with you! Everything works great.