pip install errors: moto/boto3, awscli/colorama, requests?

[jorgeorpinel]

As noted in https://discordapp.com/channels/485586884165107732/565699007037571084/629449391266070563

I already have the repo cloned. I fetch and pull the latest master branch (to this moment) and freshly regenerate my virtualenv in .env, activate it and run pip install -e ".[all,tests]" (as instructed in https://dvc.org/doc/user-guide/contributing). It succeeds but I get the following errors:

Building wheels for collected packages: psutil
...
ERROR: moto 1.3.14.dev302 has requirement boto3>=1.9.201, but you'll have boto3 1.9.115 which is incompatible.
ERROR: awscli 1.16.252 has requirement colorama<=0.3.9,>=0.2.5, but you'll have colorama 0.4.1 which is incompatible.

[pared]

@jorgeorpinel
What can you give some more details about your env? I am unable to reproduce on fresh Python 3.7.4 environment,
tried:
pip install -e ."[all,tests]"
and
pip install -e ."[all,tests]" --no-cache

[jorgeorpinel]

OK so I pull the latest version of master (449f43f3 rn), I make sure no venv is active and remove my previous .env dir. Then:

$ virtualenv --python python3 .env
...
$ source .env/bin/activate
...
$ pip install -e ".[all,tests]"  

Obtaining file:///.../dvc
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting requests>=2.22.0 (from dvc==0.61.2+449f43)
...
Collecting jsonpointer>=1.9 (from jsonpatch->cfn-lint>=0.4.0->moto>=1.3.14.dev55; extra == "tests"->dvc==0.61.2+449f43) ...
Building wheels for collected packages: pathspec, aws-sam-translator
  Building wheel for pathspec (setup.py) ... done
  ...
Successfully built pathspec aws-sam-translator
ERROR: awscli 1.16.252 has requirement colorama<=0.3.9,>=0.2.5, but you'll have colorama 0.4.1 which is incompatible.
ERROR: moto 1.3.14.dev312 has requirement boto3>=1.9.201, but you'll have boto3 1.9.115 which is incompatible.
talling collected packages: certifi, ...
$ which dvc
/.../dvc/.env/bin/dvc
$ dvc version             
DVC version: 0.60.1
Python version: 3.7.3
Platform: Darwin-18.7.0-x86_64-i386-64bit
Binary: False
Cache: reflink - True, hardlink - True, symlink - True
Filesystem type (cache directory): ('apfs', '/dev/disk1s1')
Filesystem type (workspace): ('apfs', '/dev/disk1s1')

See full output for the pip install command in https://pastebin.com/f7c59V4n

[jorgeorpinel]

BTW I just upgraded my system DVC with pip3 install dvc --upgrade and it worked but with 2 red lines:

docker-compose 1.23.2 has requirement requests!=2.11.0,!=2.12.2,!=2.18.0,<2.21,>=2.6.1, but you'll have requests 2.22.0 which is incompatible.
awsebcli 3.15.3 has requirement requests<2.21,>=2.20.1, but you'll have requests 2.22.0 which is incompatible.

which seems like a similar thing.
dvc version after update:

DVC version: 0.61.2
Python version: 3.7.3
Platform: Darwin-18.7.0-x86_64-i386-64bit
Binary: False
Cache: reflink - True, hardlink - True, symlink - True
Filesystem type (cache directory): ('apfs', '/dev/disk1s1')
Filesystem type (workspace): ('apfs', '/dev/disk1s1')

[efiop]

BTW I just upgraded my system DVC with pip3 install dvc --upgrade and it worked but with 2 red lines:

This one is about your previously installed packages, that require something else than what dvc requires. There is nothing we can do about it on our side.

[efiop]

And these

ERROR: awscli 1.16.252 has requirement colorama<=0.3.9,>=0.2.5, but you'll have colorama 0.4.1 which is incompatible.
ERROR: moto 1.3.14.dev312 has requirement boto3>=1.9.201, but you'll have boto3 1.9.115 which is incompatible.

are valid and are our fault, as our all reqs conflict with test reqs, which doesn't affect users, but does affect developers in a minor way. We should fix that, for sure.

[shcheklein]

This one is about your previously installed packages, that require something else than what dvc requires. There is nothing we can do about it on our side.

How is it possible if it was installed in the clean virtualenv?

[efiop]

@shcheklein Apparently it wasn't clean, but rather inherited from the global env. docker-compose and awsebcli are definitely not our dependencies.

[jorgeorpinel]

How is it possible if it was installed in the clean virtualenv?

Ruslan is right about that one @shcheklein. In that #2570 (comment) I was referring to updating my system DVC installed as a package with pip, not to the development version installed as a git repo. Sorry for the confusion, we can safely ignore that comment and focus on #2570 (comment):

are valid and are our fault, as our all reqs conflict with test reqs, which doesn't affect users, but does affect developers in a minor way. We should fix that, for sure.

p.s. do you mind if I hide #2570 (comment) and all its responses, off-topic and now resolved? 😋

[efiop]

p.s. do you mind if I hide #2570 (comment) and all its responses, off-topic and now resolved?

@jorgeorpinel Please don't hide comments. These are related to the topic.

is this still relevant?

[efiop]

@MrOutis Yes, it is, I've seen it myself not that long ago.