add passthrough extras for boto3 and awscli

[graingert]

Fixes #357

[graingert]

I'm not 💯% sure this will work - upload it to test.pypi.org first?

[martindurant]

upload it to test.pypi.org first?

What do I need to do?

[graingert]

@martindurant https://packaging.python.org/guides/using-testpypi/

@thehesiod if this works I think we should recommend it: https://github.com/aio-libs/aiobotocore/pull/830/files

but it might be all over by Halloween when pip 20.3 releases

[jrbourbeau]

@graingert you should be able to try out the extras_requires by installing directly from GitHub

pip install "git+https://github.com/graingert/s3fs@patch-1#egg=s3fs[boto3]"

[graingert]

hmm it doesn't seem to work with pip-compile or pip 20.2 :(

https://gist.github.com/graingert/b8c9fa1c3a018310d35146ff24587b3d

[martindurant]

This isn't right?

botocore==1.17.44 \
    --hash=sha256:1b46ffe1d13922066c873323186cbf97e77c137e08e27039d9d684552ccc4892 \
    --hash=sha256:1f6175bf59ffa068055b65f7d703eb1f748c338594a40dfdc645a6130280d8bb \
    # via aiobotocore

[graingert]

it's supposed to install boto3==1.14.44

[martindurant]

...but it's botocore that actually matters :)

[graingert]

sure but it's only a problem if boto3 is added without a pin:

pip install s3fs boto3

[graingert]

@martindurant https://gist.github.com/graingert/1f50abf7a145f6217d12e30dd5b69baf#file-pip-compile-out-L4

[graingert]

@martindurant ok so it turns out delegated/passthrough extras don't work without --use-feature=2020-resolver, so this doesn't really help

(it does make resolving much faster though: https://www.irccloud.com/pastebin/gU599gxM/

[thehesiod]

ya I've wanted to use this at work as well and found this bug: pypa/pip#3516

[martindurant]

Do you want to reboot this PR now?

[graingert]

yep these work for me on pip21

[graingert]

it's also probably worth yanking any of the old releases without .whl files, this way the 2020-resolver won't try to download and execute them - which is pretty slow, and in the future will be able to use --use-feature=fast-deps to download only the metadata portions of .whl files