use requirements.txt in setup.py

[raybellwaves]

Currently setup.py lists the packages
https://github.com/dask/adlfs/blob/master/setup.py#L21L28

I believe it can read the requirements.txt to avoid duplication.

something like

with open('requirements.txt') as f:
    INSTALL_REQUIRES = f.read().strip().split('\n')

[raybellwaves]

Actually not sure. There are differences (possibly on purpose) between

https://github.com/dask/adlfs/blob/master/requirements.txt
and
https://github.com/dask/adlfs/blob/master/setup.py#L22L27

i.e. requirements.txt has two extra packages:
pytest>=5.0,<6.0
requests>=2.22.0,<3.0

[hayesgb]

pytest can certainly be moved to dev-requirements.txt

[raybellwaves]

What about requests? Perhaps it is installed with one of the other packages. I'll check

[raybellwaves]

Tested creating requirements.txt as:

azure-core>=1.5.0
azure-datalake-store>=0.0.46,<0.1
azure-identity
azure-storage-blob>=12.0.0,<12.4.0
fsspec>=0.8.0
msrestazure

and ran:

conda create -n test_env python=3.8
conda activate test_env
pip install -r requirements.txt

which installs all package below. It brings in requests 2.24.0 which fits the pinned version requirement of >=2.22.0,<3.0. Is pinning it at <3.0 superfluous because the latest release is 2.24.0 https://github.com/psf/requests/releases. Given the adlfs works for the latest version of requests does it need to be pinned?.

(test_env) C:\Users\131416\Desktop>conda list
# packages in environment at C:\Users\131416\AppData\Local\Continuum\anaconda3\envs\test_env:
#
# Name                    Version                   Build  Channel
adal                      1.2.4                    pypi_0    pypi
azure-core                1.8.0                    pypi_0    pypi
azure-datalake-store      0.0.49                   pypi_0    pypi
azure-identity            1.4.0                    pypi_0    pypi
azure-storage-blob        12.3.2                   pypi_0    pypi
ca-certificates           2020.6.24                     0
certifi                   2020.6.20                py38_0
cffi                      1.14.2                   pypi_0    pypi
chardet                   3.0.4                    pypi_0    pypi
cryptography              3.0                      pypi_0    pypi
fsspec                    0.8.0                    pypi_0    pypi
idna                      2.10                     pypi_0    pypi
isodate                   0.6.0                    pypi_0    pypi
msal                      1.4.3                    pypi_0    pypi
msal-extensions           0.2.2                    pypi_0    pypi
msrest                    0.6.18                   pypi_0    pypi
msrestazure               0.6.4                    pypi_0    pypi
oauthlib                  3.1.0                    pypi_0    pypi
openssl                   1.1.1g               he774522_1
pip                       20.2.2                   py38_0
portalocker               1.7.1                    pypi_0    pypi
pycparser                 2.20                     pypi_0    pypi
pyjwt                     1.7.1                    pypi_0    pypi
python                    3.8.5                he1778fa_0
python-dateutil           2.8.1                    pypi_0    pypi
pywin32                   228                      pypi_0    pypi
requests                  2.24.0                   pypi_0    pypi
requests-oauthlib         1.3.0                    pypi_0    pypi
setuptools                49.6.0                   py38_0
six                       1.15.0                   pypi_0    pypi
sqlite                    3.33.0               h2a8f88b_0
urllib3                   1.25.10                  pypi_0    pypi
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.16.27012          hf0eaf9b_3
wheel                     0.34.2                   py38_0
wincertstore              0.2                      py38_0
zlib                      1.2.11               h62dcd97_4

[hayesgb]

I've updated setup.py to pull from requirements.txt in commit db9dd79 and pulled it into master. The pin to requests<3.0 exists due to a bias that any major change in the requests package should be validated, but you raise a fair question. Will open another issue to track that conversation.

[raybellwaves]

Believe this can be reopened via #101