Tidying conda, pip, and setup.py

[widdowquinn]

1. The requirements in the three dependency lists do not currently agree with each other.

The conda and pip package managers do not overlap exactly. Some packages are available only in pip and others only in conda/bioconda. The repository contains an environment.yml for conda and a requirements.txt for pip. Additionally, the setup.py file will install biopython and pandas.

These three descriptions of code requirements should - ideally - agree, but do not:

• setup.py pins biopython at >= 1.76 and pandas at >= 1.0.3
• requirements.txt does not specify any versions
• environment.yml pins biopython at 1.78, and pandas at 1.1.2

If packages are to be pinned, they should really be pinned at the same versions, unless there is some specific intent otherwise (and this should be flagged). It is fine to have a specific intent (e.g. environment.yml specifies a very detailed known good environment, but that may be overkill, here)

2. The requirements could be organised differently - and possibly be more easily maintained.

The setup.py file should state which packages are necessary to run, in the install_requires argument, though setuptools does not make any attempt to ensure that the installed packages are mutually compatible. Any version pins here should perhaps take their lead from a known good configuration?

The environment.yml file only works with conda, but it does allow version pinning and also installation of pip-specific packages. The current environment.yml appears to be autogenerated (hence the very specific pins) and includes many packages that are implied dependencies of the required packages. The environment.yml file could be trimmed down considerably to contain only those packages that are necessary to run the script/module (and these then imply the other dependencies). Any pip-only modules can be specified here under a pip field.

The requirements.txt file mixes dependencies necessary for the module/script with those that are required only for development. These could be split into separate requirements.txt (corresponding to install_requires from setup.py and requirements-dev.txt files, so that packages which are not required for the module/script are not installed unnecessarily.

3. The installation instructions can then be streamlined, and separate instructions given for those who are only using, and those who are also maintaining/developing, the tool.

The four routes for a user should be something like:

• python setup.py install (takes dependencies from setup.py's install_requires
• conda env create --file environment.yml (takes dependencies from environment.yml and optionally other pip-specific requirements files
• conda install --file requirements.txt
• pip install -r requirements.txt

then for a developer/maintainer

• conda install --file requirements-dev.txt
• pip install -r requirements-dev.txt

(then pip install -e .)
are good options for installing things like flake8/pylint etc.

4. Some necessary modules were missing from one or more dependency definition:

• requests
• mechanicalsoup
• pyyaml
• tqdm