Skip to content

Commit

Permalink
Alter dependency specification to avoid conflicts
Browse files Browse the repository at this point in the history
  • Loading branch information
@baileythegreen
baileythegreen committed Dec 22, 2021
1 parent f471992 commit 0949a46
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion requirements.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ numpy
openpyxl
pandas
Pillow
PyQt5
PyQt
scipy
seaborn
sqlalchemy==1.3.10
Expand Down

5 comments on commit 0949a46

@peterjc
Copy link
Contributor

@peterjc peterjc commented on 0949a46 Jan 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Does this work in general?

$ python --version
Python 3.7.9
$ pip install PyQt
ERROR: Could not find a version that satisfies the requirement PyQt (from versions: none)
ERROR: No matching distribution found for PyQt
$ pip install PyQt5
Requirement already satisfied: PyQt5 in /.../apps/conda/lib/python3.7/site-packages (5.12.3)

@baileythegreen
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Looks like there are differences in how pip and conda solve the dependency requirements. I see the same thing you do with pip, but when I tried conda this happened:

(with PyQt5 in requirements.txt)

(pyani_dev) ! conda config --add channels anaconda
(pyani_dev) ! make setup_env
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /Users/baileythegreen/Software/miniconda3/envs/pyani_dev

  added / updated specs:
    - bandit
    - black
    - blacken-docs
    - codecov
    - coverage
    - doc8
    - flake8
    - jinja2
    - mypy
    - pydocstyle
    - pylint
    - pytest
    - pytest-cov
    - rstcheck
    - sphinx


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    appdirs-1.4.4              |             py_0          13 KB  anaconda
    black-19.10b0              |             py_0          84 KB  anaconda
    ca-certificates-2020.10.14 |                0         127 KB  anaconda
    certifi-2020.6.20          |           py38_0         159 KB  anaconda
    codecov-2.1.10             |             py_0          20 KB  anaconda
    coverage-5.3               |   py38haf1e3a3_0         256 KB  anaconda
    jinja2-2.11.2              |             py_0          97 KB  anaconda
    pydocstyle-5.1.1           |             py_0          37 KB  anaconda
    pytest-cov-2.10.1          |             py_0          20 KB  anaconda
    regex-2020.10.15           |   py38haf1e3a3_0         349 KB  anaconda
    sphinx-3.2.1               |             py_0         1.4 MB  anaconda
    ------------------------------------------------------------
                                           Total:         2.6 MB

The following NEW packages will be INSTALLED:

  appdirs            anaconda/noarch::appdirs-1.4.4-py_0
  regex              anaconda/osx-64::regex-2020.10.15-py38haf1e3a3_0

The following packages will be SUPERSEDED by a higher-priority channel:

  black              conda-forge::black-21.12b0-pyhd8ed1ab~ --> anaconda::black-19.10b0-py_0
  ca-certificates    conda-forge::ca-certificates-2021.10.~ --> anaconda::ca-certificates-2020.10.14-0
  certifi            conda-forge::certifi-2021.10.8-py38h5~ --> anaconda::certifi-2020.6.20-py38_0
  codecov            conda-forge::codecov-2.1.11-pyhd3deb0~ --> anaconda::codecov-2.1.10-py_0
  coverage           conda-forge::coverage-6.2-py38h96a096~ --> anaconda::coverage-5.3-py38haf1e3a3_0
  jinja2             conda-forge::jinja2-3.0.3-pyhd8ed1ab_0 --> anaconda::jinja2-2.11.2-py_0
  pydocstyle         conda-forge::pydocstyle-6.1.1-pyhd8ed~ --> anaconda::pydocstyle-5.1.1-py_0
  pytest-cov         conda-forge::pytest-cov-3.0.0-pyhd8ed~ --> anaconda::pytest-cov-2.10.1-py_0
  sphinx             conda-forge::sphinx-4.3.2-pyh6c4a22f_0 --> anaconda::sphinx-3.2.1-py_0



Downloading and Extracting Packages
pydocstyle-5.1.1     | 37 KB     | ######################################################################### | 100% 
ca-certificates-2020 | 127 KB    | ######################################################################### | 100% 
regex-2020.10.15     | 349 KB    | ######################################################################### | 100% 
pytest-cov-2.10.1    | 20 KB     | ######################################################################### | 100% 
coverage-5.3         | 256 KB    | ######################################################################### | 100% 
codecov-2.1.10       | 20 KB     | ######################################################################### | 100% 
appdirs-1.4.4        | 13 KB     | ######################################################################### | 100% 
sphinx-3.2.1         | 1.4 MB    | ######################################################################### | 100% 
jinja2-2.11.2        | 97 KB     | ######################################################################### | 100% 
black-19.10b0        | 84 KB     | ######################################################################### | 100% 
certifi-2020.6.20    | 159 KB    | ######################################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - pyqt5

Current channels:

  - https://conda.anaconda.org/anaconda/osx-64
  - https://conda.anaconda.org/anaconda/noarch
  - https://conda.anaconda.org/conda-forge/osx-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://conda.anaconda.org/bioconda/osx-64
  - https://conda.anaconda.org/bioconda/noarch
  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.


make: *** [setup_env] Error 1

(with PyQt in requirements.txt)

(pyani_dev) ! conda install -c anaconda pyqt
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /Users/baileythegreen/Software/miniconda3/envs/pyani_dev

  added / updated specs:
    - pyqt


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    dbus-1.13.18               |       h18a8e69_0         550 KB  anaconda
    expat-2.2.10               |       hb1e8313_2         127 KB  anaconda
    gettext-0.19.8.1           |       h15daf44_3         3.4 MB  anaconda
    glib-2.56.2                |       hd9629dc_0         4.7 MB  anaconda
    icu-58.2                   |       h0a44026_3        22.6 MB  anaconda
    jpeg-9b                    |       he5867d9_2         236 KB  anaconda
    libiconv-1.16              |       h1de35cc_0         1.3 MB  anaconda
    libpng-1.6.37              |       ha441bb4_0         325 KB  anaconda
    pcre-8.44                  |       hb1e8313_0         226 KB  anaconda
    pyqt-5.9.2                 |   py38h655552a_2         4.4 MB  anaconda
    qt-5.9.7                   |       h468cd18_1        76.8 MB  anaconda
    sip-4.19.8                 |   py38h0a44026_0         253 KB  anaconda
    ------------------------------------------------------------
                                           Total:       114.9 MB

The following NEW packages will be INSTALLED:

  dbus               anaconda/osx-64::dbus-1.13.18-h18a8e69_0
  expat              anaconda/osx-64::expat-2.2.10-hb1e8313_2
  gettext            anaconda/osx-64::gettext-0.19.8.1-h15daf44_3
  glib               anaconda/osx-64::glib-2.56.2-hd9629dc_0
  icu                anaconda/osx-64::icu-58.2-h0a44026_3
  jpeg               anaconda/osx-64::jpeg-9b-he5867d9_2
  libiconv           anaconda/osx-64::libiconv-1.16-h1de35cc_0
  libpng             anaconda/osx-64::libpng-1.6.37-ha441bb4_0
  pcre               anaconda/osx-64::pcre-8.44-hb1e8313_0
  pyqt               anaconda/osx-64::pyqt-5.9.2-py38h655552a_2
  qt                 anaconda/osx-64::qt-5.9.7-h468cd18_1
  sip                anaconda/osx-64::sip-4.19.8-py38h0a44026_0


Proceed ([y]/n)? y


Downloading and Extracting Packages
gettext-0.19.8.1     | 3.4 MB    | ######################################################################### | 100% 
glib-2.56.2          | 4.7 MB    | ######################################################################### | 100% 
icu-58.2             | 22.6 MB   | ######################################################################### | 100% 
sip-4.19.8           | 253 KB    | ######################################################################### | 100% 
jpeg-9b              | 236 KB    | ######################################################################### | 100% 
qt-5.9.7             | 76.8 MB   | ######################################################################### | 100% 
libiconv-1.16        | 1.3 MB    | ######################################################################### | 100% 
dbus-1.13.18         | 550 KB    | ######################################################################### | 100% 
libpng-1.6.37        | 325 KB    | ######################################################################### | 100% 
pyqt-5.9.2           | 4.4 MB    | ######################################################################### | 100% 
expat-2.2.10         | 127 KB    | ######################################################################### | 100% 
pcre-8.44            | 226 KB    | ######################################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

I will need to look into the best way to handle these not playing well together.

@peterjc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh dear, that's be painful - I've been lucky to date, and not run into one of these namespaces clashes between pip and conda.

@widdowquinn
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fundamental issue is that we're using CircleCI and pip exclusively to install packages, but pyqt/PyQt5 has different names in pip/conda, so we must choose one. pyqt installs OK with conda - but our CircleCI builds fail with that.

If we choose conda we either need to use miniconda/conda in our CircleCI config, or move to another CI (e.g. GitHub Actions) or similar.

The path of least resistance - while we still need to install via both conda and pip (which we do because some packages are in pip and not conda) - is the one chosen here, and that's perfectly fine.

We should try to keep this in mind though, should all the packages become available in conda and/or we swap to another CI, as we might want to revisit this, then.

@peterjc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest making it clear in the requirements.txt which system is being used (if sadly it cannot be both) with a human readable comment,

e.g. https://github.com/peterjc/thapbi-pict/blob/v0.11.2/requirements.txt

Please sign in to comment.