Lack of max python version in Summit 0.8.4

[ilario]

• Operating System: Arch
• Python version: 3.10.6
• summit version used: 0.8.4

Description

I installed Summit via pip and the system installed 0.8.4 even if 0.8.6 is the latest release.
I suppose this happened due to the fact that in the sources of 0.8.4 there is no indication that Python 3.10 is not supported yet

summit/pyproject.toml

Line 14 in a1cecdd

python = "^3.7"

while this is present in the sources of 0.8.5 and 0.8.6.
This caused the installation of incompatible versions of BoTorch and GPyTorch resulting in messages like this:

Requirement already satisfied: botorch<0.6.0,>=0.5.0 in /usr/lib/python3.10/site-packages (from summit) (0.5.1)
Requirement already satisfied: gpytorch<2.0.0,>=1.3.0 in /usr/lib/python3.10/site-packages (from summit) (1.9.0)
[...]
Python 3.10.6 (main, Aug  3 2022, 17:39:45) [GCC 12.1.1 20220730] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from summit.benchmarks import SnarBenchmark
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/site-packages/summit/__init__.py", line 19, in <module>
    from summit.run import *
  File "/usr/lib/python3.10/site-packages/summit/run.py", line 1, in <module>
    from summit.strategies import Strategy, strategy_from_dict
  File "/usr/lib/python3.10/site-packages/summit/strategies/__init__.py", line 9, in <module>
    from .multitask import MTBO, STBO
  File "/usr/lib/python3.10/site-packages/summit/strategies/multitask.py", line 6, in <module>
    from botorch.acquisition import ExpectedImprovement as EI
  File "/usr/lib/python3.10/site-packages/botorch/__init__.py", line 7, in <module>
    from botorch import (
  File "/usr/lib/python3.10/site-packages/botorch/acquisition/__init__.py", line 7, in <module>
    from botorch.acquisition.acquisition import (
  File "/usr/lib/python3.10/site-packages/botorch/acquisition/acquisition.py", line 18, in <module>
    from botorch.models.model import Model
  File "/usr/lib/python3.10/site-packages/botorch/models/__init__.py", line 7, in <module>
    from botorch.models.cost import AffineFidelityCostModel
  File "/usr/lib/python3.10/site-packages/botorch/models/cost.py", line 16, in <module>
    from botorch.models.deterministic import DeterministicModel
  File "/usr/lib/python3.10/site-packages/botorch/models/deterministic.py", line 20, in <module>
    from botorch.models.model import Model
  File "/usr/lib/python3.10/site-packages/botorch/models/model.py", line 19, in <module>
    from botorch.models.utils import fantasize as fantasize_flag
  File "/usr/lib/python3.10/site-packages/botorch/models/utils.py", line 23, in <module>
    from gpytorch.utils.broadcasting import _mul_broadcast_shape
ImportError: cannot import name '_mul_broadcast_shape' from 'gpytorch.utils.broadcasting' (/usr/lib/python3.10/site-packages/gpytorch/utils/broadcasting.py)

The incompatibility is mentioned here:
pytorch/botorch#1370

Adding a maximum Python version in the old Summit releases should fix.

Thanks for your work.

[marcosfelt]

Hi @ilario! unfortunately, I can't update the old Summit versions, and I unfortunately only added the python 3.10 in the latest versions, as you noted (see #162). Are you able to use version 0.8.6?

pip install summit==0.8.6 or pip install -U summit

[ilario]

Hi @marcosfelt, thanks for the answer!
On my PC seems that neither 0.8.6 likes Python 3.10:

pip install summit==0.8.6
Defaulting to user installation because normal site-packages is not writeable
ERROR: Ignored the following versions that require a different python version: 0.8.5 Requires-Python >=3.8,<3.10; 0.8.6 Requires-Python >=3.8,<3.10
ERROR: Could not find a version that satisfies the requirement summit==0.8.6 (from versions: 0.5.0, 0.6.0, 0.8.0rc0, 0.8.0rc1, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4)
ERROR: No matching distribution found for summit==0.8.6

Even more strange, after installing python39 package from AUR (so that now I have both interpreters, 3.10.6 and 3.9.13) I get the exact same error, so now I maybe have to try using virtualenv?

[marcosfelt]

Yes, you'll want to create a virtual environment:

python3.9 -m venv .venv
source .venv/bin/activate
pip install -U summit

[ilario]

Before reading your comment (tomorrow I'll try again using your comment), I did:

$ virtualenv --python=/usr/bin/python3.9 py39
$ py39 bin/pip3.9 install summit==0.8.6
[...]
Collecting gpytorch<2.0.0,>=1.5.0
  Using cached gpytorch-1.9.0-py3-none-any.whl (245 kB)
[...]
Collecting botorch<0.6.0,>=0.5.0
  Using cached botorch-0.5.1-py3-none-any.whl (486 kB)

and I still have the same compatibility problem between the selected versions of BoTorch and GPyTorch:

$ bin/python3.9 
Python 3.9.13 (main, Sep  7 2022, 09:33:07) 
[GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from summit.benchmarks import SnarBenchmark
[...]
  File "/home/ilario/py39/lib/python3.9/site-packages/botorch/models/utils.py", line 23, in <module>
    from gpytorch.utils.broadcasting import _mul_broadcast_shape
ImportError: cannot import name '_mul_broadcast_shape' from 'gpytorch.utils.broadcasting' (/home/ilario/py39/lib/python3.9/site-packages/gpytorch/utils/broadcasting.py)

Tomorrow I'll try with the instructions in your comment.

[marcosfelt]

I was reading quickly and missed the issue with BOtorch. It looks like this can be fixed by upgrading to Botorch 0.7.0. For now, manually install Botorch, and I'll try to get a new version of Summit pushed ASAP.

[ilario]

And now PyTorch does support Python 3.10, since PyTorch 1.11. So the maximum version of Python could be removed, right?

[ilario]

Just tried your instructions, and installing BoTorch 0.7.0 before installing Summit.
The same thing happens:
for some reason, in the list of dependencies the BoTorch one is indicated as if the version has to be (>=0.5.0,<0.6.0). No idea where the maximum limit comes from, as in the Summit repo I can find only the minimum one.

[marcosfelt]

Can you try one more thing: pip install gpytorch==1.8.1 - this worked for me on another project. I'm going to work on pushing the new version right now as well. Thanks for continuing to work on this!

[ilario]

Amazing, installing the gpytorch 1.8.1 package solved the issue :D thanks!!!!!!!
I tried running some of the examples, all the ones from the Already Implemented Benchmarks work.
It looks amazing :D
But the example in the readme fails with "ValueError: Simplex is overfull in one dimension. Please increase threshold for stopping."

[marcosfelt]

Okay perfect! The new version should be up as soon as the tests pass and the PR is merged.

But the example in the readme fails with "ValueError: Simplex is overfull in one dimension. Please increase threshold for stopping."

Could you open a new issue for this please? :)

[marcosfelt]

Now available here: https://github.com/sustainable-processes/summit/releases/tag/0.8.7

[ilario]

Many thanks for the release :D

But the example in the readme fails with "ValueError: Simplex is overfull in one dimension. Please increase threshold for stopping."

Could you open a new issue for this please? :)

Yess :D here it is: #201