Update poetry to 1.2.2 and regenerate lockfile

[bodograumann]

inboard depends on gunicorn, which in turn depends on setuptools for pkg_resources.
Unfortunately this sub-dependency is not mentioned in the poetry.lock file.
That means when building a docker file as described in the docs, RUN poetry install will actually uninstall setuptools.

   - Removing setuptools (65.4.1)

That in turn breaks guincorn.

Traceback (most recent call last):
ModuleNotFoundError: No module named 'pkg_resources'
    import pkg_resources
  File "/app/.venv/lib/python3.10/site-packages/gunicorn/util.py", line 25, in <module>
    from gunicorn import util
  File "/app/.venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 11, in <module>
    from gunicorn.app.base import Application
  File "/app/.venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 9, in <module>
    from gunicorn.app.wsgiapp import run
  File "/app/.venv/bin/gunicorn", line 5, in <module>

All I did for this PR is run poetry lock in the repository. It added setuptools in line 199 of poetry.lock as well as in the [metadata.files] section.

[bodograumann]

Mmh, this might actually be caused by the lock file being written with poetry 1.2, while the inboard base image is still supplying poetry 1.1. Let me try updating that and let's see what happens.
Cf. python-poetry/poetry#6328 (comment)

[vercel]

Deployment failed with the following error:

Creating the Deployment Timed Out.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
inboard	✅ Ready (Inspect)	Visit Preview	Oct 13, 2022 at 3:43PM (UTC)

[bodograumann]

Ok, seems like updating poetry does the trick. I have updated the PR title accordingly.

[br3ndonland]

Thanks for taking a look into this problem. This looks like a Poetry 1.2 issue, and I think a more pragmatic solution is just to not update to Poetry 1.2.

Updating to Poetry 1.2 would be a breaking change for this project. If inboard updates to Poetry 1.2, every downstream project that uses inboard also has to update to Poetry 1.2. Otherwise, if downstream projects are still using Poetry 1.1 lockfiles, setuptools will be uninstalled, and everyone's projects will be broken.

Updating the lockfile to the Poetry 1.2 format also apparently creates the converse problem - installing a Poetry 1.2 lockfile with Poetry 1.1 results in Poetry 1.1 uninstalling setuptools.

Example terminal session (expand)

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7
❯ python --version
Python 3.10.7

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7
❯ poetry1.1 --version
Poetry version 1.1.15

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7
❯ poetry1.2 --version
Configuration file exists at /Users/brendon/Library/Application Support/pypoetry, reusing this directory.

Consider moving configuration to /Users/brendon/Library/Preferences/pypoetry, as support for the legacy directory will be removed in an upcoming release.
Poetry (version 1.2.2)

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7
❯ poetry1.1 install -E all
Creating virtualenv inboard in /Users/brendon/dev/inboard/.venv
Installing dependencies from lock file

Package operations: 67 installs, 0 updates, 0 removals

  - Installing six (1.16.0)
  - Installing idna (3.4)
  - Installing markupsafe (2.1.1)
  - Installing pyparsing (3.0.9)
  - Installing python-dateutil (2.8.2)
  - Installing pyyaml (6.0)
  - Installing sniffio (1.3.0)
  - Installing zipp (3.8.1)
  - Installing anyio (3.6.1)
  - Installing attrs (22.1.0)
  - Installing certifi (2022.9.14)
  - Installing charset-normalizer (2.1.1)
  - Installing click (8.1.3)
  - Installing distlib (0.3.6)
  - Installing filelock (3.8.0)
  - Installing ghp-import (2.1.0)
  - Installing importlib-metadata (4.12.0)
  - Installing iniconfig (1.1.1)
  - Installing jinja2 (3.1.2)
  - Installing markdown (3.3.7)
  - Installing mergedeep (1.3.4)
  - Installing packaging (21.3)
  - Installing platformdirs (2.5.2)
  - Installing pluggy (1.0.0)
  - Installing py (1.11.0)
  - Installing pyyaml-env-tag (0.1)
  - Installing tomli (2.0.1)
  - Installing typing-extensions (4.3.0)
  - Installing urllib3 (1.26.12)
  - Installing watchdog (2.1.9)
  - Installing asgiref (3.5.2)
  - Installing cfgv (3.3.1)
  - Installing h11 (0.13.0)
  - Installing httptools (0.5.0)
  - Installing identify (2.5.5)
  - Installing mccabe (0.7.0)
  - Installing mkdocs (1.3.1)
  - Installing mkdocs-material-extensions (1.0.3)
  - Installing mypy-extensions (0.4.3)
  - Installing nodeenv (1.7.0)
  - Installing pathspec (0.10.1)
  - Installing pycodestyle (2.9.1)
  - Installing pydantic (1.10.2)
  - Installing pyflakes (2.5.0)
  - Installing pygments (2.13.0)
  - Installing pymdown-extensions (9.5)
  - Installing pytest (7.1.3)
  - Installing python-dotenv (0.21.0)
  - Installing requests (2.28.1)
  - Installing starlette (0.20.4)
  - Installing toml (0.10.2)
  - Installing uvloop (0.17.0)
  - Installing virtualenv (20.16.5)
  - Installing watchgod (0.8.2)
  - Installing websockets (10.3)
  - Installing black (22.8.0)
  - Installing coverage (6.4.4)
  - Installing fastapi (0.85.0)
  - Installing flake8 (5.0.4)
  - Installing gunicorn (20.1.0)
  - Installing isort (5.10.1)
  - Installing mkdocs-material (8.5.2)
  - Installing mypy (0.971)
  - Installing pre-commit (2.20.0)
  - Installing pytest-mock (3.8.2)
  - Installing pytest-timeout (1.4.2)
  - Installing uvicorn (0.17.6)

Installing the current project: inboard (0.34.0)

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7 took 26s
❯ poetry1.1 shell
Spawning shell within /Users/brendon/dev/inboard/.venv

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7
❯ . /Users/brendon/dev/inboard/.venv/bin/activate

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ pip --version
pip 22.2.2 from /Users/brendon/dev/inboard/.venv/lib/python3.10/site-packages/pip (python 3.10)

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ pip show setuptools
Name: setuptools
Version: 65.0.0
Summary: Easily download, build, install, upgrade, and uninstall Python packages
Home-page: https://github.com/pypa/setuptools
Author: Python Packaging Authority
Author-email: distutils-sig@python.org
License:
Location: /Users/brendon/dev/inboard/.venv/lib/python3.10/site-packages
Requires:
Required-by: gunicorn, nodeenv

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ poetry1.1 show setuptools

  ValueError

  Package setuptools not found

  at ~/.local/pipx/venvs/poetry1-1/lib/python3.10/site-packages/poetry/console/commands/show.py:101 in handle
       97│                     pkg = locked
       98│                     break
       99│
      100│             if not pkg:
    → 101│                 raise ValueError("Package {} not found".format(package))
      102│
      103│             if self.option("tree"):
      104│                 self.display_package_tree(self.io, pkg, locked_repo)
      105│

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ poetry1.2 show setuptools
Configuration file exists at /Users/brendon/Library/Application Support/pypoetry, reusing this directory.

Consider moving configuration to /Users/brendon/Library/Preferences/pypoetry, as support for the legacy directory will be removed in an upcoming release.

Package setuptools not found

inboard on  develop [$] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ poetry1.2 lock --no-update
Configuration file exists at /Users/brendon/Library/Application Support/pypoetry, reusing this directory.

Consider moving configuration to /Users/brendon/Library/Preferences/pypoetry, as support for the legacy directory will be removed in an upcoming release.
Resolving dependencies... (0.2s)

Writing lock file

inboard on  develop [$!] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ poetry1.2 show gunicorn
Configuration file exists at /Users/brendon/Library/Application Support/pypoetry, reusing this directory.

Consider moving configuration to /Users/brendon/Library/Preferences/pypoetry, as support for the legacy directory will be removed in an upcoming release.
 name         : gunicorn
 version      : 20.1.0
 description  : WSGI HTTP Server for UNIX

dependencies
 - setuptools >=3.0

inboard on  develop [$!] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ poetry1.1 install -E all
Installing dependencies from lock file

Package operations: 0 installs, 0 updates, 1 removal

  - Removing setuptools (65.5.0)

Installing the current project: inboard (0.34.0)

inboard on  develop [$!] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ pip show setuptools
WARNING: Package(s) not found: setuptools

inboard on  develop [$!] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ poetry1.1 show setuptools
name         : setuptools
version      : 65.5.0
description  : Easily download, build, install, upgrade, and uninstall Python
            packages

inboard on  develop [$!] is 📦 v0.34.0 via 🐍 v3.10.7 (.venv)
❯ poetry1.2 show setuptools
Configuration file exists at /Users/brendon/Library/Application Support/pypoetry, reusing this directory.

Consider moving configuration to /Users/brendon/Library/Preferences/pypoetry, as support for the legacy directory will be removed in an upcoming release.
 name         : setuptools
 version      : 65.5.0
 description  : Easily download, build, install, upgrade, and uninstall Python packages

required by
 - gunicorn >=3.0
 - nodeenv *

Also note that, in the terminal session above, pip and Poetry report opposite information about setuptools in the same virtualenv - when pip says setuptools is installed, Poetry says it is not installed, and vice versa. They're reading two different versions of setuptools also. Very confusing.

Poetry has been helpful by putting dependency management and packaging together in one tool using pyproject.toml. However, Poetry has also created an enormous number of problems for many maintainers, and Poetry 1.2 introduces an enormous number of changes for a minor version. This setuptools issue is just one of many more problems to come.

I'm considering switching to Hatch. Please see the GitHub discussion I created for context: #56.

Regarding the original post:

That means when building a docker file as described in the docs, RUN poetry install will actually uninstall setuptools.

That would only happen if mixing Poetry 1.1 and 1.2 in downstream projects. The Docker images come with Poetry 1.1 installed, so the RUN poetry install in the example Dockerfile will be using Poetry 1.1.

Suggestions

If you have a downstream project using the inboard Docker images with Poetry 1.2, you can add RUN pipx upgrade poetry to your Dockerfile to install Poetry 1.2.

If you would like to install inboard outside of Docker for local development, but you have Poetry 1.2 installed on your system, you can use pipx to install the project with Poetry 1.1 in a couple of ways:

1. pipx install --suffix 1.1 'poetry>=1.1,<1.2' && poetry1.1 install -E all: install Poetry 1.1 separately from Poetry 1.2. poetry1.1 commands will use Poetry 1.1, and poetry commands will use Poetry 1.2 if you have it installed.
2. pipx run --spec 'poetry>=1.1,<1.2' poetry install -E all: perform a temporary install of Poetry 1.1 just for this one command.

If you would like to move this PR forward, there are a couple of changes related to Poetry 1.1 that could help:

1. You could update inboard to Poetry 1.1.15 (the last 1.1 release)
2. You could add the pipx suggestions above to the docs, on the contributing page and the Docker page.

[bodograumann]

Thanks for the in-depth response and sorry that I don't have much more time to put in here currently.
I did update poetry in our docker image, so that worked for now. Long term we should find a proper solution though.
You have probably seen that now there is even poetry 1.3 with a completely new lockfile format v2.

I hope I find some time to look into hatch and what you wrote in your comparison with poetry.

[br3ndonland]

Thanks for the in-depth response and sorry that I don't have much more time to put in here currently. I did update poetry in our docker image, so that worked for now. Long term we should find a proper solution though. You have probably seen that now there is even poetry 1.3 with a completely new lockfile format v2.

I hope I find some time to look into hatch and what you wrote in your comparison with poetry.

Sounds great, thanks for reporting back! I closed this off for now, because PR #58 migrated the project to Hatch instead, so there's no poetry.lock anymore.

Note that Poetry 1.1 is still installed in the Docker images for backwards compatibility. You can continue using the suggestions I provided in my previous comment (running pipx upgrade poetry to upgrade to a newer version of Poetry). You could also use pipx install poetry>1.2 --force, which will work whether or not Poetry is already installed by pipx. I will eventually have to remove Poetry 1.1 from the Docker containers, because it's no longer maintained, so pipx install poetry>1.2 --force is a more future-proof command than pipx upgrade poetry.

Thanks again for looking into Poetry 1.2. I would welcome other contributions from you in the future.