Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PoC - Removed setup.py to introduce poetry #391

Closed
wants to merge 2 commits into from
Closed

PoC - Removed setup.py to introduce poetry #391

wants to merge 2 commits into from

Conversation

fao89
Copy link
Member

@fao89 fao89 commented May 11, 2020 •
edited

@pulpbot
Copy link
Member

pulpbot commented May 11, 2020

Attached issue: https://pulp.plan.io/issues/6708

@fao89
Fixing travis
04e5c14 
[noissue]
@dralley
Copy link
Collaborator

dralley commented May 11, 2020

Totally on board with this change but I'm unclear on what the justification is since the associated task has an empty description. Is there a discussion to be linked or background info that could be added?

@fao89
Copy link
Member Author

fao89 commented May 12, 2020

Totally on board with this change but I'm unclear on what the justification is since the associated task has an empty description. Is there a discussion to be linked or background info that could be added?

I forgot to copy the description from the epic, I'll update there.
Basically it started with this PR:
#390
I'm working on automating the first 6 steps of the release and having the requirements on setup.py was causing some undesired effects. So I filed: https://pulp.plan.io/issues/6707
Since I would have to touch on this piece of code, I decided to open this PR as PoC and see what people think about using poetry

@fao89 fao89 mentioned this pull request May 12, 2020
Merged
@fao89
Copy link
Member Author

fao89 commented May 12, 2020 •
edited

Downside, it would require to introduce poetry on pulp_installer:

TASK [pulp : Install Pulp plugins from source] *********************************
ok: [pulp3-source-fedora30] => (item={'key': 'pulp-ansible', 'value': {'source_dir': '/home/vagrant/devel/pulp_ansible'}})
ok: [pulp3-source-fedora30] => (item={'key': 'pulp-rpm', 'value': {'source_dir': '/home/vagrant/devel/pulp_rpm', 'prereq_role': 'pulp.pulp_rpm_prerequisites'}})
failed: [pulp3-source-fedora30] (item={'key': 'pulp-file', 'value': {'source_dir': '/home/vagrant/devel/pulp_file'}}) => {
    "ansible_loop_var": "item",
    "changed": false,
    "cmd": [
        "/usr/local/lib/pulp/bin/pip3",
        "install",
        "-c",
        "/usr/local/lib/pulp/pip_constraints_for_plugins.txt",
        "-e",
        "/home/vagrant/devel/pulp_file"
    ],
    "item": {
        "key": "pulp-file",
        "value": {
            "source_dir": "/home/vagrant/devel/pulp_file"
        }
    }
}

MSG:


:stderr: ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /home/vagrant/devel/pulp_file
(A "pyproject.toml" file was found, but editable mode currently requires a setup.py based build.)

python-poetry/poetry#34 (comment)

But poetry would manage the dependency for us

@fao89 fao89 changed the title Removed setup.py to introduce poetry PoC - Removed setup.py to introduce poetry May 12, 2020
@dralley
Copy link
Collaborator

dralley commented May 12, 2020

As a tool, I really like Poetry, I think it's one of the best pieces of tooling in the Python ecosystem.

However, I'm unsure that we want to go so far as to require our users to use Poetry. Which is something of a chicken and egg problem, but nonetheless.

I also got the impression that the more developer-y bits of Poetry are kind of focused around the experience of a single package rather than the current architecture of Pulp. @rochacbruno funnily enough expressed these exact same concerns a few months ago python-poetry/poetry#34 (comment). I'm not sure if there's a good answer to this at the moment.

@dralley
Copy link
Collaborator

dralley commented May 12, 2020

Basically the current method of doing an "editable" or developer installation of a package with poetry involves making a change to the file that describes the requirements, it's not something you can do from an external context. That would make things more difficult for us, I think., unless there are new developments I'm unaware of.

@fao89
Copy link
Member Author

fao89 commented May 12, 2020 •
edited

@dralley I agree with you, basically, I wanted to see how far I could go using poetry.
I believe with these changes we would not require users to use poetry, with the new version of pip, pyproject.toml is enough to get the installation. The problem is with the development, as you pointed the editable mode does not seem to work for us yet.

@fao89
Copy link
Member Author

fao89 commented May 12, 2020

just tested locally, as user it is possible to install it

pulp_file on  6708 [$] is 📦 v1.0.0.dev via 🐍 v3.8.0 (venv) 
❯ pip install .   
Processing /home/faguiar/project/pulp_file
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: pulpcore>=3.3 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulp-file==1.0.0.dev0) (3.3.1)
Requirement already satisfied: djangorestframework~=3.10.2 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (3.10.3)
Requirement already satisfied: whitenoise<5.1.0,>=4.1.3 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (5.0.1)
Requirement already satisfied: PyYAML<5.4.0,>=5.1.1 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (5.3.1)
Requirement already satisfied: drf-nested-routers~=0.91.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.91)
Requirement already satisfied: aiodns in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.0.0)
Requirement already satisfied: gunicorn<20.1,>=19.9 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (20.0.4)
Requirement already satisfied: dynaconf<4.0,>=2.2 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.2.3)
Requirement already satisfied: coreapi~=2.3.3 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.3.3)
Requirement already satisfied: python-gnupg~=0.4.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.4.6)
Requirement already satisfied: psycopg2<2.9,>=2.7 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.8.5)
Requirement already satisfied: setuptools>=39.2.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (41.2.0)
Requirement already satisfied: pygtrie~=2.3.2 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.3.3)
Requirement already satisfied: django-filter~=2.2.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.2.0)
Requirement already satisfied: redis>=3.4.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (3.5.1)
Requirement already satisfied: aiofiles in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.5.0)
Requirement already satisfied: aiohttp in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (3.6.2)
Requirement already satisfied: Django~=2.2.3 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.2.12)
Requirement already satisfied: django-import-export~=2.0.2 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.0.2)
Requirement already satisfied: backoff in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.10.0)
Requirement already satisfied: drf-yasg~=1.17.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.17.1)
Requirement already satisfied: djangorestframework-queryfields~=1.0.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.0.0)
Requirement already satisfied: rq<1.4,>=1.1 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.3.0)
Requirement already satisfied: pycares>=3.0.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from aiodns->pulpcore>=3.3->pulp-file==1.0.0.dev0) (3.1.1)
Requirement already satisfied: python-box<4.0.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from dynaconf<4.0,>=2.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (3.4.6)
Requirement already satisfied: click in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from dynaconf<4.0,>=2.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (7.1.2)
Requirement already satisfied: toml in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from dynaconf<4.0,>=2.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.10.0)
Requirement already satisfied: python-dotenv in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from dynaconf<4.0,>=2.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.13.0)
Requirement already satisfied: uritemplate in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from coreapi~=2.3.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (3.0.1)
Requirement already satisfied: itypes in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from coreapi~=2.3.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.2.0)
Requirement already satisfied: requests in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from coreapi~=2.3.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.23.0)
Requirement already satisfied: coreschema in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from coreapi~=2.3.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from aiohttp->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.4.2)
Requirement already satisfied: attrs>=17.3.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from aiohttp->pulpcore>=3.3->pulp-file==1.0.0.dev0) (19.3.0)
Requirement already satisfied: multidict<5.0,>=4.5 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from aiohttp->pulpcore>=3.3->pulp-file==1.0.0.dev0) (4.7.5)
Requirement already satisfied: async-timeout<4.0,>=3.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from aiohttp->pulpcore>=3.3->pulp-file==1.0.0.dev0) (3.0.1)
Requirement already satisfied: chardet<4.0,>=2.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from aiohttp->pulpcore>=3.3->pulp-file==1.0.0.dev0) (3.0.4)
Requirement already satisfied: pytz in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from Django~=2.2.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (2020.1)
Requirement already satisfied: sqlparse in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from Django~=2.2.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.3.1)
Requirement already satisfied: diff-match-patch in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (20181111)
Requirement already satisfied: tablib[html,ods,xls,xlsx,yaml]>=0.14.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.1.0)
Requirement already satisfied: six>=1.10.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from drf-yasg~=1.17.0->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.14.0)
Requirement already satisfied: ruamel.yaml>=0.15.34 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from drf-yasg~=1.17.0->pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.16.10)
Requirement already satisfied: inflection>=0.3.1 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from drf-yasg~=1.17.0->pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.4.0)
Requirement already satisfied: packaging in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from drf-yasg~=1.17.0->pulpcore>=3.3->pulp-file==1.0.0.dev0) (20.3)
Requirement already satisfied: cffi>=1.5.0 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from pycares>=3.0.0->aiodns->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.14.0)
Requirement already satisfied: certifi>=2017.4.17 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from requests->coreapi~=2.3.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (2020.4.5.1)
Requirement already satisfied: idna<3,>=2.5 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from requests->coreapi~=2.3.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from requests->coreapi~=2.3.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.25.9)
Requirement already satisfied: jinja2 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from coreschema->coreapi~=2.3.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.11.2)
Requirement already satisfied: markuppy; extra == "html" in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from tablib[html,ods,xls,xlsx,yaml]>=0.14.0->django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.14)
Requirement already satisfied: odfpy; extra == "ods" in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from tablib[html,ods,xls,xlsx,yaml]>=0.14.0->django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.4.1)
Requirement already satisfied: xlrd; extra == "xls" in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from tablib[html,ods,xls,xlsx,yaml]>=0.14.0->django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.2.0)
Requirement already satisfied: xlwt; extra == "xls" in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from tablib[html,ods,xls,xlsx,yaml]>=0.14.0->django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.3.0)
Requirement already satisfied: openpyxl>=2.4.0; extra == "xlsx" in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from tablib[html,ods,xls,xlsx,yaml]>=0.14.0->django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (3.0.3)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.9" in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from ruamel.yaml>=0.15.34->drf-yasg~=1.17.0->pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.2.0)
Requirement already satisfied: pyparsing>=2.0.2 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from packaging->drf-yasg~=1.17.0->pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.4.7)
Requirement already satisfied: pycparser in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from cffi>=1.5.0->pycares>=3.0.0->aiodns->pulpcore>=3.3->pulp-file==1.0.0.dev0) (2.20)
Requirement already satisfied: MarkupSafe>=0.23 in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from jinja2->coreschema->coreapi~=2.3.3->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.1.1)
Requirement already satisfied: defusedxml in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from odfpy; extra == "ods"->tablib[html,ods,xls,xlsx,yaml]>=0.14.0->django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (0.6.0)
Requirement already satisfied: jdcal in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from openpyxl>=2.4.0; extra == "xlsx"->tablib[html,ods,xls,xlsx,yaml]>=0.14.0->django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.4.1)
Requirement already satisfied: et_xmlfile in /home/faguiar/.pyenv/versions/3.8.0/envs/venv/lib/python3.8/site-packages (from openpyxl>=2.4.0; extra == "xlsx"->tablib[html,ods,xls,xlsx,yaml]>=0.14.0->django-import-export~=2.0.2->pulpcore>=3.3->pulp-file==1.0.0.dev0) (1.0.1)
Building wheels for collected packages: pulp-file
  Building wheel for pulp-file (PEP 517) ... done
  Created wheel for pulp-file: filename=pulp_file-1.0.0.dev0-py3-none-any.whl size=47572 sha256=0fd49b716eb7ffd568cd030ab0105e0a6c0b4542894bcebd19dbd31307395da8
  Stored in directory: /tmp/pip-ephem-wheel-cache-63u3ap26/wheels/e3/76/21/28af005ddbaab0a7b348121201b846fa0f0ce5314c5a02a88a
Successfully built pulp-file
Installing collected packages: pulp-file
  Attempting uninstall: pulp-file
    Found existing installation: pulp-file 1.0.0.dev0
    Uninstalling pulp-file-1.0.0.dev0:
      Successfully uninstalled pulp-file-1.0.0.dev0
Successfully installed pulp-file-1.0.0.dev0

As developer, not yet 😢

pulp_file on  6708 [$] is 📦 v1.0.0.dev via 🐍 v3.8.0 (venv) 
❯ pip install -e .
ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /home/faguiar/project/pulp_file
(A "pyproject.toml" file was found, but editable mode currently requires a setup.py based build.)

@dralley
Copy link
Collaborator

dralley commented May 12, 2020

I definitely support the experiment :) It's good to know what the possible blockers are so that we can watch developments in that space.

@fao89 fao89 closed this May 15, 2020
@fao89 fao89 mentioned this pull request May 13, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@fao89 @pulpbot @dralley