Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not working correctly on Fedora when installing from git #48

Closed
yajo opened this issue May 13, 2021 · 12 comments
Closed

Not working correctly on Fedora when installing from git #48

yajo opened this issue May 13, 2021 · 12 comments
Labels
bug Something isn't working
Milestone

Comments

@yajo
Copy link

yajo commented May 13, 2021

Copied from https://bugzilla.redhat.com/show_bug.cgi?id=1960138

Description of problem:
Fedora's pip cannot determine properly the installed package version when done through git.

Version-Release number of selected component (if applicable):
python3-pip-21.0.1-2.fc34.noarch

Steps to Reproduce:

  1. pip install git+https://github.com/copier-org/copier
  2. copier --version

Actual results: copier 0.0.0

Expected results: copier 6.0.0a6.post27.dev0+60f9160

Additional info:
I'm copier maintainer.

It works as expected in other environments, but for some reason not in Fedora:

❯ podman container run --rm -it docker.io/python:3.9 bash -c 'pip install -q git+https://github.com/copier-org/copier && copier --version'
copier 6.0.0a6.post27.dev0+60f9160

According to Fedora maintainer, probably fixing #39 should fix this issue as a side effect.

However, since this is a separate explicit issue, I think it's better for it to be tracked separately.

See https://bugzilla.redhat.com/show_bug.cgi?id=1960138#c2 for an analysis about why this is happening and how it could be fixed.

Thanks!

@yajo yajo changed the title Not working correctly on Fedora Not working correctly on Fedora when installing from git May 13, 2021
@mtkennerly mtkennerly added the bug Something isn't working label May 13, 2021
@mtkennerly
Copy link
Owner

Hello, and thanks for the report! I do think #39 will ultimately be the solution to this, just as soon as Poetry 1.2.x is released with the plugin functionality. In the meantime, I'm not sure if there's a good fix. Maybe the plugin could delay some of its own imports until the patched Poetry functions execute? If you'd be willing to test it, I could give you a wheel with that change.

In the linked bug report, I see some references to /tmp/pip-build-env-.... Does it help at all if you use Pip's --use-feature=in-tree-build option (assuming your Pip is new enough)?

@yajo
Copy link
Author

yajo commented May 19, 2021

Does it help at all if you use Pip's --use-feature=in-tree-build option (assuming your Pip is new enough)?

I had to pip install -U pip to be able to do that, but it didn't work yet:

[root@4ed2863d38b6 /]# copier --version
copier 0.0.0
[root@4ed2863d38b6 /]# pip --version
pip 21.1.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

@mtkennerly
Copy link
Owner

Thanks for confirming. If you could test this on your system, here's a version of the plugin that delays its third-party imports until they're needed:

poetry_dynamic_versioning-0.12.7+issue.48.r1.zip

The standard library imports still happen when the plugin is loaded, but I'm hoping those are fine.

@yajo
Copy link
Author

yajo commented May 22, 2021

Thank you. How am I supposed to test that? I tried adding this diff to my pyproject.toml, but it fails.

Diff:

diff --git a/pyproject.toml b/pyproject.toml
index 3f6bd83..835563b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -115,5 +115,5 @@ multi_line_output = 3 # black interop
 use_parentheses = true
 
 [build-system]
-requires = ["poetry_core>=1.0.0", "poetry-dynamic-versioning"]
+requires = ["poetry_core>=1.0.0", "https://github.com/mtkennerly/poetry-dynamic-versioning/files/6515915/poetry_dynamic_versioning-0.12.7%2Bissue.48.r1.zip#egg=poetry-dynamic-versioning"]
 build-backend = "poetry.core.masonry.api"

Failure:

ERROR: file:///var/home/yajo/mydevel/copier has a pyproject.toml file that does not comply with PEP 518: 'build-system.requires' contains an invalid requirement: 'https://github.com/mtkennerly/poetry-dynamic-versioning/files/6515915/poetry_dynamic_versioning-0.12.7%2Bissue.48.r1.zip#egg=poetry-dynamic-versioning'
Cannot determine package name from spec '/var/home/yajo/mydevel/copier'. Check package spec for errors.

@mtkennerly
Copy link
Owner

The simplest option would probably be to extract and manually install the wheel, then run pip install --no-build-isolation git+https://github.com/copier-org/copier. If you don't have poetry-core installed globally, you'd also need to do pip install poetry-core, but you could remove it again after the test.

Here's an example of what I get on Windows:

$ pip uninstall -y poetry-dynamic-versioning
$ pip install poetry-core
$ pip install --no-build-isolation git+https://github.com/copier-org/copier
$ copier --version
copier 0.0.0

$ pip uninstall -y copier
$ pip install ./poetry_dynamic_versioning-0.12.7+issue.48.r1-py3-none-any.whl
$ pip install --no-build-isolation git+https://github.com/copier-org/copier
$ copier --version
copier 6.0.0a6.post31.dev0+20c151e

@hroncok
Copy link

hroncok commented May 25, 2021

The simplest option would probably be to extract and manually install the wheel, then run pip install --no-build-isolation git+https://github.com/copier-org/copier.

--no-build-isolation actually hides this problem. The problem happens when build isolation happens :)

In the linked bug report, I see some references to /tmp/pip-build-env-.... Does it help at all if you use Pip's --use-feature=in-tree-build option (assuming your Pip is new enough)?

The directory is the build isolation environment, not out of tree build dir.

@hroncok
Copy link

hroncok commented May 25, 2021

@mtkennerly If you give me a patch/commit instead of a tar, I can verify the fix.

@mtkennerly
Copy link
Owner

--no-build-isolation actually hides this problem. The problem happens when build isolation happens :)

Ah, right 😅

@mtkennerly If you give me a patch/commit instead of a tar, I can verify the fix.

Sure, you can use 302fb47. Thanks!

@hroncok
Copy link

hroncok commented May 25, 2021

(__venv__) [tmp]$ pip install git+https://github.com/copier-org/copier --no-clean
...
Successfully installed ... copier-0.0.0 ...

(__venv__) [tmp]$ PYTHONPATH=/tmp/pip-build-env-enf404b5/site/ PYTHONNOUSERSITE=1 python 
Error processing line 1 of /tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/zzz_poetry_dynamic_versioning.pth:

  Traceback (most recent call last):
    File "/usr/lib64/python3.9/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/poetry_dynamic_versioning/__init__.py", line 7, in <module>
      import jinja2
    File "/tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/jinja2/__init__.py", line 6, in <module>
      from markupsafe import escape
  ModuleNotFoundError: No module named 'markupsafe'

Remainder of file ignored
...

(__venv__) [tmp]$ patch /tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/poetry_dynamic_versioning/__init__.py < 302fb47f.patch 
patching file /tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/poetry_dynamic_versioning/__init__.py

(__venv__) [tmp]$ git clone https://github.com/copier-org/copier --depth 1 && cd copier/


(__venv__) [copier (master)]$ PYTHONPATH=/tmp/pip-build-env-enf404b5/site/ PYTHONNOUSERSITE=1 python 
Python 3.9.5 (default, May 14 2021, 00:00:00) 
[GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> poetry_dynamic_versioning = sys.modules['poetry_dynamic_versioning']
>>> poetry_dynamic_versioning._state.patched_core_poetry_create
True

This make me quite confident the fix works.

@mtkennerly
Copy link
Owner

Awesome, thank you for confirming :D In that case, I'll put a release together shortly.

mtkennerly added a commit that referenced this issue May 26, 2021
@mtkennerly mtkennerly added this to the v0.13.0 milestone May 26, 2021
@mtkennerly
Copy link
Owner

I just released v0.13.0 with this fix. Please let me know if you encounter any further issues.

@yajo
Copy link
Author

yajo commented May 26, 2021

🎉

❯ pipx install git+https://github.com/copier-org/copier.git
  installed package copier 6.0.0a6.post31.dev0+20c151e, Python 3.9.5
  These apps are now globally available
    - copier
done! ✨ 🌟 ✨

❯ copier --version
copier 6.0.0a6.post31.dev0+20c151e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants