Not working correctly on Fedora when installing from git

[yajo]

Copied from https://bugzilla.redhat.com/show_bug.cgi?id=1960138

Description of problem:
Fedora's pip cannot determine properly the installed package version when done through git.

Version-Release number of selected component (if applicable):
python3-pip-21.0.1-2.fc34.noarch

Steps to Reproduce:

1. pip install git+https://github.com/copier-org/copier
2. copier --version

Actual results: copier 0.0.0

Expected results: copier 6.0.0a6.post27.dev0+60f9160

Additional info:
I'm copier maintainer.

It works as expected in other environments, but for some reason not in Fedora:

❯ podman container run --rm -it docker.io/python:3.9 bash -c 'pip install -q git+https://github.com/copier-org/copier && copier --version'
copier 6.0.0a6.post27.dev0+60f9160

According to Fedora maintainer, probably fixing #39 should fix this issue as a side effect.

However, since this is a separate explicit issue, I think it's better for it to be tracked separately.

See https://bugzilla.redhat.com/show_bug.cgi?id=1960138#c2 for an analysis about why this is happening and how it could be fixed.

Thanks!

[mtkennerly]

Hello, and thanks for the report! I do think #39 will ultimately be the solution to this, just as soon as Poetry 1.2.x is released with the plugin functionality. In the meantime, I'm not sure if there's a good fix. Maybe the plugin could delay some of its own imports until the patched Poetry functions execute? If you'd be willing to test it, I could give you a wheel with that change.

In the linked bug report, I see some references to /tmp/pip-build-env-.... Does it help at all if you use Pip's --use-feature=in-tree-build option (assuming your Pip is new enough)?

[yajo]

Does it help at all if you use Pip's --use-feature=in-tree-build option (assuming your Pip is new enough)?

I had to pip install -U pip to be able to do that, but it didn't work yet:

[root@4ed2863d38b6 /]# copier --version
copier 0.0.0
[root@4ed2863d38b6 /]# pip --version
pip 21.1.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

[mtkennerly]

Thanks for confirming. If you could test this on your system, here's a version of the plugin that delays its third-party imports until they're needed:

poetry_dynamic_versioning-0.12.7+issue.48.r1.zip

The standard library imports still happen when the plugin is loaded, but I'm hoping those are fine.

[yajo]

Thank you. How am I supposed to test that? I tried adding this diff to my pyproject.toml, but it fails.

Diff:

diff --git a/pyproject.toml b/pyproject.toml
index 3f6bd83..835563b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -115,5 +115,5 @@ multi_line_output = 3 # black interop
 use_parentheses = true
 
 [build-system]
-requires = ["poetry_core>=1.0.0", "poetry-dynamic-versioning"]
+requires = ["poetry_core>=1.0.0", "https://github.com/mtkennerly/poetry-dynamic-versioning/files/6515915/poetry_dynamic_versioning-0.12.7%2Bissue.48.r1.zip#egg=poetry-dynamic-versioning"]
 build-backend = "poetry.core.masonry.api"

Failure:

ERROR: file:///var/home/yajo/mydevel/copier has a pyproject.toml file that does not comply with PEP 518: 'build-system.requires' contains an invalid requirement: 'https://github.com/mtkennerly/poetry-dynamic-versioning/files/6515915/poetry_dynamic_versioning-0.12.7%2Bissue.48.r1.zip#egg=poetry-dynamic-versioning'
Cannot determine package name from spec '/var/home/yajo/mydevel/copier'. Check package spec for errors.

[mtkennerly]

The simplest option would probably be to extract and manually install the wheel, then run pip install --no-build-isolation git+https://github.com/copier-org/copier. If you don't have poetry-core installed globally, you'd also need to do pip install poetry-core, but you could remove it again after the test.

Here's an example of what I get on Windows:

$ pip uninstall -y poetry-dynamic-versioning
$ pip install poetry-core
$ pip install --no-build-isolation git+https://github.com/copier-org/copier
$ copier --version
copier 0.0.0

$ pip uninstall -y copier
$ pip install ./poetry_dynamic_versioning-0.12.7+issue.48.r1-py3-none-any.whl
$ pip install --no-build-isolation git+https://github.com/copier-org/copier
$ copier --version
copier 6.0.0a6.post31.dev0+20c151e

[hroncok]

The simplest option would probably be to extract and manually install the wheel, then run pip install --no-build-isolation git+https://github.com/copier-org/copier.

--no-build-isolation actually hides this problem. The problem happens when build isolation happens :)

In the linked bug report, I see some references to /tmp/pip-build-env-.... Does it help at all if you use Pip's --use-feature=in-tree-build option (assuming your Pip is new enough)?

The directory is the build isolation environment, not out of tree build dir.

[hroncok]

@mtkennerly If you give me a patch/commit instead of a tar, I can verify the fix.

[mtkennerly]

--no-build-isolation actually hides this problem. The problem happens when build isolation happens :)

Ah, right 😅

@mtkennerly If you give me a patch/commit instead of a tar, I can verify the fix.

Sure, you can use 302fb47. Thanks!

[hroncok]

(__venv__) [tmp]$ pip install git+https://github.com/copier-org/copier --no-clean
...
Successfully installed ... copier-0.0.0 ...

(__venv__) [tmp]$ PYTHONPATH=/tmp/pip-build-env-enf404b5/site/ PYTHONNOUSERSITE=1 python 
Error processing line 1 of /tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/zzz_poetry_dynamic_versioning.pth:

  Traceback (most recent call last):
    File "/usr/lib64/python3.9/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/poetry_dynamic_versioning/__init__.py", line 7, in <module>
      import jinja2
    File "/tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/jinja2/__init__.py", line 6, in <module>
      from markupsafe import escape
  ModuleNotFoundError: No module named 'markupsafe'

Remainder of file ignored
...

(__venv__) [tmp]$ patch /tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/poetry_dynamic_versioning/__init__.py < 302fb47f.patch 
patching file /tmp/pip-build-env-enf404b5/overlay/lib/python3.9/site-packages/poetry_dynamic_versioning/__init__.py

(__venv__) [tmp]$ git clone https://github.com/copier-org/copier --depth 1 && cd copier/


(__venv__) [copier (master)]$ PYTHONPATH=/tmp/pip-build-env-enf404b5/site/ PYTHONNOUSERSITE=1 python 
Python 3.9.5 (default, May 14 2021, 00:00:00) 
[GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> poetry_dynamic_versioning = sys.modules['poetry_dynamic_versioning']
>>> poetry_dynamic_versioning._state.patched_core_poetry_create
True

This make me quite confident the fix works.

[mtkennerly]

Awesome, thank you for confirming :D In that case, I'll put a release together shortly.

[mtkennerly]

I just released v0.13.0 with this fix. Please let me know if you encounter any further issues.

[yajo]

🎉

❯ pipx install git+https://github.com/copier-org/copier.git
  installed package copier 6.0.0a6.post31.dev0+20c151e, Python 3.9.5
  These apps are now globally available
    - copier
done! ✨ 🌟 ✨

❯ copier --version
copier 6.0.0a6.post31.dev0+20c151e