Pipenv lock changes markers in a non deterministic way #4660
Comments
|
As a temporary workaround, I am removing the Here the section of my pipenv:
stage: analysis
script:
- cp Pipfile.lock Pipfile.lock.old
- pipenv lock
- sed -i '/\"markers\":\ \"python_version/d' Pipfile.lock # remove markers
- sed -i '/\"markers\":\ \"python_version/d' Pipfile.lock.old # remove markers
- git --no-pager diff --no-index Pipfile.lock Pipfile.lock.old | tee diff.log
- RESULT=$( head -n 1 diff.log | { grep -o 'diff' || true; } )
- RETURN=0
- if [ "$RESULT" = "diff" ]; then RETURN=1; echo "Something changed!"; fi
- exit $RETURN
after_script:
- pip install anybadge
- RESULT=$( head -n 1 diff.log | { grep -o 'diff' || true; } )
- SCORE=passed
- if [ "$RESULT" = "diff" ]; then SCORE=outdated; echo "Something changed!"; fi
- anybadge -l pipenv -v $SCORE -f pipenv.svg outdated=red passed=green
artifacts:
paths:
- pipenv.svg
- diff.log
when: always
|
|
Hi, I am also getting this result, but on every |
|
Something changed in the latest version, now I am getting both - sed -i '/\"markers\":[[:print:]]*python_[[:print:]]*version/d' Pipfile.lock # remove markers
- sed -i '/\"markers\":[[:print:]]*python_[[:print:]]*version/d' Pipfile.lock.old # remove markers |
|
@matpompili and @jshwi please see: #4967 Closing this ticket now since we are already tracking it in the newer ticket. |
|
Still having the same behaviour after upgrading to pip 22.0.4. Using pipenv 2022.5.2: diff --git tasks/Pipfile.lock tasks/Pipfile.lock
index f039f8175..086dbcb0c 100644
--- tasks/Pipfile.lock
+++ tasks/Pipfile.lock
@@ -68,7 +68,7 @@
"sha256:211800f725cdecedcbcf4c753bbd22d248312b37d130f06045434acb7d9b34e1",
"sha256:35dd9063244263e63bd0bd24ea61e3015b00272cead084b2c40d788b0f857c46"
],
- "markers": "python_version >= '3.7' and python_version < '4.0'",
+ "markers": "python_version >= '3.7' and python_version < '4'",
"version": "==1.10.0"
},
"certifi": {reproduced with pip 22.1 as well. ...is it just me? |
I'm getting this as well. |
|
Sounds like pip released a new version that may have changed in it, but pipenv vendors in pip 22.0.4 and nothing has been done to test the new version of pip. Perhaps this is a pip bug. |
|
Want to check the behavior of the newly vendored |
|
Hey @matteius , using pip 22.2.1 and pipenv 2022.7.24 keeps altering diff --git tasks/Pipfile.lock tasks/Pipfile.lock
index 2149cfcdd0..2eeb60add2 100644
--- tasks/Pipfile.lock
+++ tasks/Pipfile.lock
@@ -84,7 +84,7 @@
"sha256:84c85a9078b11105f04f3036a9482ae10e4621616db313fe045dd24743a0820d",
"sha256:fe86415d55e84719d75f8b69414f6438ac3547d2078ab91b67e779ef69378412"
],
- "markers": "python_full_version >= '3.6.0'",
+ "markers": "python_version >= '3.6'",
"version": "==2022.6.15"
},
"cffi": { |
|
I believe this is not an issue on the main branch as we force the usage of the vendor'd pip which eliminates the |
|
Still having this on the latest pipenv which should have the fix, right? |
|
@gitpushdashf Well technically it was vendoring of pip that fixed it and then we of course continue to vendor in new versions of pip as they are released. We recently merged in a change that hasn't been released to force the environment to use the vendor'd version of pip in more places but it has not been released yet. There are two possibilities -- one is that the version of pip you have installed doesn't match the version pipenv has vendored and that could perhaps create inconsistencies. Or it is possible that something changed again in pip creating this inconsistency, but it was definitely pip vendoring that fixed it the prior time. |
|
Interesting! I get the inconsistency between runs, however, on the same system. Could that happen from mismatched pip versions, locally? |
|
These overly chatty diffs are slowing down our code reviews. Would it be possible to create a command-line option |
|
@cclauss how would the Also, we just merged in these two PRs that should help in a big way in the next release to not have the markers change during lock (though adding a new dependency could change the ordering, we'll have to see how that goes): I'd recommend we see how that behaves in the next release. We could consider a no-markers or remove-markers option, but I think that would also cause problems for packages that should have markers. |
|
I don't think this is fixed, unfortunately. I ran locking once with the new 2022.8.30 version. I ran it again, and the lock file was the same. However, the third time, I did get a difference: |
|
@gitpushdashf I've run |
|
I could reproduce this. Pipfile[dev-packages]
sphinx = "==4.*"
sphinx-click = "==4.*"
sphinxcontrib-spelling = "==7.*"
click = "==8.0.3"
stdeb = {version="*", markers="sys_platform == 'linux'"}
zipp = {version = "==3.6.0", markers = "python_version < '3.10'"}
pre-commit = "==2.*"
atomicwrites = {version = "*", markers="sys_platform == 'win32'"}
pytest-cov = "==3.*"
typing-extensions = "==4.*"
[packages]
[scripts]
tests = "bash ./run-tests.sh"
test = "pytest -vvs"
[pipenv]
allow_prereleases = truescript.sh#!/usr/bin/env bash
pipenv --rm
pipenv lock --dev
for i in {1..10}:
do
mv Pipfile.lock Pipfile2.lock
pipenv lock --dev &> /dev/null
echo "attemp $i"
diff Pipfile.lock Pipfile2.lock | grep "markers"
doneDockerfileFROM archlinux
RUN pacman -Syu diffutils python3 python-pip --noconfirm
RUN python -m pip install --upgrade pip pipenv
WORKDIR /root
COPY Pipfile .
COPY script.sh .
ENTRYPOINT ["./script.sh"]ResultNo virtualenv has been created for this project yet!
Aborted!
Creating a virtualenv for this project...
Pipfile: /root/Pipfile
Using /usr/sbin/python3 (3.10.6) to create virtualenv...
⠦ Creating virtual environment...created virtual environment CPython3.10.6.final.0-64 in 335ms
creator CPython3Posix(dest=/root/.local/share/virtualenvs/root-BuDEOXnJ, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
added seed packages: pip==22.2.2, setuptools==65.3.0, wheel==0.37.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /root/.local/share/virtualenvs/root-BuDEOXnJ
Locking [packages] dependencies...
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
⠼ Locking..✔ Success!
Warning: atomicwrites==1.4.1; sys_platform == 'win32' doesn't match your environment, its dependencies won't be resolved.
zipp==3.6.0; python_version < '3.10' doesn't match your environment, its dependencies won't be resolved.
Updated Pipfile.lock (7ff3cb)!
attemp 1:
attemp 2:
attemp 3:
attemp 4:
attemp 5:
attemp 6:
< "markers": "python_full_version >= '3.6.0'",
> "markers": "python_version >= '3.6'",
< "markers": "python_full_version >= '3.6.0'",
> "markers": "python_version >= '3.6'",
< "markers": "python_full_version >= '3.6.0'",
> "markers": "python_version >= '3.6'",
< "markers": "python_full_version >= '3.6.0'",
> "markers": "python_version >= '3.6'",
< "markers": "python_full_version >= '3.6.0'",
> "markers": "python_version >= '3.6'",
< "markers": "python_full_version >= '3.6.0'",
> "markers": "python_version >= '3.6'",
< "markers": "python_full_version >= '3.6.0'",
> "markers": "python_version >= '3.6'",
< "markers": "python_full_version >= '3.6.0'",
> "markers": "python_version >= '3.6'",
< "markers": "python_full_version >= '3.6.0'",
> "markers": "python_version >= '3.6'",
attemp 7:
< "markers": "python_version >= '3.6'",
> "markers": "python_full_version >= '3.6.0'",
< "markers": "python_version >= '3.6'",
> "markers": "python_full_version >= '3.6.0'",
< "markers": "python_version >= '3.6'",
> "markers": "python_full_version >= '3.6.0'",
< "markers": "python_version >= '3.6'",
> "markers": "python_full_version >= '3.6.0'",
< "markers": "python_version >= '3.6'",
> "markers": "python_full_version >= '3.6.0'",
< "markers": "python_version >= '3.6'",
> "markers": "python_full_version >= '3.6.0'",
< "markers": "python_version >= '3.6'",
> "markers": "python_full_version >= '3.6.0'",
< "markers": "python_version >= '3.6'",
> "markers": "python_full_version >= '3.6.0'",
< "markers": "python_version >= '3.6'",
> "markers": "python_full_version >= '3.6.0'",
attemp 8:
attemp 9:
attemp 10: |
|
@dqkqd are you able to reproduce it with |
matteius
added
the
Status: Awaiting Update ⏳
|
Yes, it still happens. I created a docker repo for this. docker run $(docker build --quiet https://github.com/dqkqd/pipenv-issues-4660.git)results: |
matteius
added
Type: Bug 🐛
|
Awesome test project @dqkqd !!! On I didn't realize it stops at 15 -- but those were all same too. |
|
I got what you got too on |
matteius
added
the
Status: Awaiting Update ⏳
|
This issue still occurs. Should I change script to |
|
Perhaps too early to tell, but I maybe am onto something about this issue -- I removed the I updated @dqkqd example Dockerfile to install this branch, and ran it three times without issue: I also relocked pipenv 4 times and the markers only changed the first time suggesting that the branch that removes |
|
This is great, thank you! |
and others
Issue description
I'm running pipenv version 2020.11.15 in a Gitlab CI pipeline.
I use
pipenv lockand thengit diff Pipfile.lockto see if the Pipfile.lock was updated, meaning that the version on the repository is not up-to-date anymore.Occasionally, pipenv updates the lock file just changing the Python 4 specifier as follows:
And sometimes it does the opposite thing, changing from
python_version < '4'topython_version < '4.0'.This of course breaks the pipeline, because there is difference in the Pipfile.lock, even though that difference is not significant.
Expected result
pipenv lock should change the lock file in a deterministic way, either always with
python_version < '4'orpython_version < '4.0'.The text was updated successfully, but these errors were encountered: