Compatibility problem with Python 3.9? #102

mhjacks · 2020-08-06T13:02:29Z

Hello,

I'm the maintainer of git-up in Fedora. Recently we did a mass archive against Python 3.9 and git-up's test started failing. Here's the relevant log:

+ rm -rfv /builddir/build/BUILDROOT/git-up-1.6.1-5.fc33.x86_64/usr/bin/__pycache__
+ /usr/lib/rpm/find-debuginfo.sh -j8 --strict-build-id -m -i --build-id-seed 1.6.1-5.fc33 --unique-debug-suffix -1.6.1-5.fc33.x86_64 --unique-debug-src-base git-up-1.6.1-5.fc33.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /builddir/build/BUILD/git-up-1.6.1
find: 'debug': No such file or directory
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-ldconfig
+ /usr/lib/rpm/brp-compress
+ /usr/lib/rpm/redhat/brp-strip-lto /usr/bin/strip
+ /usr/lib/rpm/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-python-bytecompile '' 1 0
Bytecompiling .py files below /builddir/build/BUILDROOT/git-up-1.6.1-5.fc33.x86_64/usr/lib/python3.9 using /usr/bin/python3.9
+ /usr/lib/rpm/brp-python-hardlink
+ /usr/lib/rpm/redhat/brp-mangle-shebangs
Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.YFhh0w
+ umask 022
+ cd /builddir/build/BUILD
+ cd git-up-1.6.1
+ git config --global user.email koji@fedoraproject.org
+ git config --global user.name 'Koji Build System'
+ PYTHONPATH=/builddir/build/BUILDROOT/git-up-1.6.1-5.fc33.x86_64/usr/lib/python3.9/site-packages
+ nosetests-3.9
.S..S...S........................E..S
======================================================================
ERROR: test suite for <module 'PyGitUp.tests.test_up_to_date' from '/builddir/build/BUILD/git-up-1.6.1/PyGitUp/tests/test_up_to_date.py'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 209, in run
    self.setUp()
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 292, in setUp
    self.setupContext(ancestor)
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 315, in setupContext
    try_run(context, names)
  File "/usr/lib/python3.9/site-packages/nose/util.py", line 471, in try_run
    return func()
  File "/builddir/build/BUILD/git-up-1.6.1/PyGitUp/tests/test_up_to_date.py", line 32, in setup
    repo.remotes.origin.pull()
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 811, in pull
    res = self._get_fetch_info_from_stderr(proc, progress)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 708, in _get_fetch_info_from_stderr
    output.extend(FetchInfo._from_line(self.repo, err_line, fetch_line)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 708, in <genexpr>
    output.extend(FetchInfo._from_line(self.repo, err_line, fetch_line)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 292, in _from_line
    raise ValueError("Failed to parse line: %r" % line)
ValueError: Failed to parse line: '  git config pull.rebase false  # merge (the default strategy)'
-------------------- >> begin captured logging << --------------------
git.cmd: DEBUG: Popen(['git', 'init'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'cat-file', '--batch-check'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['git', 'cat-file', '--batch'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['git', 'checkout', '-b', 'initial'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'checkout', '-b', 'up-to-date'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'clone', '-b', 'up-to-date', '-v', '/tmp/PyGitUp.ufk366oy/master.up-to-date/.git', '/tmp/PyGitUp.ufk366oy/up-to-date'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=True, shell=None, istream=None)
git.repo.base: DEBUG: Cmd(['git', 'clone', '-b', 'up-to-date', '-v', '/tmp/PyGitUp.ufk366oy/master.up-to-date/.git', '/tmp/PyGitUp.ufk366oy/up-to-date'])'s unused stdout: 
git.cmd: DEBUG: Popen(['git', 'pull', '-v', 'origin'], cwd=/tmp/PyGitUp.ufk366oy/up-to-date, universal_newlines=True, shell=None, istream=None)
git.remote: DEBUG: Fetch head lines do not match lines provided via progress information
length of progress lines 6 should be equal to lines in FETCH_HEAD file 3
Will ignore extra progress lines or fetch head lines.
git.remote: DEBUG: info lines: ['  git config pull.rebase false  # merge (the default strategy)', '  git config pull.rebase true   # rebase', '  git config pull.ff only       # fast-forward only', '   eb2d67d..6adb676  up-to-date -> origin/up-to-date', ' = [up to date]      initial    -> origin/initial', ' = [up to date]      master     -> origin/master']
git.remote: DEBUG: head info : ["6adb676af33e7f95cf91d1fc3841adbbbcbc97f7\t\tbranch 'up-to-date' of /tmp/PyGitUp.ufk366oy/master.up-to-date/\n", "eb2d67d81794f11739aa098c22b41e129a2ce70d\tnot-for-merge\tbranch 'initial' of /tmp/PyGitUp.ufk366oy/master.up-to-date/\n", "eb2d67d81794f11739aa098c22b41e129a2ce70d\tnot-for-merge\tbranch 'master' of /tmp/PyGitUp.ufk366oy/master.up-to-date/\n"]
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 36 tests in 3.301s

FAILED (SKIP=4, errors=1)

I'm not sure why there would be a different hash here. Do you think this might be due to a change in Python 3.9?

The text was updated successfully, but these errors were encountered:

msiemens · 2020-08-15T15:17:31Z

Interesting. I suspect this is more due to a Git update than caused by the Python upgrade. I use Python 3.8 locally and tests started to fail for me too. I'll look into this…

msiemens · 2020-08-15T15:52:58Z

Seems like this is related to gitpython-developers/GitPython#1014 which in turn is caused by git/git@d18c950 which was published in Git v2.27.0. The issue has been fixed in GitPython v3.1.3 but we're using GitPython 2.x. Updating to GitPython 3.x means dropping Python 2 support. So I'll publish a release that drops Python 2 support in order to fix this issue.

mhjacks · 2020-08-15T16:02:44Z

Thank you! That will work well for us as we don't support Python 2 in Fedora anymore.

msiemens · 2020-08-15T16:16:02Z

Should be fixed with v2.0.0 🙃

mhjacks · 2020-08-16T00:27:55Z

I don't see setup.py in the new release tarball?

msiemens · 2020-08-25T17:15:40Z

Oh, weird. I forgot to publish the release to PyPI. The release there should contain the setup.py. I switched the build to Poetry which generates the setup.py when building the release artifacts.

mhjacks · 2020-08-25T17:41:55Z

Ah, that makes sense. Thanks! I was unaware that Poetry did that. I'll work on packaging 2.0.0 for Fedora.

mhjacks · 2020-08-25T20:20:04Z

OK, I'm working on packaging, and I've run a super-minor test problem (init.py in tests/ doesn't seem to have import os and the test suite complains about that). But there's also this:

% nosetests-3


..............................E...
======================================================================
ERROR: test suite for <module 'PyGitUp.tests.test_up_to_date' from '/home/mjackson/git-up-2.0.0/PyGitUp/tests/test_up_to_date.py'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 209, in run
    self.setUp()
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 292, in setUp
    self.setupContext(ancestor)
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 315, in setupContext
    try_run(context, names)
  File "/usr/lib/python3.9/site-packages/nose/util.py", line 471, in try_run
    return func()
  File "/home/mjackson/git-up-2.0.0/PyGitUp/tests/test_up_to_date.py", line 30, in setup
    repo.remotes.origin.pull()
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 811, in pull
    res = self._get_fetch_info_from_stderr(proc, progress)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 708, in _get_fetch_info_from_stderr
    output.extend(FetchInfo._from_line(self.repo, err_line, fetch_line)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 708, in <genexpr>
    output.extend(FetchInfo._from_line(self.repo, err_line, fetch_line)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 292, in _from_line
    raise ValueError("Failed to parse line: %r" % line)
ValueError: Failed to parse line: '  git config pull.rebase false  # merge (the default strategy)'
-------------------- >> begin captured logging << --------------------
git.cmd: DEBUG: Popen(['git', 'init'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'cat-file', '--batch-check'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['git', 'cat-file', '--batch'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['git', 'checkout', '-b', 'initial'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'checkout', '-b', 'up-to-date'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'clone', '-b', 'up-to-date', '-v', '/tmp/PyGitUp.y1q5d8ld/master.up-to-date/.git', '/tmp/PyGitUp.y1q5d8ld/up-to-date'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=True, shell=None, istream=None)
git.repo.base: DEBUG: Cmd(['git', 'clone', '-b', 'up-to-date', '-v', '/tmp/PyGitUp.y1q5d8ld/master.up-to-date/.git', '/tmp/PyGitUp.y1q5d8ld/up-to-date'])'s unused stdout: 
git.cmd: DEBUG: Popen(['git', 'pull', '-v', 'origin'], cwd=/tmp/PyGitUp.y1q5d8ld/up-to-date, universal_newlines=True, shell=None, istream=None)
git.remote: DEBUG: Fetch head lines do not match lines provided via progress information
length of progress lines 6 should be equal to lines in FETCH_HEAD file 3
Will ignore extra progress lines or fetch head lines.
git.remote: DEBUG: info lines: ['  git config pull.rebase false  # merge (the default strategy)', '  git config pull.rebase true   # rebase', '  git config pull.ff only       # fast-forward only', '   4ee720c..78fba12  up-to-date -> origin/up-to-date', ' = [up to date]      initial    -> origin/initial', ' = [up to date]      master     -> origin/master']
git.remote: DEBUG: head info : ["78fba12789eaf1eb43fbf45239b0efdb553a062b\t\tbranch 'up-to-date' of /tmp/PyGitUp.y1q5d8ld/master.up-to-date/\n", "4ee720cca1268fb98eaef7f6cb4832d009703729\tnot-for-merge\tbranch 'initial' of /tmp/PyGitUp.y1q5d8ld/master.up-to-date/\n", "4ee720cca1268fb98eaef7f6cb4832d009703729\tnot-for-merge\tbranch 'master' of /tmp/PyGitUp.y1q5d8ld/master.up-to-date/\n"]
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 33 tests in 4.027s

Is this another case of "the output changed" or is it something more significant?

msiemens · 2020-08-26T07:53:54Z

That's strange. Seems like this is the exact issue that you had at the start. Could you post the output of running python -c 'import git; print(git.__version__)' in your build environment where PyGitUp is installed?

mhjacks · 2020-08-26T18:27:05Z

% python -c 'import git; print(git.__version__)'
3.0.9

Which is interesting, because the package claims to be 3.1.0:
python3-GitPython-3.1.0-3.fc33.noarch

Clearly the package is installing 3.0.9 though:
% rpm -ql python3-GitPython-3.1.0-3.fc33.noarch
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/PKG-INFO
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/SOURCES.txt
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/dependency_links.txt
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/not-zip-safe
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/requires.txt
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/top_level.txt

mhjacks · 2020-08-30T03:52:19Z

It appears we're past the initially reported issue, so I'm ready to close this (Fedora mislabeling a package version isn't really your problem). I see you also loosened the dependency on colorama. Thanks! Would you please consider adding the import os to the test suite as well?

msiemens · 2020-09-02T16:05:57Z

Would you please consider adding the import os to the test suite as well?

Where is it missing? As far as I can tell, the tests run without errors

mhjacks · 2020-09-02T19:39:41Z

When I run nosetests on f32

I get this error:

% nosetests
..................................E
======================================================================
ERROR: test suite for <module 'PyGitUp.tests' from '/home/mjackson/git-up-2.0.1/PyGitUp/tests/__init__.py'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/nose/suite.py", line 228, in run
    self.tearDown()
  File "/usr/lib/python3.8/site-packages/nose/suite.py", line 351, in tearDown
    self.teardownContext(ancestor)
  File "/usr/lib/python3.8/site-packages/nose/suite.py", line 367, in teardownContext
    try_run(context, names)
  File "/usr/lib/python3.8/site-packages/nose/util.py", line 471, in try_run
    return func()
  File "/home/mjackson/git-up-2.0.1/PyGitUp/tests/__init__.py", line 43, in teardown
    os.chdir(join(basepath, '..'))
NameError: name 'os' is not defined

----------------------------------------------------------------------
Ran 34 tests in 4.596s

FAILED (errors=1)

I do not see an import os in this file:
https://github.com/msiemens/PyGitUp/blob/master/PyGitUp/tests/__init__.py so I suspect that's why I'm seeing this error. I could be wrong, though.

msiemens · 2020-09-03T07:36:53Z

Funny! Since PyGitUp 2.0 I switched to PyTest for running the tests and there everything runs fine. But running with nosetests throws an exception as in your example. I'll publish a post-release with a fix

msiemens · 2020-09-03T07:47:09Z

This should be fixed now with v2.0.1.post1

mhjacks · 2020-09-03T14:10:49Z

Excellent! Thanks.

mhjacks closed this as completed Sep 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compatibility problem with Python 3.9? #102

Compatibility problem with Python 3.9? #102

mhjacks commented Aug 6, 2020

msiemens commented Aug 15, 2020

msiemens commented Aug 15, 2020

mhjacks commented Aug 15, 2020

msiemens commented Aug 15, 2020

mhjacks commented Aug 16, 2020

msiemens commented Aug 25, 2020

mhjacks commented Aug 25, 2020

mhjacks commented Aug 25, 2020 •

edited

msiemens commented Aug 26, 2020

mhjacks commented Aug 26, 2020 •

edited by msiemens

mhjacks commented Aug 30, 2020

msiemens commented Sep 2, 2020

mhjacks commented Sep 2, 2020 •

edited

msiemens commented Sep 3, 2020

msiemens commented Sep 3, 2020

mhjacks commented Sep 3, 2020

Compatibility problem with Python 3.9? #102

Compatibility problem with Python 3.9? #102

Comments

mhjacks commented Aug 6, 2020

msiemens commented Aug 15, 2020

msiemens commented Aug 15, 2020

mhjacks commented Aug 15, 2020

msiemens commented Aug 15, 2020

mhjacks commented Aug 16, 2020

msiemens commented Aug 25, 2020

mhjacks commented Aug 25, 2020

mhjacks commented Aug 25, 2020 • edited

msiemens commented Aug 26, 2020

mhjacks commented Aug 26, 2020 • edited by msiemens

mhjacks commented Aug 30, 2020

msiemens commented Sep 2, 2020

mhjacks commented Sep 2, 2020 • edited

msiemens commented Sep 3, 2020

msiemens commented Sep 3, 2020

mhjacks commented Sep 3, 2020

mhjacks commented Aug 25, 2020 •

edited

mhjacks commented Aug 26, 2020 •

edited by msiemens

mhjacks commented Sep 2, 2020 •

edited