Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ModuleNotFoundError: No module named 'gitdb.utils.compat' #983

Closed
mell0 opened this issue Feb 17, 2020 · 20 comments
Closed

ModuleNotFoundError: No module named 'gitdb.utils.compat' #983

mell0 opened this issue Feb 17, 2020 · 20 comments

Comments

@mell0
Copy link

mell0 commented Feb 17, 2020

Requirments file includes gitdb2 and, but "compat.py" file is importing "gitdb" instead and throwing missing "gitdb" module error when this library is fresh installed in a new enviroment with missing "gitdb" dependency library. Please fix the problem.

@davidwneary
Copy link

Experiencing the same here.

from gitdb.utils.compat import (
 ModuleNotFoundError: No module named 'gitdb.utils.compat'

We're using the 3.0.5 release. Looks like the latest release to gitdb (3.0.2) has caused the issue - if I add gitdb2==3.0.1 to my requirements.txt, it's fixed.

@Harmon758
Copy link
Member

Harmon758 commented Feb 17, 2020

@davidwneary Ah, that issue is due to GitDB v3.0.2 removing gitdb.utils.compat.
I'll push a fix momentarily.
That import was removed with GitPython v3.0.6 (b5dd2f0, a10ceef, 5d22d57) so the latest versions shouldn't have that issue.

@mell0 gitdb2 is the same gitdb library renamed on PyPI due to loss of access to the original PyPI account. See gitpython-developers/gitdb#45, gitpython-developers/gitdb#47, and pypi/support#204. The import remains gitdb.

@Harmon758
Copy link
Member

@davidwneary Actually, that import was removed with v3.0.6. This should be resolved in the latest versions.

@Harmon758
Copy link
Member

@mell0 I'm unable to reproduce this in a new environment with the latest versions of both libraries, i.e. gitpython v3.0.8 and gitdb2 v3.0.2. If your issue is different, can you provide the details of your environment, e.g. gitpython, gitdb2, and python versions, and the full traceback?

@mell0
Copy link
Author

mell0 commented Feb 17, 2020

Hi,

Problem is when trying to install an older version for GitPython which I had in my test requirements file, to be specific v3.0.4. I understood that this was fixed in versions above v3.0.6. It's just that v3.0.4 should have used in its requirements working gitdb2 version with compat included, because now I have a lot of deployment fails of my apps and I'm sure it's not only me. In my opinion requirements should contain more specific version of the dependencies not just >=, because thus way people using older version and relying on it are suffering and in general at the moment older versions <=v3.0.6 are not working.

@mell0
Copy link
Author

mell0 commented Feb 17, 2020

To reproduce, just create new virtual env, try to install GitPython==3.0.4, and try to import it.

@Harmon758
Copy link
Member

I agree that the versioning is less than ideal here. In fact, GitPython shouldn't have been using GitDB compatibility shims in the first place, causing the removal to be a breaking change, as they should have been internal rather than exposed.

I've gone ahead and restricted the gitdb2 version to < 4 now with e81fd54 to mitigate future issues like this, and it should be part of the next release.

However, there's not much that can be done at this point for older versions that rely on gitdb2 < 3.0.2. Even if the GitDB removal was reverted, since it wasn't restricted to < or <= any certain version for GitPython, any breaking change in the future would still break older GitPython versions.
But since the only GitPython version requirement for gitdb2 was >= 2 for older versions, you can simply specify gitdb2 < 3.0.2 in your own dependencies if you want to continue using an older version of GitPython.

In fact, there was an issue with the formatting of the version specifier that wasn't fixed until GitPython v3.0.6, so you should be able to specify and install any gitdb2 version (although I can't guarantee that it'll work with < 2). It's just that, by default, the latest version will be installed.

@fmigneault
Copy link

@Harmon758
You could do a post release (https://www.python.org/dev/peps/pep-0440/#post-releases).
An hotfix for such a thing would avoid many unexpected problems by devs that tagged this version in order to have stable code, but suddenly still get an error.

I would also argue that using the patch number instead of (at least) minor for such big refactoring isn't really a best practice to avoid situations like this in the future.

@mitar
Copy link

mitar commented Feb 18, 2020

When can we expect this next release?

@Harmon758
Copy link
Member

I don't think creating a post-release for this is a good idea. As PEP 440 says, it's meant for issues that don't affect the actual software, like changes in release notes, and shouldn't be used for actual bug fixes. In addition, to cover every old version affected by the issue, we'd have to do 9 post-releases for 2.1.12 - 2.1.14 and 3.0.0 - 3.0.5.

I agree that some of the recent releases should have been minor version bumps rather than patch version bumps, but I've only very recently become a maintainer and Byron has been doing the releases. We're still working on our development and release process, but we have an open line of communication now that should help.

Apologies for any hassles this issue might have caused, but the root cause of the issue is that gitdb2 was not version locked. Having a minor version bump instead of a patch version bump wouldn't have affected the issue at all. In fact, any breaking change in GitDB, even with a major version bump, would have had the potential of breaking old GitPython versions.

Future issues like this should be mitigated now with gitdb2 version-locked to < 4. If an unintentional breaking change in a patch version bump were to occur again, we could simply revert the change in another patch version bump and release it with a major version bump.

As for resolving the current issue, if you're using an older version of GitPython, you can simply upgrade to the latest version of either major version, i.e. 3.0.8 or 2.1.15. Upgrading to 3.0.6 or 3.0.7 should resolve the issue as well. If you need to use an older version and can't upgrade, you'll have to add a requirement for gitdb2 < 3.0.2.

@Byron
Copy link
Member

Byron commented Feb 18, 2020

And should it become necessary, we might even be able to fight fire with fire, and overwrite a previous release of gitdb.
Personally I would prefer the affected folks to upgrade, but arguments can be made for the alternative above.

klml added a commit to klml/drfly that referenced this issue Apr 8, 2020
@Byron Byron unpinned this issue Apr 11, 2020
danielvdende added a commit to danielvdende/takeoff that referenced this issue May 4, 2020
The gitpython version that was in use has a problem (see issue here:
gitpython-developers/GitPython#983). As
mentioned in the thread, this can be resolved by pinning the version to
3.1.x
hannes-ucsc added a commit to DataBiosphere/azul that referenced this issue Sep 10, 2020
gitpython-developers/GitPython#983

======================================================================
ERROR: test_doctests (unittest.loader._FailedTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/azul/azul/test/test_doctests.py", line 50, in load_tests
    tests.addTests(doctest.DocTestSuite(load_module(root + '/scripts/check_branch.py', 'check_branch')))
  File "/builds/azul/azul/src/azul/modules.py", line 19, in load_module
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/builds/azul/azul/scripts/check_branch.py", line 4, in <module>
    import git
  File "/build/.venv/lib/python3.6/site-packages/git/__init__.py", line 38, in <module>
    from git.exc import *                       # @nomove @IgnorePep8
  File "/build/.venv/lib/python3.6/site-packages/git/exc.py", line 9, in <module>
    from git.compat import UnicodeMixin, safe_decode, string_types
  File "/build/.venv/lib/python3.6/site-packages/git/compat.py", line 16, in <module>
    from gitdb.utils.compat import (
ModuleNotFoundError: No module named 'gitdb.utils.compat'
----------------------------------------------------------------------
alangoldman added a commit to alangoldman/git-get-merge that referenced this issue Sep 23, 2020
paterit added a commit to paterit/django-microservice-template that referenced this issue Oct 18, 2020
@mmaithani
Copy link

Experiencing the same here.

from gitdb.utils.compat import (
 ModuleNotFoundError: No module named 'gitdb.utils.compat'

We're using the 3.0.5 release. Looks like the latest release to gitdb (3.0.2) has caused the issue - if I add gitdb2==3.0.1 to my requirements.txt, it's fixed.

yeah 3.0.1 worked for me too, Thanks !

sergeyklay added a commit to sergeyklay/gstore that referenced this issue Dec 24, 2020
jose added a commit to jose/schwa that referenced this issue Feb 17, 2021
lvarin added a commit to CSCfi/docs-preview that referenced this issue Mar 1, 2021
kevinlul added a commit to ccmbioinfo/FORCAST that referenced this issue Dec 15, 2021
Only used to write commit hashes to MongoDB (but most are blank or ab62755)
Current version if reinstalled has bug gitpython-developers/GitPython#983 without upgrading
danielmcmir added a commit to danielmcmir/SNOW-File-Comparator that referenced this issue Jun 9, 2022
frewmack added a commit to ccmbioinfo/FORCAST that referenced this issue Aug 3, 2022
commit 63001da
Author: NourHanafi <nour.hanafi@mail.utoronto.ca>
Date:   Fri Jul 22 18:33:00 2022 +0000

    updated .gitignore

commit d1070fc
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Thu Jun 23 13:24:44 2022 -0400

    Delete sample.env

commit 74c0ddd
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Jun 22 16:51:30 2022 -0400

    Fix README formatting

commit d2c78b5
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Jun 22 13:53:24 2022 -0400

    Update README for current install setup

commit 8189add
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Jun 22 13:22:23 2022 -0400

    yeet old setup scripts

commit f57f5af
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Jun 13 14:00:27 2022 -0400

    Add favicon to JBrowse

commit 4c44647
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Jun 13 13:50:36 2022 -0400

    Remove duplicate icon

commit a0d9a0d
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Jun 13 13:31:08 2022 -0400

    Bump JBrowse patch version

commit 00a218c
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Jun 6 13:22:01 2022 -0400

    Fix unnecessary +x outside of src/

commit 3d02099
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Jun 3 17:52:35 2022 -0400

    Disable Apache directory listing

commit 7418d6d
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Jun 3 17:23:01 2022 -0400

    Replace JavaScript / redirect with proper server redirect

commit e85c179
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Jun 3 16:45:59 2022 -0400

    Store JBrowse genome dropdown on bind mount

commit 3ebed57
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Jun 3 11:52:12 2022 -0400

    Directly download UCSC Genome Browser kent binaries to /usr/local/bin

commit d7c95d8
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Jun 3 11:18:44 2022 -0400

    Docker: mount tmpfs /run, /tmp

commit 68e2b9d
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Jun 3 11:17:43 2022 -0400

    Fix Dicey module import paths as a result of Python 3 migration

commit f6f0ab2
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Jun 3 11:17:02 2022 -0400

    Fix missing names and permissions in new install process

commit 358a088
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Thu Jun 2 17:53:56 2022 -0400

    Fix not-updated Jinja error template path

commit a30b411
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Thu Jun 2 17:53:31 2022 -0400

    Fix some cursed regex buried in JSON string to find assembly name

commit 6999d56
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Thu Jun 2 17:52:27 2022 -0400

    Remove unused Python dependencies

commit e05fd48
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Thu Jun 2 17:51:40 2022 -0400

    Install Dicey to /usr/local/bin now that temporary directory is not needed

commit 3ffe093
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Thu Jun 2 17:14:10 2022 -0400

    Fix Dicey, using Python 3, NamedTemporaryFile, new index paths

commit 940bd59
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Jun 1 17:49:37 2022 -0400

    Fix Dicey index

commit de47a89
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Jun 1 17:17:09 2022 -0400

    Slight cleanup create_segments

commit 687e234
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Jun 1 17:10:20 2022 -0400

    Update paths for processed assemblies

commit 950ee2a
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Jun 1 14:46:25 2022 -0400

    New setup script that verifies assembly name, builds Dicey index, removes temporaries

commit ed195e9
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Apr 6 15:21:27 2022 -0400

    Improve download script to use checksum file for finding files

    Does not work with GRCm38 regulatory build due to checksum file inconsistencies

commit 095e600
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Apr 6 14:59:18 2022 -0400

    Initial draft of new standalone download script

commit 9bbbad9
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Mar 21 18:03:00 2022 -0400

    Remove unused contents of bin/ that were also uncited

    Sources: UCSC, CRISPOR, possibly others
    - https://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/
    - https://github.com/ucscGenomeBrowser/kent
    - https://github.com/maximilianh/crisporWebsite/tree/master/bin
    - https://github.com/maximilianh/maxtools

commit 7e21763
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Mar 21 17:31:24 2022 -0400

    Remove licence until we get correct licensing details and attributions

commit 835263e
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Mar 21 17:29:00 2022 -0400

    Remove vendored JBrowse

commit 04f6477
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Mar 21 15:38:59 2022 -0400

    Cite background image

commit c3e711a
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Mar 21 15:30:43 2022 -0400

    Clean up landing page cruft code and fix titles

commit b24380f
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Mar 21 15:16:17 2022 -0400

    Cite the favicon

commit 888430e
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Mar 21 14:50:18 2022 -0400

    Switch JBrowse data directories to be under data/ and create Docker mount

    Increase Ensembl REST API timeout

commit 86ba295
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Mar 21 13:15:28 2022 -0400

    Fix up Dicey install script

commit 56e3a88
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Dec 15 15:50:40 2021 -0500

    Remove old install script

commit 1ca839a
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Dec 15 15:35:19 2021 -0500

    Fix Dicey installation

commit 99f1e22
Merge: aa1aed8 c7da06b
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Dec 15 11:41:42 2021 -0500

    Merge remote-tracking branch 'origin/master' into kevinlul/reproducibility

commit aa1aed8
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Dec 15 11:40:54 2021 -0500

    Add paths.conf (to be removed) for image builds

commit 49e57f7
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Dec 15 11:39:10 2021 -0500

    Remove dependency on outdated GitPython

    Only used to write commit hashes to MongoDB (but most are blank or ab62755)
    Current version if reinstalled has bug gitpython-developers/GitPython#983 without upgrading

commit e008ff0
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Dec 15 11:37:36 2021 -0500

    Fix newly added genomes not being added to JBrowse.conf correctly

commit 78171dc
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Dec 15 11:36:56 2021 -0500

    Write new Dockerfile, using apt and prebuilt binaries where possible

commit ba8bd76
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed Dec 15 10:55:55 2021 -0500

    Redownload some UCSC Genome Browser binaries to fix dynamic linking issues

    Source: https://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/
    Fixes missing libpng12.so problem that was hacked around

commit c7da06b
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Dec 3 18:47:33 2021 +0000

    Fix mysterious hardcoded genome in manual primer entry

    Parse querystring natively

commit 7505d24
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Dec 3 18:24:33 2021 +0000

    Fix outdated script and stylesheet references in manual primer entry

commit 7d728a2
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Nov 22 14:15:27 2021 -0500

    Ongoing work to restore reproducible builds

    Missing JBrowse, Primer3, BLAST, Dicey installs

commit 012edd8
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Fri Oct 8 18:23:00 2021 +0000

    Add helper scripts used for moving guides and primers between assemblies

commit 777dca7
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed May 26 17:51:39 2021 +0000

    Fix hardcoded gene assembly name in APEs

commit 7952bf7
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Wed May 26 17:46:10 2021 +0000

    Fix FetchAPE for changed directory structure, support all genomes

commit 6dc68d1
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Mon Apr 26 16:26:58 2021 -0400

    Replace contact links

commit 96d49d2
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Thu Mar 18 21:41:15 2021 +0000

    Add genome assembly name to splash dropdown

    Disambiguates between assemblies for the same species

commit dc25970
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Thu Mar 18 21:35:23 2021 +0000

    Fix bootstrap path for frontend, missing import in setup

commit f47a093
Author: Hillary Elrick <47190916+helrick@users.noreply.github.com>
Date:   Wed Mar 17 14:30:45 2021 +0000

    Permissions bugfix for guide design (#7)

    * ensuring permissions for web access are set

    * slightly more helpful error message

    (fixes issue with tmp.fa not being written)

commit 1180e97
Author: Hillary Elrick <47190916+helrick@users.noreply.github.com>
Date:   Tue Mar 16 21:31:18 2021 +0000

    Contents link typo + reorder README

commit 2f43f8b
Author: Kevin Lu <6320810+kevinlul@users.noreply.github.com>
Date:   Tue Mar 16 15:46:35 2021 -0400

    Update README.md
hannes-ucsc added a commit to DataBiosphere/azul that referenced this issue Nov 17, 2022
gitpython-developers/GitPython#983

======================================================================
ERROR: test_doctests (unittest.loader._FailedTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/azul/azul/test/test_doctests.py", line 50, in load_tests
    tests.addTests(doctest.DocTestSuite(load_module(root + '/scripts/check_branch.py', 'check_branch')))
  File "/builds/azul/azul/src/azul/modules.py", line 19, in load_module
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/builds/azul/azul/scripts/check_branch.py", line 4, in <module>
    import git
  File "/build/.venv/lib/python3.6/site-packages/git/__init__.py", line 38, in <module>
    from git.exc import *                       # @nomove @IgnorePep8
  File "/build/.venv/lib/python3.6/site-packages/git/exc.py", line 9, in <module>
    from git.compat import UnicodeMixin, safe_decode, string_types
  File "/build/.venv/lib/python3.6/site-packages/git/compat.py", line 16, in <module>
    from gitdb.utils.compat import (
ModuleNotFoundError: No module named 'gitdb.utils.compat'
----------------------------------------------------------------------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

9 participants