unexpected error - b'worktreeConfig'

[ereli-sevenai]

Bug Report

Issue name

pull: unexpected error - b'worktreeConfig'

Description

When attempting to use the dvc pull command in a repository with the worktreeConfig extension enabled in Git, DVC throws an unexpected error. The error message received is "ERROR: unexpected error - b'worktreeConfig'". This issue persists even after attempting to set worktreeConfig to false via the Git config.

Reproduce

1. Initialize a Git repository with git init.
2. Enable worktreeConfig in the Git configuration with git config extensions.worktreeConfig false.
3. Initialize DVC within the repository with dvc init.
4. Add a dataset or file to manage with DVC using dvc add file.
5. Try to pull data or synchronize with a remote storage using dvc pull.

Expected

The dvc pull command should execute without any errors, syncing the DVC-tracked files from the remote storage. The presence of the worktreeConfig setting in Git should not impact DVC operations.

[shcheklein]

I can reproduce this.

Full stack trace:

Traceback (most recent call last):
  File "/Users/ivan/Projects/dvc/dvc/cli/__init__.py", line 210, in main
    cmd = args.func(args)
  File "/Users/ivan/Projects/dvc/dvc/cli/command.py", line 21, in __init__
    self.repo: "Repo" = Repo(uninitialized=self.UNINITIALIZED)
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 218, in __init__
    self._ignore()
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 457, in _ignore
    self.scm_context.ignore(file)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/funcy/objects.py", line 25, in __get__
    res = instance.__dict__[self.fget.__name__] = self.fget(instance)
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 320, in scm_context
    return SCMContext(self.scm, self.config)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/funcy/objects.py", line 25, in __get__
    res = instance.__dict__[self.fget.__name__] = self.fget(instance)
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 308, in scm
    return SCM(self.root_dir, no_scm=no_scm)
  File "/Users/ivan/Projects/dvc/dvc/scm.py", line 111, in SCM
    return Git(root_dir, search_parent_directories=search_parent_directories)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/scmrepo/git/__init__.py", line 106, in __init__
    first_ = first(self.backends.values())
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/funcy/seqs.py", line 63, in first
    return next(iter(seq), None)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/_collections_abc.py", line 871, in __iter__
    yield self._mapping[key]
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/scmrepo/git/__init__.py", line 63, in __getitem__
    initialized = backend(*self.args, **self.kwargs)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 191, in __init__
    self.repo = Repo.discover(start=root_dir)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/dulwich/repo.py", line 1263, in discover
    return cls(path)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/dulwich/repo.py", line 1195, in __init__
    raise UnsupportedExtension(extension)
dulwich.repo.UnsupportedExtension: b'worktreeConfig'

Traceback (most recent call last):
  File "/Users/ivan/Projects/dvc/dvc/cli/__init__.py", line 210, in main
    cmd = args.func(args)
  File "/Users/ivan/Projects/dvc/dvc/cli/command.py", line 21, in __init__
    self.repo: "Repo" = Repo(uninitialized=self.UNINITIALIZED)
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 218, in __init__
    self._ignore()
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 457, in _ignore
    self.scm_context.ignore(file)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/funcy/objects.py", line 25, in __get__
    res = instance.__dict__[self.fget.__name__] = self.fget(instance)
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 320, in scm_context
    return SCMContext(self.scm, self.config)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/funcy/objects.py", line 25, in __get__
    res = instance.__dict__[self.fget.__name__] = self.fget(instance)
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 308, in scm
    return SCM(self.root_dir, no_scm=no_scm)
  File "/Users/ivan/Projects/dvc/dvc/scm.py", line 111, in SCM
    return Git(root_dir, search_parent_directories=search_parent_directories)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/scmrepo/git/__init__.py", line 106, in __init__
    first_ = first(self.backends.values())
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/funcy/seqs.py", line 63, in first
    return next(iter(seq), None)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/_collections_abc.py", line 871, in __iter__
    yield self._mapping[key]
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/scmrepo/git/__init__.py", line 63, in __getitem__
    initialized = backend(*self.args, **self.kwargs)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 191, in __init__
    self.repo = Repo.discover(start=root_dir)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/dulwich/repo.py", line 1263, in discover
    return cls(path)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/dulwich/repo.py", line 1195, in __init__
    raise UnsupportedExtension(extension)
dulwich.repo.UnsupportedExtension: b'worktreeConfig'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ivan/Projects/dvc/.venv/bin/dvc", line 8, in <module>
    sys.exit(main())
  File "/Users/ivan/Projects/dvc/dvc/cli/__init__.py", line 236, in main
    ret = _log_exceptions(exc) or 255
  File "/Users/ivan/Projects/dvc/dvc/cli/__init__.py", line 147, in _log_exceptions
    _log_unknown_exceptions()
  File "/Users/ivan/Projects/dvc/dvc/cli/__init__.py", line 49, in _log_unknown_exceptions
    logger.debug("Version info for developers:\n%s", get_dvc_info())
  File "/Users/ivan/Projects/dvc/dvc/info.py", line 38, in get_dvc_info
    with Repo() as repo:
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 218, in __init__
    self._ignore()
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 457, in _ignore
    self.scm_context.ignore(file)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/funcy/objects.py", line 25, in __get__
    res = instance.__dict__[self.fget.__name__] = self.fget(instance)
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 320, in scm_context
    return SCMContext(self.scm, self.config)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/funcy/objects.py", line 25, in __get__
    res = instance.__dict__[self.fget.__name__] = self.fget(instance)
  File "/Users/ivan/Projects/dvc/dvc/repo/__init__.py", line 308, in scm
    return SCM(self.root_dir, no_scm=no_scm)
  File "/Users/ivan/Projects/dvc/dvc/scm.py", line 111, in SCM
    return Git(root_dir, search_parent_directories=search_parent_directories)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/scmrepo/git/__init__.py", line 106, in __init__
    first_ = first(self.backends.values())
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/funcy/seqs.py", line 63, in first
    return next(iter(seq), None)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/_collections_abc.py", line 871, in __iter__
    yield self._mapping[key]
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/scmrepo/git/__init__.py", line 63, in __getitem__
    initialized = backend(*self.args, **self.kwargs)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 191, in __init__
    self.repo = Repo.discover(start=root_dir)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/dulwich/repo.py", line 1263, in discover
    return cls(path)
  File "/Users/ivan/Projects/dvc/.venv/lib/python3.9/site-packages/dulwich/repo.py", line 1195, in __init__
    raise UnsupportedExtension(extension)
dulwich.repo.UnsupportedExtension: b'worktreeConfig'

Also, CI is failing with this. Probably related to some libraries update. Not sure, but seems a p0 to research at least.

[DimaAmega]

Yes, we also get an error in CI
https://github.com/evidentlyai/evidently/actions/runs/8790124101/job/24121478623
Corresponding workflow job

[shcheklein]

@DimaAmega yep, it seems tbh that Github has changed something with the way they setup git on CI. Again, sorry about that. We are looking into this.

[0x2b3bfa0]

• worktreeConfig capitalization problem jelmer/dulwich#1285

[shcheklein]

Temporary workaround on CI:

      - name: Temporary workaround for GitVersion
        shell: bash
        run: |
          git config --unset-all extensions.worktreeconfig
          # See https://github.com/GitTools/actions/issues/1115

(borrowed from one the linked PRs in the Dulwich issue)

[shcheklein]

This is fixed upstream. A new fixed version of scmrepo and dulwich were published. @skshetry let me know if we should in this case bump the scmrepo on the DVC end.