asv run fails with cygwin git under Windows

[ankostis]

(reported initially on gh-331)

On Windows, with cygwin git therun command fails with path-errors like the following,
depepnding on the repo field:

• When "repo": "https://github.com/team/project.git":

  · Cloning project
  · Error running d:\cygwin\bin\git.exe clone --mirror https://github.com/team/project d:\projdir\cprojname.asv
          STDOUT -------->
  
          STDERR -------->
          Cloning into bare repository 'd:\projdir\cprojname.asv'...
          /cygdrive/d/projdir/d:\projdir\projname.asv: No such file or directory
  

• When "repo": ".":

  Cloning into 'd:\projdir\env\77f2c104c7e2c3fba9eaad6360cdf4e8\project'...
               fatal: '/cygdrive/d/projdir/d:\projdir' does not appear to be a git repository
               fatal: Could not read from remote repository.
  

[pv]

This may be possible to address when also Python is the cygwin Python.
If cygwin really is a complete POSIX environment, you can try replacing WIN = (os.name == 'nt') with WIN = (sys.platform == 'win32') in the code.

The mixed configuration with non-cygwin Python + cygwin git is probably not reasonable to support ---- unless perhaps there is some way to reliably detect that the git executable is the cygwin one.

[ankostis]

Have you considered GitPython?

On 18 October 2015 at 15:54, Pauli Virtanen notifications@github.com
wrote:

This may be possible to address when also Python is the cygwin Python.
If cygwin really is a complete POSIX environment, you can try replacing WIN
= (os.name == 'nt') with WIN = (sys.platform == 'win32') in the code.

The mixed configuration with non-cygwin Python + cygwin git is probably
not reasonable to support.

—
Reply to this email directly or view it on GitHub
#333 (comment).

[pv]

Do you know if it does the necessary path manipulations for cygwin? Based in a quick look, also it does not seem to have special cygwin handling, so it's not clear if using it helps here.

[ankostis]

I've created 4 Relative GitPython TCs & 4 Absolute TCs for evaluating what you have asked:
Various combination of Winpython-3.4/CygPython-2.7 against CygGit-2.5.3/GitBash-1.8.4.

• The "Relative" TCs pass under various combinations of python + git plus the 4 first absolute TCs.
• The failing combination is unfortunately the one i'm using: Cygwin-git + WinPython.
• I have also tried to see if this "interesting" property of GitPython would help: https://github.com/gitpython-developers/GitPython/blob/master/git/cmd.py#L258
  No :-(
• The results are better the current implementation in asv, so probably by excluding certain combinations it might be possible to achieve 100% success (i'm particular insterested in `WinPython-3.4 + Cygwin-git-2.5.3 + USE_SHELL=0/1`` case).
• The results are described in the TC's docstrings.
• Sorry for the complexity.

TCs:

"""
Script to test GitPython library on various git-implementations of Windows.

Setting up correctly the Python interpreter and git instance to run,
is the duty of the caller.
"""

import os
import pathlib
import shutil
import tempfile
import unittest
import time

import git
from git.util import RemoteProgress
import logging
import sys

logging.basicConfig(level=0)

#: Just some small repo.
git_remote_url = 'https://github.com/cloudera/repository-example/'

git.Git.USE_SHELL = False

infos = {
    'py-version': sys.version_info,
    'py-prefix': sys.prefix,
    'git-version': git.Git().version_info,
    'Git.USE_SHELL': git.Git.USE_SHELL,
}
print('\n'.join(str(i) for i in sorted(infos.items())))


def _add_demo_file_to_repo(repo):
    # Copied from tutorial.
    #
    new_file_path = os.path.join(repo.working_tree_dir, 'my-new-file')
    open(new_file_path, 'wb').close()
    repo.index.add([new_file_path])
    repo.index.commit("Added a new file in the past - for later merege")


class TRelative(unittest.TestCase):
    """
    Uses REALTIVE paths, so assumes a single parent dir for both source and
    destination repos.

    Results for GitPython-1.0.1 under *Windows*::

        Cygwin-python-2.7 + Cygwin-git-2.5.3 + USE_SHELL=0 + launched cmd.exe & cyg-bash:
                ALL OK (launched from cmd.exe & cyg-bash)
        Cygwin-python-2.7 + Cygwin-git-2.5.3 + USE_SHELL=1:
                ALL FAIL!! (Cannot find Shell when launched from cmd.exe & cyg-bash)

        Cygwin-python-2.7 + MsysGit-1.8.4 + USE_SHELL=0 + launched from cmd.exe & git-bash:
                - init, clone_local_from, clone_remote_url: OK
                - clone_local: FAIL (due to `/cygdrive/...` prefix on src-path)
        Cygwin-python-2.7 + MsysGit-1.8.4 + USE_SHELL=0 + launched from XYZ:
                ALL FAIL!! (Cannot find Shell when launched from cmd.exe & cyg-bash)

        WinPython-3.4 + Cygwin-git-2.5.3 + USE_SHELL=0/1
                - init, clone_local_from, clone_remote_url: OK
                - clone_local:  FAIL (due to `/cygdrive/...` prefix on src-path)

        WinPython-3.4 + MsysGit-1.8.4 + USE_SHELL=0/1:
                OK
    """

    def setUp(self):
        self.test_dpath = tempfile.mkdtemp()
        os.chdir(self.test_dpath)

    def tearDown(self):
        shutil.rmtree(self.test_dpath, ignore_errors=1)

    def _git_local_fpath(self):
        return 'r.git'

    def _git_clone_fpath(self):
        return 'r2.git'

    def test_init(self):
        local_rpath = self._git_local_fpath()

        r = git.Repo.init(str(local_rpath))
        try:
            _add_demo_file_to_repo(r)
        finally:
            r.config_writer().release()
        self.assertTrue(os.path.isdir(local_rpath))

    def test_clone_local(self):
        local_rpath = self._git_local_fpath()
        clone_rpath = os.path.abspath(self._git_clone_fpath()) ## NOTE: Absolute

        # Create local src-repo.
        #
        r = git.Repo.init(local_rpath)
        try:
            _add_demo_file_to_repo(r)
        finally:
            r.config_writer().release()
        self.assertTrue(os.path.isdir(local_rpath))

        # Clone src-repo.
        #
        r = git.Repo(local_rpath)
        try:
            rr = r.clone(clone_rpath)
            rr.config_writer().release()
        finally:
            r.config_writer().release()
        self.assertTrue(os.path.isdir(clone_rpath))

    def test_clone_local_from(self):
        local_rpath = self._git_local_fpath()
        clone_rpath = self._git_clone_fpath()

        # Create local src-repo.
        #
        r = git.Repo.init(local_rpath)
        try:
            _add_demo_file_to_repo(r)
        finally:
            r.config_writer().release()
        self.assertTrue(os.path.isdir(local_rpath))

        # Clone src-repo.
        #
        r = git.Repo.clone_from(local_rpath, clone_rpath)
        r.config_writer().release()
        self.assertTrue(os.path.isdir(clone_rpath))

    def test_clone_remote_url(self):
        clone_rpath = self._git_clone_fpath()
        r = git.Repo.clone_from(git_remote_url, clone_rpath)
        r.config_writer().release()
        self.assertTrue(os.path.isdir(clone_rpath))


class TAbsolute(unittest.TestCase):
    """
    Results under *Windows*::

        Cygwin-python-2.7 + Cygwin-Git-2.5.3 + USE_SHELL=0:
                OK
        Cygwin-python-2.7 + Cygwin-Git-2.5.3 + USE_SHELL=1:
                FAIL!!

        Cygwin-python-2.7 + MsysGit-1.8.4 + USE_SHELL=0 + launched from cmd.exe & git-bash:
                - clone_local:       FAIL (due to `/cygdrive/...` prefix on src-path)
                - clone_local_from:  FAIL (due to `/cygdrive/...` prefix on src-path)
                - clone_remot_url:   FAIL (due to `/cygdrive/...` prefix on src-path)
        Cygwin-python-2.7 + MsysGit-1.8.4 + USE_SHELL=0 + launched from XYZ:
                ALL FAIL!! (Cannot find Shell when launched from cmd.exe & cyg-bash)

        WinPython-3.4 + Cygwin-git-2.5.3 + USE_SHELL=0/1
                - init, clone_remote_url: OK
                - clone_local:       FAIL!! (due to `/cygdrive/...` prefix on src-path)
                - clone_local_from:  FAIL!! (due to `/cygdrive/...` prefix on src-path)

        WinPython-3.4 + MsysGit-1.8.4 + USE_SHELL=0/1:
                OK
    """

    def setUp(self):
        self.test_dir = pathlib.Path(tempfile.mkdtemp())

    def tearDown(self):
        shutil.rmtree(str(self.test_dir), ignore_errors=True)

    def _git_local_fpath(self):
        return self.test_dir.joinpath('r.git')

    def _git_clone_fpath(self):
        return self.test_dir.joinpath('r2.git')

    def test_init(self):
        local_rpath = self._git_local_fpath()
        try:
            r = git.Repo.init(str(local_rpath.absolute()))
            _add_demo_file_to_repo(r)
        finally:
            r.config_writer().release()

        self.assertTrue(local_rpath.is_dir())

    def test_clone_local(self):
        local_rpath = self._git_local_fpath()
        clone_rpath = self._git_clone_fpath()

        # Create local src-repo.
        #
        try:
            r = git.Repo.init(str(local_rpath.absolute()))
            _add_demo_file_to_repo(r)
        finally:
            r.config_writer().release()
        self.assertTrue(local_rpath.is_dir())

        # Clone src-repo.
        #
        r = git.Repo(str(local_rpath.absolute()))
        try:
            rr = r.clone(str(clone_rpath.absolute()))
            rr.config_writer().release()
        finally:
            r.config_writer().release()
        self.assertTrue(clone_rpath.is_dir())

    def test_clone_local_from(self):
        local_rpath = self._git_local_fpath()
        clone_rpath = self._git_clone_fpath()

        # Create local src-repo.
        #
        r = git.Repo.init(str(local_rpath.absolute()))
        try:
            _add_demo_file_to_repo(r)
        finally:
            r.config_writer().release()
        self.assertTrue(local_rpath.is_dir())

        # Clone src-repo.
        #
        r = git.Repo.clone_from(str(local_rpath.absolute()),
                str(clone_rpath.absolute()))
        r.config_writer().release()

        self.assertTrue(clone_rpath.is_dir())

    def test_clone_remote_url(self):
        clone_rpath = self._git_clone_fpath()
        r = git.Repo.clone_from(git_remote_url, str(clone_rpath.absolute()))
        r.config_writer().release()

        self.assertTrue(clone_rpath.is_dir())

[pv]

Ok, thanks for checking it out. The Cygwin-python/Cygwin-git and Mingw-python/Mingw-git combinations should be fairly easy to support with current asv code.

The mixed combinations are probably easier to support without gitpython --- asv requires only a few git commands, so dealing with that is probably simpler. One easy workaround could also be if the cygwin git understands the \\?\d:\ style win32 long path names. I'm not going to promise to look into this in the near future myself, but I'd guess patches are welcome.