Skip to content

Override bash executable, defaulting to Git for Windows git bash over WSL bash #1791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
address first round of comments from PR #1791
  • Loading branch information
emanspeaks committed Jan 21, 2024
commit f065d1fba422a528a133719350e027f1241273df
10 changes: 5 additions & 5 deletions .github/workflows/pythonpackage.yml
Original file line number Diff line number Diff line change
@@ -35,11 +35,11 @@ jobs:
python-version: ${{ matrix.python-version }}
allow-prereleases: ${{ matrix.experimental }}

- name: Set up WSL (Windows)
if: startsWith(matrix.os, 'windows')
uses: Vampire/setup-wsl@v2.0.2
with:
distribution: Debian
# - name: Set up WSL (Windows)
# if: startsWith(matrix.os, 'windows')
# uses: Vampire/setup-wsl@v2.0.2
# with:
# distribution: Debian
Comment on lines +38 to +42
Copy link
Member

@EliahKagan EliahKagan Jan 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the improvement from this PR, we shouldn't need to install a WSL distribution on CI anymore, so this commented-out code can be removed altogether. It will still be in the history and can be brought back if needed in the future. (I'm guessing you may have been planning to do this anyway, but I figured I'd mention it where applicable.)


- name: Prepare this repo for tests
run: |
9 changes: 8 additions & 1 deletion git/__init__.py
Original file line number Diff line number Diff line change
@@ -120,7 +120,14 @@


def refresh(path: Optional[PathLike] = None) -> None:
"""Convenience method for setting the git executable path."""
"""
Convenience method for setting the git and bash executable paths.

Note that the default behavior of invoking commit hooks on Windows has
changed to not prefer WSL bash with the introduction of
`Git.GIT_PYTHON_BASH_EXECUTABLE`. See the `refresh_bash()` documentation
for details on the default values and search paths.
"""
global GIT_OK
GIT_OK = False

193 changes: 62 additions & 131 deletions git/cmd.py
Original file line number Diff line number Diff line change
@@ -11,13 +11,13 @@
import logging
import os
import signal
from subprocess import Popen, PIPE, DEVNULL, run, CalledProcessError
from subprocess import Popen, PIPE, DEVNULL
import subprocess
import threading
from textwrap import dedent
from pathlib import Path

from git.compat import defenc, force_bytes, safe_decode, is_win
from git.compat import defenc, force_bytes, safe_decode
from git.exc import (
CommandError,
GitCommandError,
@@ -364,24 +364,27 @@ def __setstate__(self, d: Dict[str, Any]) -> None:
_bash_exec_env_var = "GIT_PYTHON_BASH_EXECUTABLE"

bash_exec_name = "bash"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems not to be used. I'm not sure it's needed. If it is kept, then _get_default_bash_path should probably return it on non-Windows systems and possibly return f"{cla.bash_exec_name}.exe" as the fallback on Windows systems. Although I believe it was for symmetry with git_exec_name, the treatment of git and bash has become less similar in recent changes, so I'm not sure there's a need for this anymore.

"""Default bash command that should work on Linux, Windows, and other systems."""
"""Default bash command."""

GIT_PYTHON_BASH_EXECUTABLE = None
"""Provide the full path to the bash executable. Otherwise it assumes bash is in the path.

Note that the bash executable is actually found during the refresh step in
the top level ``__init__``.
"""
Provides the path to the bash executable used for commit hooks. This is
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend adding "on Windows" to the first sentence, both here and in the refresh_bash docstring. It is set on other systems, and future software that uses GitPython may itself use it directly on other systems (though that would be unusual), but GitPython itself is only using it on Windows.

ordinarily set by `Git.refresh_bash()`. Note that the default behavior of
invoking commit hooks on Windows has changed to not prefer WSL bash with
the introduction of this variable. See the `Git.refresh_bash()`
documentation for details on the default values and search paths.
"""

@classmethod
def _get_default_bash_path(cls):
def _get_default_bash_path(cls) -> str:
# Assumes that, if user is running in Windows, they probably are using
# Git for Windows, which includes Git BASH and should be associated
# with the configured Git command set in `refresh()`. Regardless of
# if the Git command assumes it is installed in (root)/cmd/git.exe or
# (root)/bin/git.exe, the root is always up two levels from the git
# command. Try going up to levels from the currently configured
# git command, then navigate to (root)/bin/bash.exe. If this exists,
# with the configured Git command set in `refresh()`.
# Uses the output of `git --exec-path` for the currently configured
# Git command to find its `git-core` directory. If one assumes that
# the `git-core` directory is always three levels deeper than the
# root directory of the Git installation, we can try going up three
# levels and then navigating to (root)/bin/bash.exe. If this exists,
# prefer it over the WSL version in System32, direct access to which
# is reportedly deprecated. Fail back to default "bash.exe" if
# the Git for Windows lookup doesn't work.
@@ -392,145 +395,73 @@ def _get_default_bash_path(cls):
# independently of the Windows Git. A noteworthy example are repos
# with Git LFS, where Git LFS may be installed in Windows but not
# in WSL.
if not is_win:
if os.name != 'nt':
return "bash"
try:
wheregit = run(["where", Git.GIT_PYTHON_GIT_EXECUTABLE], check=True, stdout=PIPE).stdout
except CalledProcessError:
return "bash.exe"
gitpath = Path(wheregit.decode(defenc).splitlines()[0])
gitroot = gitpath.parent.parent
gitcore = Path(cls()._call_process("--exec-path"))
gitroot = gitcore.parent.parent.parent
gitbash = gitroot / "bin" / "bash.exe"
return str(gitbash) if gitbash.exists() else "bash.exe"

@classmethod
def refresh_bash(cls, path: Union[None, PathLike] = None) -> bool:
"""This gets called by the refresh function (see the top level __init__)."""
"""
Refreshes the cached path to the bash executable used for executing
commit hook scripts. This gets called by the top-level `refresh()`
Comment on lines +408 to +409
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above for GIT_PYTHON_BASH_EXECUTABLE, I suggest adding "on Windows" to the first sentence, since the path this sets is not used by GitPython on other systems. Otherwise, this is saying that hook scripts are executed with this bash interpreter on all systems, which is not accurate, and the documentation of the behavior on non-Windows systems lower down in this same docstring--which is itself not a problem--will reinforce that wrong information.

function on initial package import (see the top level __init__), but
this method may be invoked manually if the path changes after import.

This method only checks one path for a valid bash executable at a time,
using the first non-empty path provided in the following priority
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The path argument seems to be used if not None, even if it is the empty string. Although people shouldn't do that, the behavior seems correct, since passing a string as path should cause that to be used.

This description could be changed to just say "the first path provided..." because it is reasonable to interpret an empty environment variable as not providing a path, which is what I would suggest. But if you feel that's not accurate enough, then it could be expanded to be more specific, or something about the environment variable's value being nonempty could be added in item 2 of the numbered list.

order:

1. the explicit `path` argument to this method
2. the environment variable `GIT_PYTHON_BASH_EXECUTABLE` if it is set
and available via `os.environ` upon calling this method
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this line about os.environ is not needed. But this is subjective and if you feel it is better to keep this then I think that is okay.

It's possible to set environment variables in a way that does not make them available in os.environ, such as by calling os.putenv directly, but this is rare and the limitations of doing so are documented. Environment variables set that way are not expected to be visible to Python code in the same process. Another possible reason to mention os.environ is for the distinction between it and os.environb, though other places where a path is obtained from os.environ don't mention this. (If the distinction between Unicode and bytes is what makes this significant, then that should probably be stated more directly.)

3. if the current platform is not Windows, the simple string `"bash"`
4. if the current platform is Windows, inferred from the current
provided Git executable assuming it is part of a Git for Windows
distribution.
Comment on lines +420 to +423
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This omits the final fallback of bash.exe on Windows.


The current platform is checked based on the call `os.name`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is either an implementation detail, or an attempt to state something about the behavior that should be stated directly instead, and that it could be removed. But this is another subjective thing that I think you should feel free to leave as is, if you believe it is best to have it.

If any information related to the platform check is given, then I think the most relevant is that Cygwin isn't treated as Windows for this purpose. It is not treated as Windows for most purposes (in GitPython and also more broadly) so I don't think that has to be said, but the confusion that led to git.compat.is_win being deprecated was related to Cygwin.


This is a change to the default behavior from previous versions of
GitPython. In the event backwards compatibility is needed, the `path`
argument or the environment variable may be set to the string
`"bash.exe"`, which on most systems invokes the WSL bash by default.

This change to default behavior addresses issues where git hooks are
intended to run assuming the "native" Windows environment as seen by
git.exe rather than inside the git sandbox of WSL, which is likely
configured independently of the Windows Git. A noteworthy example are
repos with Git LFS, where Git LFS may be installed in Windows but not
in WSL.
"""
# Discern which path to refresh with.
if path is not None:
new_bash = os.path.expanduser(path)
new_bash = os.path.abspath(new_bash)
# new_bash = os.path.abspath(new_bash)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commented-out line can be removed. If it is valuable to note that this step is not being done, then a comment could be added stating the relevant behavior.

else:
new_bash = os.environ.get(cls._bash_exec_env_var)
if not new_bash:
new_bash = cls._get_default_bash_path()

# Keep track of the old and new bash executable path.
old_bash = cls.GIT_PYTHON_BASH_EXECUTABLE
# old_bash = cls.GIT_PYTHON_BASH_EXECUTABLE
cls.GIT_PYTHON_BASH_EXECUTABLE = new_bash

# Test if the new git executable path is valid. A GitCommandNotFound error is
# spawned by us. A PermissionError is spawned if the git executable cannot be
# executed for whatever reason.
has_bash = False
try:
run([cls.GIT_PYTHON_BASH_EXECUTABLE, "--version"], check=True, stdout=PIPE)
has_bash = True
except CalledProcessError:
pass

# Warn or raise exception if test failed.
if not has_bash:
err = dedent(
f"""\
Bad bash executable.
The bash executable must be specified in one of the following ways:
- be included in your $PATH
- be set via ${cls._bash_exec_env_var}
- explicitly set via git.refresh_bash()
"""
)

# Revert to whatever the old_bash was.
cls.GIT_PYTHON_BASH_EXECUTABLE = old_bash

if old_bash is None:
# On the first refresh (when GIT_PYTHON_GIT_EXECUTABLE is None) we only
# are quiet, warn, or error depending on the GIT_PYTHON_REFRESH value.

# Determine what the user wants to happen during the initial refresh we
# expect GIT_PYTHON_REFRESH to either be unset or be one of the
# following values:
#
# 0|q|quiet|s|silence|n|none
# 1|w|warn|warning
# 2|r|raise|e|error

mode = os.environ.get(cls._refresh_env_var, "raise").lower()

quiet = ["quiet", "q", "silence", "s", "none", "n", "0"]
warn = ["warn", "w", "warning", "1"]
error = ["error", "e", "raise", "r", "2"]

if mode in quiet:
pass
elif mode in warn or mode in error:
err = (
dedent(
"""\
%s
All commit hook commands will error until this is rectified.

This initial warning can be silenced or aggravated in the future by setting the
$%s environment variable. Use one of the following values:
- %s: for no warning or exception
- %s: for a printed warning
- %s: for a raised exception

Example:
export %s=%s
"""
)
% (
err,
cls._refresh_env_var,
"|".join(quiet),
"|".join(warn),
"|".join(error),
cls._refresh_env_var,
quiet[0],
)
)

if mode in warn:
print("WARNING: %s" % err)
else:
raise ImportError(err)
else:
err = (
dedent(
"""\
%s environment variable has been set but it has been set with an invalid value.

Use only the following values:
- %s: for no warning or exception
- %s: for a printed warning
- %s: for a raised exception
"""
)
% (
cls._refresh_env_var,
"|".join(quiet),
"|".join(warn),
"|".join(error),
)
)
raise ImportError(err)

# We get here if this was the init refresh and the refresh mode was not
# error. Go ahead and set the GIT_PYTHON_BASH_EXECUTABLE such that we
# discern the difference between a first import and a second import.
cls.GIT_PYTHON_BASH_EXECUTABLE = cls.bash_exec_name
else:
# After the first refresh (when GIT_PYTHON_BASH_EXECUTABLE is no longer
# None) we raise an exception.
raise GitCommandNotFound("bash", err)

# Test if the new git executable path exists.
has_bash = Path(cls.GIT_PYTHON_BASH_EXECUTABLE).exists()
return has_bash

@classmethod
def refresh(cls, path: Union[None, PathLike] = None) -> bool:
"""This gets called by the refresh function (see the top level __init__)."""
"""
This gets called by the refresh function (see the top level __init__).

Note that calling this method directly does not automatically update
the cached path to `bash`; either invoke the top level `refresh()`
function or call `Git.refresh_bash()` directly.
"""
# Discern which path to refresh with.
if path is not None:
new_git = os.path.expanduser(path)
Loading
Oops, something went wrong.