Skip to content

Commit

Permalink
Added support of NixOS core tools in predict_threadable (#5440)
Browse files Browse the repository at this point in the history
### Motivation

Closes #5003

### The case

Core utils in Nix are symlinks to one binary file that contains all
utils:

```xsh
docker run --rm -it nixos/nix bash

which echo
# /nix/store/k6h0vjh342kqlkq69sxjj8i5y50l6jfr-coreutils-9.3/bin/echo

ls -la /nix/store/k6h0vjh342kqlkq69sxjj8i5y50l6jfr-coreutils-9.3/bin/
# b2sum -> coreutils
# base32 -> coreutils
# ...
# All tools are symlinks to one binary file - `coreutils`.
```

When
[`default_predictor_readbin`](https://github.com/xonsh/xonsh/blob/61bda708c992a300eab212f877718cf0050a5e3a/xonsh/commands_cache.py#L392)
read `coreutils` it catches `(b'isatty', b'tcgetattr', b'tcsetattr')`
and return `threadable=False` forever.

The list of Nix coreutils tools are exactly the same as in [brew
coreutils](https://formulae.brew.sh/formula/coreutils). So I can check
the real predicts on distinct binaries and see that only 2 tools among
106 are unthreadable and they already covered by
`default_threadable_predictors` (If it's wrong please add unthreadable
tools to the
[default_threadable_predictors](https://github.com/xonsh/xonsh/blob/61bda708c992a300eab212f877718cf0050a5e3a/xonsh/commands_cache.py#L518)):

```xsh
brew install coreutils

ls /opt/homebrew/opt/coreutils/libexec/gnubin/ | wc -l
# 106

for t in p`/opt/homebrew/opt/coreutils/libexec/gnubin/.*`:
    if not (th := __xonsh__.commands_cache.predict_threadable([t.name])):
        print($(which @(t.name)), th)
# /opt/homebrew/opt/coreutils/libexec/gnubin/cat False
# /opt/homebrew/opt/coreutils/libexec/gnubin/yes False

defaults = __import__('xonsh').commands_cache.default_threadable_predictors().keys()
defaults['cat']
# <function xonsh.commands_cache.predict_false>
defaults['yes']
# <function xonsh.commands_cache.predict_false>
```

So the rest of we need is to check the symlink and apply default
prediction if the tools is the symlink to coreutils. This implements
this PR. Test included.

## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**

---------

Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
3 people committed May 24, 2024
1 parent 61bda70 commit f582a33
Show file tree
Hide file tree
Showing 3 changed files with 78 additions and 0 deletions.
23 changes: 23 additions & 0 deletions news/fix_nix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
**Added:**

* Added support of NixOS core tools in ``predict_threadable``.

**Changed:**

* <news item>

**Deprecated:**

* <news item>

**Removed:**

* <news item>

**Fixed:**

* <news item>

**Security:**

* <news item>
28 changes: 28 additions & 0 deletions tests/test_commands_cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -202,3 +202,31 @@ def test_update_cache(xession, tmp_path):
cached = cache.update_cache()

assert file2.samefile(cached[basename][0])


@skip_if_on_windows
def test_nixos_coreutils(tmp_path):
"""On NixOS the core tools are the symlinks to one universal ``coreutils`` binary file."""
path = tmp_path / "core"
coreutils = path / "coreutils"
echo = path / "echo"
echo2 = path / "echo2"
echo3 = path / "echo3"
cat = path / "cat"

path.mkdir()
coreutils.write_bytes(b"Binary with isatty, tcgetattr, tcsetattr.")
echo.symlink_to(echo2)
echo2.symlink_to(echo3)
echo3.symlink_to(coreutils)
cat.symlink_to(coreutils)

for toolpath in [coreutils, echo, echo2, echo3, cat]:
# chmod a+x toolpath
current_permissions = toolpath.stat().st_mode
toolpath.chmod(current_permissions | 0o111)

cache = CommandsCache({"PATH": [path]})

assert cache.predict_threadable(["echo", "1"]) is True
assert cache.predict_threadable(["cat", "file"]) is False
27 changes: 27 additions & 0 deletions xonsh/commands_cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,27 @@ def all_commands(self):
self.update_cache()
return self._cmds_cache

def resolve_symlink(self, path):
visited = set()
current_path = path
while os.path.islink(current_path):
if current_path in visited:
# Detected a loop while resolving symlink
return None
visited.add(current_path)
try:
current_path = os.readlink(current_path)
except Exception:
return None
if not os.path.isabs(current_path):
current_path = os.path.join(os.path.dirname(path), current_path)
current_path = os.path.normpath(current_path)

if current_path == path:
return None

return current_path

def update_cache(self):
env = self.env
# iterate backwards so that entries at the front of PATH overwrite
Expand Down Expand Up @@ -383,6 +404,12 @@ def default_predictor_readbin(self, name, cmd0, timeout, failure):
return failure
if not os.path.isfile(fname):
return failure
if (link := self.resolve_symlink(fname)) and link.endswith("coreutils"):
"""
On NixOS the core tools are the symlinks to one universal ``coreutils`` binary file.
Detect it and use the default mode.
"""
return failure

try:
fd = os.open(fname, os.O_RDONLY | os.O_NONBLOCK)
Expand Down

0 comments on commit f582a33

Please sign in to comment.