Crash from subprocess.run() after print() in alias

[wlritchi]

xonfig

$ xonfig
+------------------+-------------------------------------+
| xonsh            | 0.14.1                              |
| Python           | 3.11.3                              |
| PLY              | 3.11                                |
| have readline    | True                                |
| prompt toolkit   | None                                |
| shell type       | readline                            |
| history backend  | sqlite                              |
| pygments         | None                                |
| on posix         | True                                |
| on linux         | True                                |
| distro           | unknown                             |
| on wsl           | False                               |
| on darwin        | False                               |
| on windows       | False                               |
| on cygwin        | False                               |
| on msys2         | False                               |
| is superuser     | False                               |
| default encoding | utf-8                               |
| xonsh encoding   | utf-8                               |
| encoding errors  | surrogateescape                     |
| xontrib          | []                                  |
| RC file 1        | /home/wlritchi/.wlrenv/xonshtest.py |
+------------------+-------------------------------------+

xonshrc

#!/usr/bin/env python3

import subprocess
from xonsh.built_ins import XSH

def _foo():
    print('bar')
    subprocess.run(['true'], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)

XSH.aliases['foo'] = _foo

Expected Behavior

xonsh should not crash when running aliases.

Current Behavior

Run any command followed by the foo alias. xonsh crashes:

/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/shell.py:248: UserWarning: 'prompt-toolkit' python package is not installed. Falling back to readline.
  shell_type = self.choose_shell_type(shell_type, env)
wlritchi@europa ~ @ true
<xonsh-code>:1:0 - true
<xonsh-code>:1:0 + ![true]
wlritchi@europa ~ @ foo
<xonsh-code>:1:0 - foo
<xonsh-code>:1:0 + ![foo]
Traceback (most recent call last):
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/base_shell.py", line 391, in default
    raise exc_info[1]
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/codecache.py", line 63, in run_compiled_code
    func(code, glb, loc)
  File "<stdin>", line 1, in <module>
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/built_ins.py", line 205, in subproc_captured_hiddenobject
    return xonsh.procs.specs.run_subproc(cmds, captured="hiddenobject", envs=envs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/procs/specs.py", line 901, in run_subproc
    return _run_specs(specs, cmds)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/procs/specs.py", line 936, in _run_specs
    command.end()
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/procs/pipelines.py", line 458, in end
    self._end(tee_output=tee_output)
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/procs/pipelines.py", line 466, in _end
    for _ in self.tee_stdout():
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/procs/pipelines.py", line 372, in tee_stdout
    sys.stdout.buffer.write(line)
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/base_shell.py", line 109, in write
    return self.membuf.write(b)
           ^^^^^^^^^^^^^^^^^^^^
ValueError: I/O operation on closed file.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/main.py", line 469, in main
    sys.exit(main_xonsh(args))
             ^^^^^^^^^^^^^^^^
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/main.py", line 513, in main_xonsh
    shell.shell.cmdloop()
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/readline_shell.py", line 613, in cmdloop
    self._cmdloop(intro=intro)
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/readline_shell.py", line 596, in _cmdloop
    stop = self.onecmd(line)
           ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/cmd.py", line 211, in onecmd
    return self.default(line)
           ^^^^^^^^^^^^^^^^^^
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/base_shell.py", line 402, in default
    print_exception(exc_info=exc_info)
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/tools.py", line 1059, in print_exception
    sys.stderr.write(
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/procs/proxies.py", line 182, in write
    r = h.write(s)
        ^^^^^^^^^^
  File "/home/wlritchi/.local/pipx/venvs/xonshtest/lib/python3.11/site-packages/xonsh/base_shell.py", line 182, in write
    self.mem.write(s)
ValueError: I/O operation on closed file.
Xonsh encountered an issue during launch
Failback to /bin/bash

Steps to Reproduce

Define an alias with a print statement followed by a call to platform.uname()._asdict(). Note that namedtuple._asdict() is a documented public method; there's no internal access here.

Define an alias with a print statement followed by a call to subprocess.run() with stderr redirected to subprocess.DEVNULL. Minimal RC file reproducing the issue is above.

Run at least one command and then run the alias. These are order-sensitive, and the crash only occurs if the alias is not the first command you run! Illustrative example:

true
true
foo # crash

foo
true
foo
foo
true
true
foo
# no crashes

Notes from earlier reproduction

I don't really know why they're order sensitive like this, nor why the issue only appears if a print statement occurs prior to the platform.uname()._asdict() call. I tried it with other namedtuples and with more aliases; as far as I can tell, the exact conditions are:

1. Start xonsh.
2. Run one or more commands or aliases, as long as none of the aliases call platform.uname()._asdict() (they're allowed to call platform.uname(), or call _asdict() on other namedtuples).
3. Run an alias that contains both a print() call and a platform.uname()._asdict() call, in that order, with any number of other operations in between.

Edit: One way to model the above is to imagine that a call to platform.uname()._asdict() can "prime" the system in a way that prevents the bug from occurring. However, in the original setting where I encountered the bug, it was repeated calls to the same alias that caused the bug, so there's some as-yet-unidentified step that can "un-prime" the system too.

For community

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment

[wlritchi]

Dug through the source of the platform lib and was able to minimize the repro a lot more - turns out platform.uname() calls subprocess.check_output(), but only lazily. It now makes a lot more sense how these things could be related. Edited the OP to reflect the new simpler case.

[wlritchi]

I did some more testing. This is a regression; the first affected release was 0.13.0. Bisecting the changes puts the blame on #4858. Not sure how exactly removing amalgamation caused this bug (or put another way, how amalgamation prevented it), but here we are.

Edit: and with $XONSH_NO_AMALGAMATE = True, the bug occurs at least as far back as 0.10.1. I'll test earlier versions once I figure out how to adapt my xonshrc, since the XSH builtin doesn't seem to exist in earlier versions.