Deadlock on ssh_domain connect when stdout is printed from remote .bashrc

[tycho-kirchner]

What Operating System(s) are you seeing this problem on?

Linux X11

WezTerm version

20220408-101518-b908e2dd

Did you try the latest nightly build to see if the issue is better (or worse!) than your current version?

Yes, and I updated the version box above to show the version of the nightly that I tried

Describe the bug

Hi, and thanks for this gorgeous project!
I encountered the following issue: when connecting via ssh_domains and the remote bashrc prints something to stdout, a deadlock occurs.
The following is printed, before it is stuck:

Connecting to $HOST using SSH
Using libssh-rs to connect to$USER@$HOST:$PORT
SSH-2.0-OpenSSH_8.4
Running: wezterm cli proxy
Checking server version

When the echo line is removed from remote ~/.bashrc, everything is fine.

To Reproduce

On the remote host, put the following to the beginning of ~/.bashrc (bash must be ssh's login shell)

echo foobar

On localhost, add the respective ssh_domain to wezterm's config file (configure remote_hostname as usual in ~/.ssh/config)

ssh_domains = {
    {
      name = "remote-test",
      remote_address = "remote_hostname",
    },
}

Then on localhost execute wezterm connect remote-test, to see it deadlock.

Configuration

see above.

Expected Behavior

No deadlock (:

Logs

No response

Anything else?

No response

[wez]

stdout is reserved for the mux protocol when using an ssh_domain.

In general, you should avoid printing to stdout from your shell rcfiles as this sort of thing can impact a variety of different programs, not just wezterm!

If you must output stuff from your rcfiles, I would recommend making it conditional on being an interactive shell, and outputting to stderr instead of stdout.

[tycho-kirchner]

Thanks for the quick response. In fact usually I'm far from being dependent on printing stuff to stdout from my bashrc - that was just a relict from a recent debug session 😅. Maybe wezterm/multiplexing.html is a good place to document that? Btw. in my experience the following is usually better than checking for interactive flags case $- in *i*):

# Put to the beginning of the bashrc.
if [[ -n ${BASH_EXECUTION_STRING+x} && $BASH_EXECUTION_STRING == 'wezterm'* ]] ; then
    return 0
fi

[wez]

I've added some logic to try to detect this case and error out more quickly with a hint:

; wezterm connect somewhere
21:11:12.177  ERROR  wezterm_client::client > going to run wezterm cli proxy
21:11:12.271  ERROR  wezterm_client::client > Error while decoding response pdu: decoding a PDU: decode_raw: Corrupt Response
21:11:12.401  ERROR  wezterm_client::domain > detached domain 3
21:11:12.401  ERROR  mux                    > domain detached panes: []
21:11:12.428  ERROR  wezterm_gui            > Internal channel was closed prior to sending request. This may indicate that the remote host output invalid data to stdout prior to running the requested command; terminating
21:11:12.428  ERROR  mux::connui            > while running ConnectionUI loop: recv_timeout: channel is empty and disconnected

[tycho-kirchner]

Thanks @wez . Just tried it and indeed see above output. Personally, I'm fine with the
Corrupt Response hint, however, to be even more clear it may be desirable to print an ascii-representation of the exact corrupt response. Anyway, thanks so far 🙂.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.