Slow (10s) startup when shell startup file flushes TTY

[krobelus]

Is there an existing issue for this?

• I have searched the existing issues

Midnight Commander version and build configuration

$ mc -V
GNU Midnight Commander 4.8.33-365-g05a6cf97e

Operating system

Linux

Is this issue reproducible using the latest version of Midnight Commander?

• I confirm the issue is still reproducible with the latest version of Midnight Commander

How to reproduce

Run this shell script

gcc -xc - <<'EOF'
#include <assert.h>
#include <termios.h>
#include <fcntl.h>
#include <unistd.h>

int
main (void)
{
    int ttyfd = STDOUT_FILENO;
    assert (isatty (ttyfd));
    int ftty = open ("/dev/tty", O_RDWR | O_NOCTTY | O_CLOEXEC);
    assert (ftty >= 0);
    static struct termios oldTerm;
    int ok;
    ok = tcgetattr (ftty, &oldTerm) != -1;
    assert (ok);
    ok = tcsetattr (ftty, TCSAFLUSH, &oldTerm) != -1;
    assert (ok);
}
EOF
echo '[ -n "$MC_SID" ] && '"$PWD"'/a.out' >> ~/.bashrc
SHELL=$(which bash) mc

Expected behavior

mc should start quickly

Actual behavior

mc hangs for 10 seconds, apparently because the ./a.out process has flushed the TTY, so mc can't read the prompt.

Additional context

Originally reported in #4766
This happens on Garuda Linux which configures its shell (fish) to run the "fastfetch" program on startup (if interactive).
That program flushes the TTY like above in a function called ffDetectTerminalSize, called when they want to output an image to the TTY.
I'm not yet sure if this could be fixed in "fastfetch" rather than mc.
cc @egmontkob

[egmontkob]

I can't see a clean solution to this problem.

When the shell starts up, mc quickly types some characters that initialize the shell for mc's purposes. These run in the shell's interactive mode, after the shell executes its ini file(s).

We don't know when the shell finishes with its ini file(s) and is about to continue with the letters mc typed; we're just about to install our hacks that would allow us to know.

---

There's one hack I can think of.

Most of the modern shells use some powerful line editing. For that, whenever they're ready to read user input, they switch the tty line from cooked (icanon and others) to raw (-icanon and others) mode.

At startup, for a short time (couple of seconds at most), if such a shell is being used, mc could poll the tty line's settings maybe ~60 times per second to notice when the shell switches it from cooked to raw. mc would delay sending the init string until this happens.

Such busy loops are quite nasty, but if they're limited to a few hundred executions tops, with (in computer terms) large delays in between, then they don't impose any noticeable system load increase. Also, a ~60Hz poll would only delay mc's startup by at most one display frame on typical monitors. This, while certainly hackish, could be a good compromise.

---

In the mean time, I'm tempted to say that no sane terminal-based app should flush their output, and it really should be removed from there – unless they have a strong reason to have it there, which I can't foresee. We could check their git blame / changelogs to find out the reason, or if there's none then ask them to stop it. Even without mc, it's unkind to wipe the keypresses the user might have typed ahead.

No matter how hard we try, we'll never be able to work perfectly with every possible shell startup file. A line has to be drawn somewhere. (E.g. there's another ticket that a user sends out an escape sequence from the ini file and wants to read the response... or the ini file could run a bit of interactive code waiting for the user's input... all broken now.) If mc decides not to support such shell startup files then IMO it's a quite fair decision.

[egmontkob]

Inside mc, anything output from the subshell's initialization files is lost, doesn't appear on the screen (not even after hiding the panels). During the shell's startup, mc is deliberately in a mode where it swallows output from the child, rather than forwarding to the host terminal.

See #4253 for a related story and a bit of further explanation.

That is, running fastfetch / neofetch / any similar utility inside mc, even if they worked and didn't cause a lockup, would be pointless and a slight waste of resources.

A great workaround would be for the said distro to configure its shells to only execute fastfetch if $MC_SID is unset.

[krobelus]

I'm tempted to say that no sane terminal-based app should flush their
output, and it really should be removed from there – unless they have
a strong reason to have it there, which I can't foresee

They're doing it before and after querying the terminal.
So I guess if we concurrently have

• mc sends a few keystrokes to bash
• fastfetch run from bashrc

then when fastfetch is reading query responses,
it might also see some keystrokes.

I submitted fastfetch-cli/fastfetch#2009 as draft
which fixes the delay in practice, but I guess the ambiguity remains
(but the problem that key strokes interfere with querying exists even with TCSAFLUSH, so I don't think TCSAFLUSH helps)

We don't know when the shell finishes with its ini file(s) and is about to continue with the letters mc typed

with fish this should be possible with the --init-command flag, which runs after configuration files are sourced,
and before the shell starts its REPL
For example "fish --init-command '... initialize mc ...'".
Obviously this wouldn't work for other shells but it seems like a robust solution for mc+fish.
I'll give it a try.

[egmontkob]

Oh, so fastfetch, at least with some nondefault flags or config, sends query escape sequences to the terminal and waits for its response. I can see in its source that it can emit \e[6n, \e[>c, \e[18t and alike (although with its default settings it doesn't seem to emit any of these for me on Ubuntu).

This means that the similarity with #4253 is even stronger than I expected, and it is even more so not expected to work under mc (currently). Therefore the distro should really go with not executing this utility under mc, that looks like a much more straightforward approach to me than to patch fastfetch (and how knows about all the other similar neofetch / whateverfetch tools out there).

---

fish [...] --init-command flag, which runs after configuration files are sourced, and before the shell starts its REPL

I so much wish every shell had a feature like this!!! This would solve so many of our problems.

I briefly had the idea that we could launch as subshell something like bash --rcfile /usr/share/mc/etc/bashrc or equivalent; the said file first source'ing bash's init files, and then doing mc's business. Would be much cleaner than "autotyping" our init commands.

The problem with this is: It's practically impossible to know which startup file(s) the shell would source if we didn't specify our one. It might depend on shell version (or even which particular shell implementation is behind a certain name), shell config, distro patches (and no, distros won't take care to patch mc equivalently to how they patch their shells), environment variables, whatnot.

Anyway, let's keep this in mind as a possibility.

[zyv]

Yes, I believe that we even already have mechanisms of passing user initialization files to most shells in mc, but the annoying part as you say is that I think they mostly override default ones and are not run after those upon startup, so that might lead to completely weird and unexpected behavior :(

For zsh I was able to find a project that according to my understanding implements it by replicating shell configuration reading logic and just injects additional configuration there. I think in practice this might work quite reliably, since this stuff changes so rarely if at all, but actually it's quite a horrible solution that I'm totally unexcited about :(

https://github.com/romkatv/zshi/blob/master/zshi

That question is what is worse... trying to find a way to know when to start our initialization or doing something like that.

I wonder if there are any "reliable" and "easy" but unconventional approaches to that? Just as a sick idea, look for the moment when the shell starts doing something upon window changes?

[egmontkob]

I also came across zshi and didn't like this concept either, it's way too hacky for my taste. Could maybe work for zsh, but is definitely way too much trouble across all shells and all their variations out there.

I don't understand your window size idea... okay, mc could initially lie about the window size, set an incorrect one for the inner tty, and later fix it... but how would you spot the time the shell reacts to that?