Skip to content

less is suspended/backgrounded when using two pipes with grep #8699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
malthejorgensen opened this issue Feb 5, 2022 · 1 comment
Closed
Assignees
Labels
bug Something that's not working as intended
Milestone

Comments

@malthejorgensen
Copy link
Contributor

malthejorgensen commented Feb 5, 2022

I have a minor issue where less gets suspended/backgrounded unexpectedly when using two pipes and the last one is a grep (with some caveats, see below). It's easy to work around, so not a big priority but I wanted to raise it if other people stumble upon it.

How to reproduce the issue

# Make a file called "minimal_example.txt" that contains 50 empty lines
# (it has to be more lines than can fit in your terminal vertically)
> string repeat -n 50 \n > minimal_example.txt
> cat minimal_example.txt | grep '.*' | less
:fish: Job 1, 'cat minimal_example.txt | grep…' has stopped
# less is suspended/backgrounded immediately as the command is run

I can reliably reproduce the issue both using fish --no-config and sh -c 'env HOME=$(mktemp -d) fish'.

Interestingly, replacing cat minimal_example.txt with string repeat -n 50 \n or replacing grep '.*' with cat "fixes" the issue. Also the pipeline must have two pipes, e.g. grep '.*' minimal_example.txt | less works perfectly fine.
Furthermore, it seems related to terminal height. If I resize the terminal window to be tall enough for all the lines to fit the window the issue also does not occur.

The issue happens both when fish is run in Terminal.app and iTerm2. It does not occur in bash and zsh (not that fish should emulate any of those -- but this feels like bug).

In this asciinema video of the issue, the issue only happens on the second invocation, so it might be some kind race-condition. I currently hit the bug 95% of the time I run the test command, but it's been a bit lower previously.

fish version

> fish --version
fish, version 3.3.1
> echo $version
3.3.1
> echo $TERM
xterm-256color
> uname -a
Darwin Malthes-MBP.lan 21.3.0 Darwin Kernel Version 21.3.0: Wed Jan  5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000 arm64

I'm on a Macbook Pro (Apple M1 Pro) with macOS 12.2.

@ridiculousfish
Copy link
Member

ridiculousfish commented Feb 6, 2022

Great find and amazing repro steps! I am sure it was tedious to reduce this, thank you for taking the time.

This is definitely a bug in fish. I am pretty sure the sequence is:

  1. fish launches cat and less, and begins executing the grep function, all in parallel. These all live in a new pgroup whose leader is cat.
  2. fish knows it has to transfer ownership of the tty to the pgroup. This is an essential race; here the race winner is the /usr/bin/grep command within the grep function. So grep has transferred ownership!
  3. grep finishes and so fish reclaims the tty. This is the bug: fish should NOT reclaim the tty until the outermost job is finished.
  4. less tries to manipulate the tty or output something and so it gets SIGTTOU, stopping it.

The right fix here is to only call tcsetpgrp at "top level," when running an outermost job - never when running a nested job like a function. Unfortunately a proper fix will have to wait to post 3.4.0, but a workaround is to replace the fish function with a real command; for example this:

 cat minimal_example.txt | command grep '.*' | less

won't trigger it because there's no fish functions running.

@ridiculousfish ridiculousfish self-assigned this Feb 6, 2022
@ridiculousfish ridiculousfish added the bug Something that's not working as intended label Feb 6, 2022
@ridiculousfish ridiculousfish added this to the fish-future milestone Feb 6, 2022
ridiculousfish added a commit to ridiculousfish/fish-shell that referenced this issue Mar 19, 2022
This is a big cleanup to how tty transfer works. Recall that when job
control is active, we transfer the tty to jobs via tcsetpgrp().

Previously, transferring was done "as needed" in continue_job. That is, if
we are running a job, and the job wants the terminal and does not have it,
we will transfer the tty at that point.

This got pretty weird when running mixed pipelines. For example:

    cmd1 | func1 | cmd2

Here we would run `func1` before calling continue_job. Thus the tty
would be transferred by the nested function invocation, and also restored
by that invocation, potentially racing with tty manipulation from cmd1 or
cmd2.

In the new model, migrate the tty transfer responsibility outside of
continue_job. The caller of continue_job is then responsible for setting up
the tty. There's two places where this gets done:

1. In `exec_job`, where we run a job for the first time.

2. In `builtin_fg` where we continue a stopped job in the foreground.

Fixes fish-shell#8699
ridiculousfish added a commit to ridiculousfish/fish-shell that referenced this issue Mar 19, 2022
This is a big cleanup to how tty transfer works. Recall that when job
control is active, we transfer the tty to jobs via tcsetpgrp().

Previously, transferring was done "as needed" in continue_job. That is, if
we are running a job, and the job wants the terminal and does not have it,
we will transfer the tty at that point.

This got pretty weird when running mixed pipelines. For example:

    cmd1 | func1 | cmd2

Here we would run `func1` before calling continue_job. Thus the tty
would be transferred by the nested function invocation, and also restored
by that invocation, potentially racing with tty manipulation from cmd1 or
cmd2.

In the new model, migrate the tty transfer responsibility outside of
continue_job. The caller of continue_job is then responsible for setting up
the tty. There's two places where this gets done:

1. In `exec_job`, where we run a job for the first time.

2. In `builtin_fg` where we continue a stopped job in the foreground.

Fixes fish-shell#8699
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that's not working as intended
Projects
None yet
Development

No branches or pull requests

2 participants