Can't read pipe to function #206

Open
Tyilo opened this Issue Jul 5, 2012 · 11 comments

Comments

Projects
None yet
10 participants
@Tyilo
Contributor

Tyilo commented Jul 5, 2012

Test:

function testfun
    set input (cat)
    echo $input
end

echo testing123 | testfun

This should output "testing123", but produces nothing.

It works perfectly in bash:

function testfun
{
    input="$(cat)"
    echo $input
}

echo testing123 | testfun
@maxfl

This comment has been minimized.

Show comment
Hide comment
@maxfl

maxfl Jul 5, 2012

Contributor

You can use 'read' function as workaround.

function t
    while read -l line
        echo $line
    end
end
Contributor

maxfl commented Jul 5, 2012

You can use 'read' function as workaround.

function t
    while read -l line
        echo $line
    end
end
@maxfl

This comment has been minimized.

Show comment
Hide comment
@maxfl

maxfl Jul 6, 2012

Contributor

I just understood, that in fish it doesn't work, because the stdin is piped to 'set' instead of 'cat'.

Contributor

maxfl commented Jul 6, 2012

I just understood, that in fish it doesn't work, because the stdin is piped to 'set' instead of 'cat'.

@waterhouse

This comment has been minimized.

Show comment
Hide comment
@waterhouse

waterhouse Jan 18, 2014

Contributor

This issue goes deeper than just "set" working a certain way. Piping into a function at all is screwed up. And terminal I/O sent into a function does work, but is still a bit weird--it appears to buffer the input and deliver it all at once. Observe what happens with this function:

~> function meh
       cat
   end
~> # First, the way it's supposed to work.
~> # As input, we press the keys: a RET b RET control-D
~> cat
a
a
b
b
~> cat | cat
a
a
b
b
~> # Now...
~> meh
a
a
b
b
~> # So far so good, but...
~> cat | meh
a
b
^D
... um...
^D
control-D repeatedly does not work
try control-C
Job 1, “cat | meh” has stopped
~> fg
Send job 1, “cat | meh” to foreground
cat: stdin: Interrupted system call
~> jobs
jobs: There are no jobs
~> # Dear lord.
~> # For completeness...
~> meh | cat
a
b
aD
b
~> 

Also, cat | meh | cat behaves the same way, as does cat | begin; cat; end.
I can tell you further that the "cat" that complains about an interrupted system call in cat | meh is the first "cat". That is:

~> cp /bin/cat mycat
~> ./mycat | meh
Job 1, “./mycat | meh” has stopped  #after control-C
~> fg
Send job 1, “./mycat | meh” to foreground
mycat: stdin: Interrupted system call

So there's that. Obviously this is something to do with how fish calls functions and how it constructs pipes into them. Does anyone happen to know about this?

Contributor

waterhouse commented Jan 18, 2014

This issue goes deeper than just "set" working a certain way. Piping into a function at all is screwed up. And terminal I/O sent into a function does work, but is still a bit weird--it appears to buffer the input and deliver it all at once. Observe what happens with this function:

~> function meh
       cat
   end
~> # First, the way it's supposed to work.
~> # As input, we press the keys: a RET b RET control-D
~> cat
a
a
b
b
~> cat | cat
a
a
b
b
~> # Now...
~> meh
a
a
b
b
~> # So far so good, but...
~> cat | meh
a
b
^D
... um...
^D
control-D repeatedly does not work
try control-C
Job 1, “cat | meh” has stopped
~> fg
Send job 1, “cat | meh” to foreground
cat: stdin: Interrupted system call
~> jobs
jobs: There are no jobs
~> # Dear lord.
~> # For completeness...
~> meh | cat
a
b
aD
b
~> 

Also, cat | meh | cat behaves the same way, as does cat | begin; cat; end.
I can tell you further that the "cat" that complains about an interrupted system call in cat | meh is the first "cat". That is:

~> cp /bin/cat mycat
~> ./mycat | meh
Job 1, “./mycat | meh” has stopped  #after control-C
~> fg
Send job 1, “./mycat | meh” to foreground
mycat: stdin: Interrupted system call

So there's that. Obviously this is something to do with how fish calls functions and how it constructs pipes into them. Does anyone happen to know about this?

@waterhouse

This comment has been minimized.

Show comment
Hide comment
@waterhouse

waterhouse Jan 18, 2014

Contributor

Ok, I am finding that running
pbpaste | begin; cat; end
repeatedly in a fresh fish shell, with the clipboard being "23\n", will sometimes just print 23 back out, and will sometimes cause the shell to lock up, at which point control-C can do nothing. I assume this must be a race condition of some sort. Oh boy.

Contributor

waterhouse commented Jan 18, 2014

Ok, I am finding that running
pbpaste | begin; cat; end
repeatedly in a fresh fish shell, with the clipboard being "23\n", will sometimes just print 23 back out, and will sometimes cause the shell to lock up, at which point control-C can do nothing. I assume this must be a race condition of some sort. Oh boy.

@waterhouse

This comment has been minimized.

Show comment
Hide comment
@waterhouse

waterhouse Jan 18, 2014

Contributor

Meanwhile, it looks like the signal SIGTTIN is sent to the "mycat" in ./mycat | begin; cat; end:

     21    SIGTTIN      stop process         background read attempted from
                                             control terminal

Then, according to the GNU libc manual: "A process cannot read from the user's terminal while it is running as a background job. When any process in a background job tries to read from the terminal, all of the processes in the job are sent a SIGTTIN signal."

So, looks like the "mycat" either gets started in the background, or is started and then put in the background, when it gets piped into a fish function-kind-of-thing. Perhaps this knowledge will help.

Contributor

waterhouse commented Jan 18, 2014

Meanwhile, it looks like the signal SIGTTIN is sent to the "mycat" in ./mycat | begin; cat; end:

     21    SIGTTIN      stop process         background read attempted from
                                             control terminal

Then, according to the GNU libc manual: "A process cannot read from the user's terminal while it is running as a background job. When any process in a background job tries to read from the terminal, all of the processes in the job are sent a SIGTTIN signal."

So, looks like the "mycat" either gets started in the background, or is started and then put in the background, when it gets piped into a fish function-kind-of-thing. Perhaps this knowledge will help.

@smokku

This comment has been minimized.

Show comment
Hide comment
@smokku

smokku Apr 18, 2016

This backgrounds both sides of a pipe apparently... But giving fg command pulls the process from background allowing it to work as it supposed to.

~ $ alias pjson='python -m json.tool | pygmentize -l json'
~ $ curl -u smoku -X GET -H "Content-Type: application/json" 'https://jira.......' | pjson
Job 4, 'curl -u smoku -X GET…' has stopped
~ $ fg
Enter host password for user 'smoku': ********
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   593    0   593    0     0   1372      0 --:--:-- --:--:-- --:--:--  1375
~ $ fg
{
    "expand": "renderedFields,names,schema,transitions,operations,editmeta,changelog",
    "id": "29874"
}

A bit annoying that I needed to create pjson wrapper script in $PATH instead of simple alias... :(

smokku commented Apr 18, 2016

This backgrounds both sides of a pipe apparently... But giving fg command pulls the process from background allowing it to work as it supposed to.

~ $ alias pjson='python -m json.tool | pygmentize -l json'
~ $ curl -u smoku -X GET -H "Content-Type: application/json" 'https://jira.......' | pjson
Job 4, 'curl -u smoku -X GET…' has stopped
~ $ fg
Enter host password for user 'smoku': ********
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   593    0   593    0     0   1372      0 --:--:-- --:--:-- --:--:--  1375
~ $ fg
{
    "expand": "renderedFields,names,schema,transitions,operations,editmeta,changelog",
    "id": "29874"
}

A bit annoying that I needed to create pjson wrapper script in $PATH instead of simple alias... :(

@krader1961 krader1961 added the bug label Apr 18, 2016

@simotek

This comment has been minimized.

Show comment
Hide comment
@simotek

simotek Jun 3, 2016

For my reference this is also openSUSE bug https://bugzilla.opensuse.org/show_bug.cgi?id=963548

simotek commented Jun 3, 2016

For my reference this is also openSUSE bug https://bugzilla.opensuse.org/show_bug.cgi?id=963548

@milieu

This comment has been minimized.

Show comment
Hide comment
@milieu

milieu Oct 20, 2016

Yay! I think I found my workaround! Thanks to gustafj's comment in issue #110 explaining fish piping syntax, I have come up with this:

function line --argument-names n
    cat 1>| tail -n +$n | head -1
end

milieu commented Oct 20, 2016

Yay! I think I found my workaround! Thanks to gustafj's comment in issue #110 explaining fish piping syntax, I have come up with this:

function line --argument-names n
    cat 1>| tail -n +$n | head -1
end
@msoucy

This comment has been minimized.

Show comment
Hide comment
@msoucy

msoucy Nov 26, 2016

This issue, coupled with the default grep function for grep, results in quite a few problems - If this issue isn't going to be fixed any time soon, the default grep alias should probably be removed (or replaced with an abbreviation, maybe?) to at least minimize the occurances.

Using cat as @milieu suggests doesn't seem to fix the issue for me, on Fish 2.3.1 (which I just realized is slightly behind, but it's the version packaged for Fedora 25)

msoucy commented Nov 26, 2016

This issue, coupled with the default grep function for grep, results in quite a few problems - If this issue isn't going to be fixed any time soon, the default grep alias should probably be removed (or replaced with an abbreviation, maybe?) to at least minimize the occurances.

Using cat as @milieu suggests doesn't seem to fix the issue for me, on Fish 2.3.1 (which I just realized is slightly behind, but it's the version packaged for Fedora 25)

@layus

This comment has been minimized.

Show comment
Hide comment
@layus

layus Mar 30, 2017

It seems that there is a difference between execution within the shell and from the command line:

(zsh)$ ./fish -c "read n | grep nothing"    
read> lol
(zsh)$ ./fish
(fish)$ read n | grep nothing
read> 
# Stuck forever, needs to kill the terminal. ^C, ^Z have no impact.

Maybe this can help debug the issue ?

layus commented Mar 30, 2017

It seems that there is a difference between execution within the shell and from the command line:

(zsh)$ ./fish -c "read n | grep nothing"    
read> lol
(zsh)$ ./fish
(fish)$ read n | grep nothing
read> 
# Stuck forever, needs to kill the terminal. ^C, ^Z have no impact.

Maybe this can help debug the issue ?

@faho

This comment has been minimized.

Show comment
Hide comment
@faho

faho Mar 31, 2017

Member

@layus: No, that's #3805, an issue where fish itself can't gain control of the terminal.

Member

faho commented Mar 31, 2017

@layus: No, that's #3805, an issue where fish itself can't gain control of the terminal.

mqudsi added a commit to mqudsi/fish-shell that referenced this issue Aug 21, 2017

Fixes job handling involving functions and terminal control
While the idiomatic fix to fish' myriad of job control issues would be
to parse all functions prior to beginning the job pipeline so that
everything in the command line can be executed in the context of a
single job, that would require a huge effort to rewrite the core job
flow in fish and does not make sense at this time.

Instead, this patch fixes #3952 and #206 (but notably not #4238) by
having jobs that are part of a single command pipeline, including those
that are functions executing external commands, use the same process
group. This prevents a (parent|child) from crashing with SIGTTIN or
hanging at SIGTTOU because it has a different process group than the
process currently in control of the terminal.

Additionally, since this fix involves removing the code that forces fish
to run in its own process group (which IMHO never made sense, as job
control is the job of the shell, not the process being launched), it
also fixes #3805 and works around BashOnWindows#1653.

@fish-shell fish-shell deleted a comment from c02y Aug 21, 2017

@fish-shell fish-shell deleted a comment from milieu Aug 21, 2017

mqudsi added a commit to mqudsi/fish-shell that referenced this issue Sep 8, 2017

Fixes job handling involving functions and terminal control
While the idiomatic fix to fish' myriad of job control issues would be
to parse all functions prior to beginning the job pipeline so that
everything in the command line can be executed in the context of a
single job, that would require a huge effort to rewrite the core job
flow in fish and does not make sense at this time.

Instead, this patch fixes #3952 and #206 (but notably not #4238) by
having jobs that are part of a single command pipeline, including those
that are functions executing external commands, use the same process
group. This prevents a (parent|child) from crashing with SIGTTIN or
hanging at SIGTTOU because it has a different process group than the
process currently in control of the terminal.

Additionally, since this fix involves removing the code that forces fish
to run in its own process group (which IMHO never made sense, as job
control is the job of the shell, not the process being launched), it
also fixes #3805 and works around BashOnWindows#1653.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment