How to redirect stderr but continue to pipe stdout? #10271
Labels
question
the issue author asks something
redirection-pipe
All related to redirection to files or more complex pipelines with STDERR
Milestone
Comments
|
I can only think of 3 things that I'd try.
I agree with you that @WindSoilder, any ideas about this? |
|
I'm not sure but a quick look through
workround can be use ❯ cat test.sh
#!/bin/bash
echo this is stderr message >> /dev/stderr
echo stdout message
echo stdout message line 2
❯ run-external ./test.sh --redirect-stdout --redirect-stderr | lines
╭───┬───────────────────────╮
│ 0 │ stdout message │
│ 1 │ stdout message line 2 │
╰───┴───────────────────────╯the following quick patch makes diff --git a/crates/nu-engine/src/eval.rs b/crates/nu-engine/src/eval.rs
index 462dbb347..cea130517 100644
--- a/crates/nu-engine/src/eval.rs
+++ b/crates/nu-engine/src/eval.rs
@@ -203,7 +203,6 @@ fn eval_external(
match redirect_target {
RedirectTarget::Piped(redirect_stdout, redirect_stderr) => {
- if redirect_stdout {
call.add_named((
Spanned {
item: "redirect-stdout".into(),
@@ -211,10 +210,7 @@ fn eval_external(
},
None,
None,
- ))
- }
-
- if redirect_stderr {
+ ));
call.add_named((
Spanned {
item: "redirect-stderr".into(),
@@ -223,7 +219,6 @@ fn eval_external(
None,
None,
))
- }
}
RedirectTarget::CombinedPipe => call.add_named((
Spanned {
@@ -746,10 +741,11 @@ pub fn eval_element_with_input(
PipelineData::ExternalStream { exit_code, .. } => exit_code.take(),
_ => None,
};
- let input = match (redirection, input) {
+ let (input_to_save, output_stdout, output_stderr) = match (redirection, input) {
(
Redirection::Stderr,
PipelineData::ExternalStream {
+ stdout,
stderr,
exit_code,
span,
@@ -757,15 +753,34 @@ pub fn eval_element_with_input(
trim_end_newline,
..
},
- ) => PipelineData::ExternalStream {
+ ) => (PipelineData::ExternalStream {
stdout: stderr,
stderr: None,
exit_code,
span,
metadata,
trim_end_newline,
- },
- (_, input) => input,
+ }, stdout, None),
+ (
+ Redirection::Stdout,
+ PipelineData::ExternalStream {
+ stdout,
+ stderr,
+ exit_code,
+ span,
+ metadata,
+ trim_end_newline,
+ ..
+ },
+ ) => (PipelineData::ExternalStream {
+ stdout: stdout,
+ stderr: None,
+ exit_code,
+ span,
+ metadata,
+ trim_end_newline,
+ }, None, stderr),
+ (_, input) => (input, None, None),
};
if let Some(save_command) = engine_state.find_decl(b"save", &[]) {
@@ -798,7 +813,7 @@ pub fn eval_element_with_input(
redirect_stderr: false,
parser_info: HashMap::new(),
},
- input,
+ input_to_save,
)
.map(|_| {
// save is internal command, normally it exists with non-ExternalStream
@@ -806,8 +821,8 @@ pub fn eval_element_with_input(
// So nu handles exit_code correctly
(
PipelineData::ExternalStream {
- stdout: None,
- stderr: None,
+ stdout: output_stdout,
+ stderr: output_stderr,
exit_code,
span: *span,
metadata: None, |
sholderbach
added
the
redirection-pipe
|
Sorry, I don't have much idea, I can try to look into this when I get time |
This was referenced Oct 27, 2023
WindSoilder
added a commit
that referenced
this issue
Nov 23, 2023
#10851) # Description Fixes: #10271 Given the following script: ```shell # test.sh echo aaaaa echo bbbbb 1>&2 echo cc ``` This pr makes the following command possible: ```nushell bash test.sh err> /dev/null | lines | each {|line| $line | str length} ``` ## General idea behind the change: When nushell redirect stderr message to external file 1. it take stdout of external stream, and pass this stream to next command, so it won't block next pipeline command from running. 2. relative stderr stream are handled by `save` command These two streams are handled separately, so we need to delegate a thread to `save` command, or else we'll have a chance to hang nushell, we have meet a similar before: #5625. ### One case to consider What if we're failed to save to an external stream? (Like we don't have a permission to save to a file)? In this case nushell will just print a waning message, and don't stop the following scripts from running. # User-Facing Changes ## Before ```nushell ❯ bash test2.sh err> /dev/null | lines | each {|line| $line | str length} aaaaa cc ``` ## After ```nushell ❯ bash test2.sh err> /dev/null | lines | each {|line| $line | str length} ╭───┬───╮ │ 0 │ 5 │ │ 1 │ 2 │ ╰───┴───╯ ``` BTY, after this pr, the following commands are impossible either, it's important to make sure that the implementation doesn't introduce too much costs: ```nushell ❯ echo a e> a.txt e> a.txt Error: × Can't make stderr redirection twice ╭─[entry #1:1:1] 1 │ echo a e> a.txt e> a.txt · ─┬ · ╰── try to remove one ╰──── ❯ echo a o> a.txt o> a.txt Error: × Can't make stdout redirection twice ╭─[entry #2:1:1] 1 │ echo a o> a.txt o> a.txt · ─┬ · ╰── try to remove one ╰──── ```
hardfau1t
pushed a commit
to hardfau1t/nushell
that referenced
this issue
Dec 14, 2023
nushell#10851) # Description Fixes: nushell#10271 Given the following script: ```shell # test.sh echo aaaaa echo bbbbb 1>&2 echo cc ``` This pr makes the following command possible: ```nushell bash test.sh err> /dev/null | lines | each {|line| $line | str length} ``` ## General idea behind the change: When nushell redirect stderr message to external file 1. it take stdout of external stream, and pass this stream to next command, so it won't block next pipeline command from running. 2. relative stderr stream are handled by `save` command These two streams are handled separately, so we need to delegate a thread to `save` command, or else we'll have a chance to hang nushell, we have meet a similar before: nushell#5625. ### One case to consider What if we're failed to save to an external stream? (Like we don't have a permission to save to a file)? In this case nushell will just print a waning message, and don't stop the following scripts from running. # User-Facing Changes ## Before ```nushell ❯ bash test2.sh err> /dev/null | lines | each {|line| $line | str length} aaaaa cc ``` ## After ```nushell ❯ bash test2.sh err> /dev/null | lines | each {|line| $line | str length} ╭───┬───╮ │ 0 │ 5 │ │ 1 │ 2 │ ╰───┴───╯ ``` BTY, after this pr, the following commands are impossible either, it's important to make sure that the implementation doesn't introduce too much costs: ```nushell ❯ echo a e> a.txt e> a.txt Error: × Can't make stderr redirection twice ╭─[entry #1:1:1] 1 │ echo a e> a.txt e> a.txt · ─┬ · ╰── try to remove one ╰──── ❯ echo a o> a.txt o> a.txt Error: × Can't make stdout redirection twice ╭─[entry nushell#2:1:1] 1 │ echo a o> a.txt o> a.txt · ─┬ · ╰── try to remove one ╰──── ```
dmatos2012
pushed a commit
to dmatos2012/nushell
that referenced
this issue
Feb 20, 2024
nushell#10851) # Description Fixes: nushell#10271 Given the following script: ```shell # test.sh echo aaaaa echo bbbbb 1>&2 echo cc ``` This pr makes the following command possible: ```nushell bash test.sh err> /dev/null | lines | each {|line| $line | str length} ``` ## General idea behind the change: When nushell redirect stderr message to external file 1. it take stdout of external stream, and pass this stream to next command, so it won't block next pipeline command from running. 2. relative stderr stream are handled by `save` command These two streams are handled separately, so we need to delegate a thread to `save` command, or else we'll have a chance to hang nushell, we have meet a similar before: nushell#5625. ### One case to consider What if we're failed to save to an external stream? (Like we don't have a permission to save to a file)? In this case nushell will just print a waning message, and don't stop the following scripts from running. # User-Facing Changes ## Before ```nushell ❯ bash test2.sh err> /dev/null | lines | each {|line| $line | str length} aaaaa cc ``` ## After ```nushell ❯ bash test2.sh err> /dev/null | lines | each {|line| $line | str length} ╭───┬───╮ │ 0 │ 5 │ │ 1 │ 2 │ ╰───┴───╯ ``` BTY, after this pr, the following commands are impossible either, it's important to make sure that the implementation doesn't introduce too much costs: ```nushell ❯ echo a e> a.txt e> a.txt Error: × Can't make stderr redirection twice ╭─[entry nushell#1:1:1] 1 │ echo a e> a.txt e> a.txt · ─┬ · ╰── try to remove one ╰──── ❯ echo a o> a.txt o> a.txt Error: × Can't make stdout redirection twice ╭─[entry nushell#2:1:1] 1 │ echo a o> a.txt o> a.txt · ─┬ · ╰── try to remove one ╰──── ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Question
I'm having trouble invoking an external command and redirecting stderr (to /dev/null) while piping stdout to the next step.
Maybe there's some other kind of redirection I should be using? Maybe it's a bug?
Sample:
Almost what I want, except for the ugly WARNING that didn't flow into the pipe.
(try to) Fix it by redirecting stderr:
Does suppress the WARNING, but now
filterisn't getting any input...At first I though this had something to do with
sudo, but you can repro with nonprivileged externaldircommandThe missing directory error is reported to stderr as in the previous case.
I don't quite get why each newline-delimited line of output wasn't broken into a separate line item, like it seems to have done with
sudo apt, but never mind: the key thing is filter returned a list with one element, as desired.Note that the dir output isn't being piped into
filter,at all, apparently.Additional context and details
No response
The text was updated successfully, but these errors were encountered: