Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to capture output of programs that use alternate mode #2404

Open
nh2 opened this issue Jun 12, 2017 · 10 comments

Comments

@nh2
Copy link
Contributor

commented Jun 12, 2017

In bash you can do

OUTPUT=$(fzf)

then select an entry in the fzf list (which runs in alternate mode), and afterwards you can

echo $OUTPUT

and will obtain what you selected.

In xonsh, this doesn't work:

  • OUTPUT = $(fzf) doesn't show the alternate mode (fzf is running but invisible)
  • OUTPUT = $[fzf] makes it visible but that doesn't capture the final stdout that fzf produces

How to do this?

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jun 16, 2017

This is actually somewhat a dealbreaker, because in bash you can do git show $(git branch | fzf) and it works out of the box as expected, but in xonsh it doesn't.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jun 16, 2017

Is it possible that this is just a bug?

The docs say:

The $() syntax captures and returns the standard output stream of the command as a Python string. This is similar to how $() performs in Bash. For example,

But in bash $() doesn't capture stderr, and it seems that in xonsh it does.

To reproduce:

$ echo $(bash -c '(>&2 echo "error")')

In bash this prints error, in xonsh this prints nothing. All-in-one repro:

$ bash -c "echo $(bash -c '(>&2 echo error)')"
error

$ xonsh -c "echo $(bash -c '(>&2 echo error)')"

Another indication that this might be a bug is that the function that implements $() is called subproc_captured_stdout() which doesn't sound at all like it should capture stderr.

@gforsyth

This comment has been minimized.

Copy link
Collaborator

commented Jun 17, 2017

Hey @nh2, thanks for reporting! Can you please paste the output of xonfig? fzf works on my machine, although the usage will vary slightly since xonsh is not bash.
so I can do

> $OUTPUT = $(git branch | fzf)
> $OUTPUT
'  master\n'

Not sure about those leading spaces, but the trailing newline is the expected behavior of xonsh grabbing output from something.

For your second comment, instead of doing

git checkout $(git branch | fzf)

you can do

git checkout @$(git branch | fzf)

where the @$ operator will automatically strip the spaces and trailing newline. (This is also how you would accomplish the bash-like vim $(which xonsh) -> vim @$(which xonsh)

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jun 18, 2017

$ xonfig
+------------------+----------------------+
| xonsh            | 0.5.12               |
| Git SHA          | e9129111             |
| Commit Date      | Jun 18 01:29:22 2017 |
| Python           | 3.5.2                |
| PLY              | 3.9                  |
| have readline    | True                 |
| prompt toolkit   | 1.0.14               |
| shell type       | prompt_toolkit       |
| pygments         | 2.2.0                |
| on posix         | True                 |
| on linux         | True                 |
| distro           | Ubuntu               |
| on darwin        | False                |
| on windows       | False                |
| on cygwin        | False                |
| is superuser     | False                |
| default encoding | utf-8                |
| xonsh encoding   | utf-8                |
| encoding errors  | surrogateescape      |
+------------------+----------------------+
$ $OUTPUT = $(git branch | fzf)
^C

@gforsyth Ah, it actually works for you? For me it just hangs, not showing any output, as in the paste above.

@nhoad

This comment has been minimized.

Copy link
Contributor

commented Jul 11, 2017

FWIW, I'm having some very similar issues.

Capturing the output of fzf actually breaks my terminal such that I can't highlight text, or interact with it via my mouse at all. If I run xonsh with readline instead of prompt_toolkit, then mouse interaction causes tty scruft to start getting printed to the screen. I'm not sure which specific changeset introduced this, but I've figured out that this started happening in 0.5.5. If I run fzf again without capturing it and ^C it, then my terminal returns to normal.

Also, capturing fzf output sometimes works, e.g. I'm selecting master in both invocations below:

$ xonsh --no-rc

Welcome to the xonsh shell (0.5.12)

~ Break out of your shell ~


xonfig tutorial    ->    Launch the tutorial in the browser
xonfig wizard      ->    Run the configuration wizard and claim your shell 
(Note: Run the Wizard or create a ~/.xonshrc file to suppress the welcome screen)

nhoad@nhoad-laptop ~/projects/xonsh master $ xonfig
+------------------+----------------------+
| xonsh            | 0.5.12               |
| Git SHA          | f6fb9ba5             |
| Commit Date      | Apr 18 04:41:11 2017 |
| Python           | 3.6.1                |
| PLY              | 3.9                  |
| have readline    | True                 |
| prompt toolkit   | 1.0.14               |
| shell type       | prompt_toolkit       |
| pygments         | 2.2.0                |
| on posix         | True                 |
| on linux         | True                 |
| distro           | arch                 |
| on darwin        | False                |
| on windows       | False                |
| on cygwin        | False                |
| is superuser     | False                |
| default encoding | utf-8                |
| xonsh encoding   | utf-8                |
| encoding errors  | surrogateescape      |
+------------------+----------------------+
nhoad@nhoad-laptop ~/projects/xonsh master $ $OUTPUT = $(git branch | fzf)
nhoad@nhoad-laptop ~/projects/xonsh master $ 

~/projects/xonsh master $ $OUTPUT = $(git branch | fzf)
~/projects/xonsh master $ $OUTPUT
''
~/projects/xonsh master $ $OUTPUT = $(git branch | fzf)
~/projects/xonsh master $ $OUTPUT
'* master\n'

I've straced fzf and verified that it is indeed writing '* master\n' to stdout in both situations, so I'm not sure what's going on. A race condition of some sort, maybe? I'm not sure how best to debug xonsh, so if there's anything I can do to help figure this out, let me know.

In case it's relevant, I've tried this with both readline and prompt_toolkit, same result. I've gone back a couple of releases and this has been present in all of them for me.

EDIT: If it's any consolation to people who want to be able to use something in the meantime, I've found that neither of my issues occur with https://github.com/garybernhardt/selecta or https://github.com/rschmitt/heatseeker, so I'm just going to migrate to using one of these. I'm still happy to help out with debugging the issues I've found though.

@nhoad

This comment has been minimized.

Copy link
Contributor

commented Jul 14, 2017

I've got two potential fixes for my issues, one of which might also fix your issues, @nh2. Would you be willing to try applying this patch, and see if it works for you?

index eef3d933..8e218a44 100644
--- a/xonsh/built_ins.py
+++ b/xonsh/built_ins.py
@@ -741,7 +741,7 @@ def _update_last_spec(last):
     elif ON_WINDOWS and not callable_alias:
         last.universal_newlines = True
         last.stderr = None  # must truly stream on windows
-    else:
+    elif captured != 'stdout':
         r, w = pty.openpty() if use_tty else os.pipe()
         _safe_pipe_properties(w, use_tty=use_tty)
         last.stderr = safe_open(w, 'w')

This changes things such that the final process in a spec won't have its stderr captured unless the capture mode indicates it should be, which should be closer in behaviour to what more traditional shells do, and matches the documentation for $() (it currently kind of matches the doc, but I won't go into it).

My other patch is a little more involved and is more of a workaround than a fix, so I'm more interested in this one first, as it takes xonsh in the right direction (IMO).

If one of the fantastic Xonsh devs could weigh in on whether or not this tiny patch is acceptable, that'd be good! I'm happy to write up some tests for it and submit a PR. I've found a good work around is to throw tee on the end, e.g. git branch | fzf | tee -, then fzf's stderr won't be captured and things will work fine (but of course, fixing the code would be preferable!).

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 14, 2017

@nathan-hoad I tried your patch, it does seem to bring an improvement but also breaks something else:

Before your patch:

$ @("ls ".strip())
CHANGELOG.rst     codecov.yml            requirements-docs.txt   xonsh.egg-info
CONTRIBUTING.rst  conftest.py            requirements-tests.txt  xonsh.sublime-project

after your patch:

$ @("ls ".strip())
CHANGELOG.rst     codecov.yml            requirements-docs.txt   xonsh.egg-info
                                                                               CONTRIBUTING.rst  conftest.py            requirements-tests.txt  xonsh.sublime-project
@nhoad

This comment has been minimized.

Copy link
Contributor

commented Jul 14, 2017

Huh, interesting! I don't experience that behaviour. Is ls an alias that adds colour options or anything? If you try running xonsh with --no-rc, does it still happen?

What OS + terminal are you using? If I can try to match your setup, I might be able to replicate it.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 18, 2017

@nathan-hoad

Is ls an alias that adds colour options or anything?

Yes, it seems so, if I do which ls it just prints ls --color=auto -v. It also does that when I run it from xonsh --no-rc.

What OS + terminal are you using? If I can try to match your setup, I might be able to replicate it.

Ubuntu 16.04 and terminator.

But I think it was just that the state of that terminal was broken. Retrying it now and it works. In urxvt it also works.

So I need to revise my answer, your patch seems to have have the desired result for me.

@scopatz

This comment has been minimized.

Copy link
Member

commented Jul 22, 2017

Any chance we could get a pull request, please? Thanks for bringing this up!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.