Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tmux session in weird semi-zombie state #298

Closed
wincent opened this issue Feb 8, 2016 · 46 comments
Closed

tmux session in weird semi-zombie state #298

wincent opened this issue Feb 8, 2016 · 46 comments

Comments

@wincent
Copy link

@wincent wincent commented Feb 8, 2016

This happens to me about once per day. It started when I moved from 1.9a to 2.1.

I'll attach to a running tmux session and all of the windows/panes in it appear frozen. But they're not actually frozen; they're just not redrawing when they receive input.

  1. Type some letters in a window, see nothing reflected on screen.
  2. Move to another pane, for example with <Prefix>o; see the cursor move to the next pane but note that typing still produces no visible output.
  3. Use <Prefix>c to create a new window, see only an empty window created (no shell prompt visible).
  4. Go back to previous window (ie. <Prefix>l) and see that the window has now updated with whatever you typed at steps 1 and 2 above.
  5. Go back to step 1 and repeat. Observe the exact same symptoms.

The OS X "Sample" tool shows this is what tmux is up to when this happens. Nothing looks too suspicious in there.

The workarounds are either:

  1. Kill all frozen panes (with <Prefix>x). This is always all of them, so you end up having to recreate the session.
  2. Use <Prefix>D to get a list of attached sessions and detach all others but the one you created. Voila. Everything gets unblocked.

Workarounds that don't work:

  • Deattaching then reattaching.

Not sure what the cause of this is, or whether it should be considered a bug or is the result of something funky I am doing. For reference, my .tmux.conf is here.

@maxjacobson
Copy link

@maxjacobson maxjacobson commented Feb 8, 2016

Wow, I was going to create a similar issue. Let me ask: how do you start tmux? I found that when I start tmux via a helper script function, this issue occurs, and when I start it via an alias, it does not occur.

Here's a GIF of what is occurring for me (does it look familiar?):

tmux

and here are the aliases I'm happily/safely using now: maxjacobson/dotfiles@6d883df

@wincent
Copy link
Author

@wincent wincent commented Feb 9, 2016

Interesting. I'm starting tmux via a function defined in here.

@maxjacobson
Copy link

@maxjacobson maxjacobson commented Feb 9, 2016

Oo that's a spicy function. Well, I think it might be a bug with tmux, but this should work for now:

alias 'tmux'='local SOCK_SYMLINK=~/.ssh/ssh_auth_sock; if [ -r "$SSH_AUTH_SOCK" -a ! -L "$SSH_AUTH_SOCK" ]; then ln -sf "$SSH_AUTH_SOCK" $SOCK_SYMLINK; fi; if [[ -n "$@" ]]; then env SSH_AUTH_SOCK=$SOCK_SYMLINK tmux "$@"; return; fi; if [ -x .tmux ]; then local DIGEST="$(openssl sha -sha512 .tmux)"; if ! grep -q "$DIGEST" ~/..tmux.digests 2> /dev/null; then cat .tmux; read -k 1 -r "REPLY?Trust (and run) this .tmux file? (t = trust, otherwise = skip) "; echo; if [[ $REPLY =~ ^[Tt]$ ]]; then echo "$DIGEST" >> ~/..tmux.digests; ./.tmux; return; fi; else ./.tmux; return; fi; fi; SESSION_NAME=$(basename "$(pwd)"); env SSH_AUTH_SOCK=$SOCK_SYMLINK tmux new -A -s "$SESSION_NAME"'

😉

@nicm
Copy link
Member

@nicm nicm commented Feb 10, 2016

Hi

What libevent version?

Does this happen if you run tmux without a .tmux.conf?

On Mon, Feb 08, 2016 at 12:12:56PM -0800, Greg Hurrell wrote:

This happens to me about once per day. It started when I moved from 1.9a
to 2.1.

I'll attach to a running tmux session and all of the windows/panes in it
appear frozen. But they're not actually frozen; they're just not redrawing
when they receive input.

1. Type some letters in a window, see nothing reflected on screen.
2. Move to another pane, for example with <Prefix>o; see the cursor move
   to the next pane but note that typing still produces no visible
   output.
3. Use <Prefix>c to create a new window, see only an empty window created
   (no shell prompt visible).
4. Go back to previous window (ie. <Prefix>l) and see that the window has
   now updated with whatever you typed at steps 1 and 2 above.
5. Go back to step 1 and repeat. Observe the exact same symptoms.

The OS X "Sample" tool shows [1]this is what tmux is up to when this
happens. Nothing looks too suspicious in there.

The workarounds are either:

1. Kill all frozen panes (with <Prefix>x). This is always all of them, so
   you end up having to recreate the session.
2. Use <Prefix>D to get a list of attached sessions and detach all others
   but the one you created. Voila. Everything gets unblocked.

Workarounds that don't work:

 * Deattaching then reattaching.

Not sure what the cause of this is, or whether it should be considered a
bug or is the result of something funky I am doing. For reference, my
.tmux.conf is [2]here.

--
Reply to this email directly or [3]view it on GitHub.

Reverse link: [4]unknown

References

Visible links

  1. https://gist.github.com/wincent/920a81585c90719a760c
  2. https://github.com/wincent/wincent/blob/master/roles/dotfiles/files/.tmux.conf
  3. #298
  4. #298
@maxjacobson
Copy link

@maxjacobson maxjacobson commented Feb 10, 2016

Hi!

What libevent version?

Using libevent 2.0.22 (via homebrew on a Mac)

Does this happen if you run tmux without a .tmux.conf?

Yes:

tmux

@wincent
Copy link
Author

@wincent wincent commented Feb 10, 2016

Same here (OS X 10.11.3, libevent 2.0.22 via Homebrew, tmux 2.1 via Homebrew).

@maxjacobson
Copy link

@maxjacobson maxjacobson commented Feb 17, 2016

I think this might have been resolved by the patch that came out of this issue #311 (comment) (782dd94). I tried installing from HEAD and couldn't reproduce the issue anymore, where previously I could reproduce it reliably. 🎉

@wincent
Copy link
Author

@wincent wincent commented Feb 18, 2016

Great! Let's assume you're right until proven otherwise.

@wincent wincent closed this Feb 18, 2016
@maxjacobson
Copy link

@maxjacobson maxjacobson commented Feb 18, 2016

actually I just tried to re-open a tmux session and it's frozen again. apologies for the confusion, I think this issue persists 😬

@wincent
Copy link
Author

@wincent wincent commented Feb 18, 2016

Awww.

@wincent wincent reopened this Feb 18, 2016
@nicm
Copy link
Member

@nicm nicm commented Feb 18, 2016

Are you attaching to a session that is also attached elsewhere?

@maxjacobson
Copy link

@maxjacobson maxjacobson commented Feb 18, 2016

I don't think so, although I'm not really sure

@wincent
Copy link
Author

@wincent wincent commented Feb 19, 2016

Are you attaching to a session that is also attached elsewhere?

In my case, no. But when I look at the list of attached sessions, I can see tmux thinks I still have other sessions. Killing them (ie. with <prefix>-D) makes things unfreeze.

@nicm
Copy link
Member

@nicm nicm commented Feb 19, 2016

Does this still happen in tmux from master? I changed how the backoff stuff works.

@wincent
Copy link
Author

@wincent wincent commented Feb 19, 2016

Just updated to master. Will let you know if I see it happen again.

@wincent
Copy link
Author

@wincent wincent commented Feb 19, 2016

Alas, just repro'd on master.

@maxjacobson
Copy link

@maxjacobson maxjacobson commented Feb 20, 2016

Same

@nicm
Copy link
Member

@nicm nicm commented Feb 21, 2016

Hmm I don't know what is going on.

Please kill tmux entirely (tmux kill-server) then start tmux with "tmux -vvvv new" and reproduce then send me the server log from the directory where you started tmux.

@maxjacobson
Copy link

@maxjacobson maxjacobson commented Feb 22, 2016

tmux2

OK reproduced, here's the server log: https://gist.github.com/maxjacobson/19a29d0c52cbac084419

(deleted the earlier comment because I was able to reproduce again and this time with a small enough log that I could upload it to gist)

@nicm
Copy link
Member

@nicm nicm commented Feb 22, 2016

There is a client that has probably disconnected but tmux has not been
told it has disconnected, it can't tell that it isn't just slow, so the
backoff mechanism is slowing down all the panes that are in the session
attached to that client. That's why if you kill the other clients it
starts working again.

On Sun, Feb 21, 2016 at 04:23:09PM -0800, Max Jacobson wrote:

[1]tmux2

OK reproduced, here's the server log:
[2]https://gist.github.com/maxjacobson/19a29d0c52cbac084419

(deleted the earlier comment because I was able to reproduce again and
this time with a small enough log that I could upload it to gist)

--
Reply to this email directly or [3]view it on GitHub.

Reverse link: [4]unknown

References

Visible links

  1. https://cloud.githubusercontent.com/assets/1421211/13206649/6ee8d556-d8d0-11e5-97b8-2c2a94cfcbff.gif
  2. https://gist.github.com/maxjacobson/19a29d0c52cbac084419
  3. #298 (comment)
  4. #298 (comment)
@tnn2
Copy link

@tnn2 tnn2 commented Feb 23, 2016

Here are my steps to reproduce:

  1. run "tmux attach" on the same session in two xterms
  2. xkill one of the xterms
  3. resume work in the first xterm
  4. typically it gets wedged within 5 seconds

(This is on tmux 2.1 as shipped with NetBSD -current)

@Duologic
Copy link

@Duologic Duologic commented Feb 29, 2016

I can confirm this issue on Arch Linux:

libevent 2.0.22-1
tmux 2.1-1

I've got a desktop which has a terminal connected to the tmux and a Macbook connecting over SSH to the same tmux. It usually gets stuck on my Macbook when my connection dropped for a few seconds and I had to reconnect to SSH, however I don't see the terminal that was connected via SSH, only the local one. Removing (<Prefix>D) the locally attached terminal fixes it again.

@boecko
Copy link

@boecko boecko commented Mar 11, 2016

i can confirm this with tmux 2.1 on SnowLeopard

  • start a tmux session with 2 windows and just 2 shells (no further output is produced)
  • don't detach and kill the Tab in iTerm/xterm
  • re-attach
  • type some letters

observations:

  • the window doesn't refresh (you don't see the letters you typed)
  • the cpu spikes after you type in something
  • if you switch between the windows, the screen is refreshed (with the typed letters)
  • the cpu goes down to normal level after the switch
  • tmux thinks the session is still attached, if you do "tmux list-sessions" outside TMUX
  • after killing the old zombie "tmux attach" -process everything is fine

This only happens on sessions, where no output is produced. Sessions, which output something all the time, behave just fine.

@nicm
Copy link
Member

@nicm nicm commented Mar 17, 2016

If you bump READ_SIZE and READ_BACKOFF in tmux.h to something huge this problem will probably go away.

@solotim
Copy link

@solotim solotim commented Mar 22, 2016

My current approach to "recover" is WAIT. Generally about 10 minutes later, it suddenly comes back normal.
Hope developers address this issue asap...

@bridgesense
Copy link

@bridgesense bridgesense commented Mar 23, 2016

I hope there's a cure for this soon. :)

@boecko
Copy link

@boecko boecko commented Mar 30, 2016

If you bump READ_SIZE and READ_BACKOFF in tmux.h to something huge this problem will probably go away.

It just postpones the problem.

@boecko
Copy link

@boecko boecko commented Mar 30, 2016

I've made git bisect to narrow down the problem
tmux_issue_298_bisect.txt

@boecko
Copy link

@boecko boecko commented Mar 30, 2016

The bad commit is 3f4ee98
The corresponding merge-commit to master is 00471dc

@nicm
Copy link
Member

@nicm nicm commented Apr 29, 2016

I have removed the backoff timer so this should be resolved in master.

@nicm nicm closed this Apr 29, 2016
@maxjacobson
Copy link

@maxjacobson maxjacobson commented Apr 29, 2016

Thanks a lot @nicm 😄

@tamasgal
Copy link

@tamasgal tamasgal commented Apr 29, 2016

Nice ☺️

@tmartinx
Copy link

@tmartinx tmartinx commented May 4, 2016

This is also happening on Ubuntu 16.04:

tmux 2.1-3build1
libevent-2.0-5:amd64 2.0.21-stable-2

I'll fill a bug report on Launchpad and link it here.

@bridgesense
Copy link

@bridgesense bridgesense commented May 5, 2016

Yeah, this is working for me now in the latest build. Thank you!

@leeourand
Copy link

@leeourand leeourand commented Jun 30, 2016

I'm running 2.2, and still can replicate this on OS X by starting tmux, killing my terminal, starting a new terminal, and attaching. Here's a server log:
https://gist.github.com/leeourand/9688d67bc1ce73a6e6200d3b78211db0

@nicm
Copy link
Member

@nicm nicm commented Jun 30, 2016

2.2 is the same as 2.1.

On Thu, Jun 30, 2016 at 06:56:53AM -0700, Lee Ourand wrote:

I'm running 2.2, and still can replicate this on OS X by starting tmux,
killing my terminal, starting a new terminal, and attaching. Here's a
server log:
[1]https://gist.github.com/leeourand/9688d67bc1ce73a6e6200d3b78211db0

--
You are receiving this because you were mentioned.
Reply to this email directly, [2]view it on GitHub, or [3]mute the thread.

Reverse link: [4]unknown

References

Visible links

  1. https://gist.github.com/leeourand/9688d67bc1ce73a6e6200d3b78211db0
  2. #298 (comment)
  3. https://github.com/notifications/unsubscribe/AASkc8z7ii2MjXyjHuZeLmrx4Hgjf_VDks5qQ8slgaJpZM4HVxBf
  4. #298 (comment)
@jay-hankins
Copy link

@jay-hankins jay-hankins commented Jul 27, 2016

I am also seeing this issue with tmux 2.2 and iTerm 2 version 3.0.

@redhotvengeance
Copy link

@redhotvengeance redhotvengeance commented Aug 1, 2016

Running into this issue with tmux 2.2 on Terminal.app on OS X 10.11.6.

@jmurinello
Copy link

@jmurinello jmurinello commented Aug 10, 2016

I ran into the same tmux semi-zombie issue.

I found out that the Zsh framework I'm using had a module that originally came with the following Zsh shell configuration:

unsetopt HUP

As soon as I changed that line unto a comment I no longer experienced the semi-zombie behaviour.

I know it will sound lame but to make sure it wasn't the Zsh framework causing it I disabled the framework an placed the unsetopt HUP in the .zshrc file and manage to replicate this issue.

Can anyone tell me what's with tmux and the disabled shell HUP?

Cheers!

UPDATE
I installed tmux using Homebrew and I guess that was my problem to begin with. It seems that brew install tmux wasn't getting it from the build from the Git master.

After I installed via git everything was just fine even with unsetopt HUP .

@NHDaly
Copy link

@NHDaly NHDaly commented Dec 5, 2016

I'm not sure if this is known, but I'm still seeing this issue in tmux 2.3.

Ubuntu 14.04 LTS
tmux 2.3
zsh 5.0.5 (x86_64-pc-linux-gnu)

It happens both to my zsh panes as well as any panes running vim.

@neerajbadlani
Copy link

@neerajbadlani neerajbadlani commented Dec 7, 2016

@NHDaly Did you try Prefix key + Capital D (detach other sessions) . It happens to me when I am generally moving from one wifi to other . The terminal feels like its stuck .

@NHDaly
Copy link

@NHDaly NHDaly commented Dec 8, 2016

@neerajbadlani: Yes, this fixed the issue for me! I almost always connect to my server running tmux over mosh (which tells you when there are other connections still active, and i always kill those before attaching to tmux), so I'd never seen this issue before. Thanks for the help.

@vikramsg
Copy link

@vikramsg vikramsg commented Mar 7, 2017

I have tmux 2.4 running and whenever there's too much text showing up the screen, it hangs. To test it I just did du -h inside a directory twice and it froze.

@fnicastri
Copy link

@fnicastri fnicastri commented Mar 10, 2017

same here, cat a log and it froze

@tmartinx
Copy link

@tmartinx tmartinx commented Apr 21, 2017

Thank you @neerajbadlani!

@lock
Copy link

@lock lock bot commented Feb 16, 2020

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Feb 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.