pty handling issue with native driver (breaks SSH server)

[jpetazzo]

TL,DR: we should symlink /dev/ptmx to /dev/pts/ptmx unconditionally, not only for containers with a pty.

Details now.

When we start a container, we mount a new devpts instance on /dev/pts.

If we request a terminal (with docker run -t …), the native driver calls the setupPtmx function in nsinit/mount.go:

https://github.com/dotcloud/docker/blob/master/pkg/libcontainer/nsinit/mount.go#L240

This function does two things:

• it removes /dev/ptmx and replaces it with a symlink to /dev/pts/ptmx;
• it bind-mounts the current pseudo-terminal to /dev/console.

If we start a container with the -t option, everything works fine.

Now, if we do this:

• start a container without the -t option
• start a SSH server inside the container
• try to SSH to that server
  ... then we won't get a terminal.
  SSH debug info will indicate that the openpty() function call failed, and strace will show that the SSH server tried to stat something like /dev/pts/10 instead of /dev/pts/0.

This is because when you open the ptmx device, the kernel identifies which instance is accessed by looking at the device on which it resides. When /dev/ptmx is directly a character device, it resides on /dev, which is itself on the container's rootfs, which itself is "not known" to the devpts system. Now, when /dev/ptmx is a symlink to /dev/pts/ptmx, then ptmx actually resides on /dev/pts, which is a devpts filesystem and identified as such by the pty kernel code.

[mingfang]

+1

[tianon]

Just out of curiosity, what's wrong with specifying -t? There's been some talk "around the campfire" of trying to make -t default to on anyhow, but that's neither here nor there.

[jpetazzo]

Nothing wrong with -t; but IMHO it shouldn't be necessary to allocate a PTY for every container.

To put it with different words: I would be very concerned if we had to fallback to that kind of hacks ("Let's enable -t by default because some things break if we don't, and we have no idea why!"). If that makes sense :-)

[tianon]

Yep, I'm tracking now. :)

[wking]

On Tue, Mar 11, 2014 at 06:46:53PM -0700, Jérôme Petazzoni wrote:

What I do:

Run SSH server in a container which doesn't have a controlling terminal.
Docker 0.9.0.

What I see:

SSH fails to allocate a PTY, and my remote session doesn't get one.

To broaden the scope a bit, this is also an issue with 'docker build'
commands. I just got an "openpty failed: 'out of pty devices'" error
while emerging dev-libs/eventlog-0.2.12 in a 'docker build' run
(Gentoo container) with Docker 0.9.0 on a Debian-testing host
(dockerfile template here 1). Since 'docker build' doesn't have an
option to attach a pty device, I had to restart my docker daemon with
'-e lxc', which avoids this “native driver lacks default pty” issue.

[jpetazzo]

How to reproduce:

docker run -t -i ubuntu python -c "import pty; print pty.openpty()"
(3, 4)

docker run -i ubuntu python -c "import pty; print pty.openpty()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/pty.py", line 29, in openpty
    master_fd, slave_name = _open_terminal()
  File "/usr/lib/python2.7/pty.py", line 70, in _open_terminal
    raise os.error, 'out of pty devices'
OSError: out of pty devices

[rafecolton]

For what it's worth, if you specify -t and you're running the container as an upstart service, when you run initctl stop <container>, it stops the upstart service but fails to stop the actual container. A manual docker stop <container> is necessary. That was not an issue before this regression.