Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closing stdin stream #53

Closed
dininski opened this issue Mar 29, 2014 · 12 comments
Closed

Closing stdin stream #53

dininski opened this issue Mar 29, 2014 · 12 comments
Labels

Comments

@dininski
Copy link

After attaching and writing data to stdin I need to close it somehow. I have some code that is being executed and expects the stdin in the container to be closed at a certain poinrt. Is there a way for this to be accomplished?

@apocas
Copy link
Owner

apocas commented Mar 30, 2014

Something like this? #8 :)

@dininski
Copy link
Author

Not quite. I am attaching stdin with StdinOnce and AttachStdin set to true. The result is that I am getting an HttpDuplex which does not have destroy(). The exact code is here:
https://github.com/dininski/CodeExecutionServer/blob/master/src/Modules/CodeExecution/Executors/BaseExecutor.js#L34
After writing to the stream I try to close it. But the container's stdin does not close at all. My test is the following: I have a php code that needs to be executed. It reads the entire stdin in a while loop and once stdin is closed the php code execution continues. However stdin never gets closed and I am stuck in the while loop.

@apocas
Copy link
Owner

apocas commented Mar 31, 2014

Oh, Yeah attaching to stdin will use HttpDuplex.

Have you tried using httpduplex's end? https://github.com/apocas/docker-modem/blob/master/lib/http_duplex.js#L44

@dininski
Copy link
Author

Yes, I did. However the this.req.end did not accept a callback. Even after updating the code and trying to explicitly call the callback after calling this.req.end() it still wouldn't work as expected. Any ideas?

@apocas
Copy link
Owner

apocas commented Mar 31, 2014

That's weird.
Have to debug it to see what's happening.

@apocas
Copy link
Owner

apocas commented Aug 20, 2014

Using something similar to this:
https://github.com/apocas/dockerode/blob/master/examples/run_stdin.js#L82
(without process related stuff)

I was able to close the stdin stream and even reattach later directly on docker.

@masonforest
Copy link

I'm having trouble closing stdin as well.

I'm trying to emulate running this: echo hello | docker run -i ubuntu wc

Here's a gist that should print 1 1 6. Instead it hangs.

Notice when you change line 14 to

'Cmd': ['echo','hello'],

it outputs hello as expected.

Thanks for all your hard work @apocas! Let me know if there is anything else I can provide to make this easier to debug.

@vincentwoo
Copy link
Contributor

I am also finding that .end() on an HttpDuplex is insufficient - lsof shows a socket leak. I'll make a repro script shortly.

@vincentwoo
Copy link
Contributor

Here's a repro.

First, pull ubuntu:12.04.

Them run this file (coffeescript, but easily compiled to JS if you like):

docker = new (require('dockerode'))(socketPath: '/var/run/docker.sock')

createContainer = (i) ->
  console.log "creating container #{i}"
  docker.createContainer {
    Cmd:          '/bin/bash'
    Image:        'ubuntu:12.04'
    OpenStdin:    true
    Tty:          true
  }, (err, container) ->
    console.log "attaching to container #{i}"
    container.attach {
      stream: true,
      stdin: true,
      stdout: true
    }, (err, ttyStream) ->
      setTimeout ->
        console.log "ending container #{i} tty stream"
        ttyStream.end()
        container.remove { force: true }, ->
          console.log "container #{i} removed"
      , 5000

for i in [0..20]
  setTimeout createContainer, i * 250, i

Now, lsof your still-running node process (it won't terminate, which is also a red flag):

$ sudo lsof -p 12534
COMMAND   PID USER   FD   TYPE             DEVICE SIZE/OFF   NODE NAME
node    12534 vwoo  cwd    DIR               0,32      544      0 /home/vwoo/coderpad-tty
node    12534 vwoo  rtd    DIR                8,1     4096      2 /
node    12534 vwoo  txt    REG                8,1  8442472 534361 /usr/bin/nodejs
node    12534 vwoo  mem    REG                8,1  1853400 130981 /lib/x86_64-linux-gnu/libc-2.17.so
node    12534 vwoo  mem    REG                8,1   135757 131069 /lib/x86_64-linux-gnu/libpthread-2.17.so
node    12534 vwoo  mem    REG                8,1    88408 136446 /lib/x86_64-linux-gnu/libgcc_s.so.1
node    12534 vwoo  mem    REG                8,1  1063328 131017 /lib/x86_64-linux-gnu/libm-2.17.so
node    12534 vwoo  mem    REG                8,1   979056 526754 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.18
node    12534 vwoo  mem    REG                8,1    31760 131075 /lib/x86_64-linux-gnu/librt-2.17.so
node    12534 vwoo  mem    REG                8,1    14664 130995 /lib/x86_64-linux-gnu/libdl-2.17.so
node    12534 vwoo  mem    REG                8,1   149312 130961 /lib/x86_64-linux-gnu/ld-2.17.so
node    12534 vwoo    0u   CHR              136,3      0t0      6 /dev/pts/3
node    12534 vwoo    1u   CHR              136,3      0t0      6 /dev/pts/3
node    12534 vwoo    2u   CHR              136,3      0t0      6 /dev/pts/3
node    12534 vwoo    3r  FIFO                0,8      0t0 462080 pipe
node    12534 vwoo    4w  FIFO                0,8      0t0 462080 pipe
node    12534 vwoo    5u  0000                0,9        0   6264 anon_inode
node    12534 vwoo    6r  FIFO                0,8      0t0 462081 pipe
node    12534 vwoo    7w  FIFO                0,8      0t0 462081 pipe
node    12534 vwoo    8u  0000                0,9        0   6264 anon_inode
node    12534 vwoo    9r   DIR                8,1     4096      2 /
node    12534 vwoo   10u  unix 0xffff8800a5420a80      0t0 462145 socket
node    12534 vwoo   11u  unix 0xffff880083435f80      0t0 462442 socket
node    12534 vwoo   12u  unix 0xffff880083437b80      0t0 462447 socket
node    12534 vwoo   13u  unix 0xffff88004c825f80      0t0 463457 socket
node    12534 vwoo   14u  unix 0xffff880083435180      0t0 462513 socket
node    12534 vwoo   15u  unix 0xffff880083434a80      0t0 462407 socket
node    12534 vwoo   16u  unix 0xffff880083435c00      0t0 462505 socket
node    12534 vwoo   17u  unix 0xffff8800a5423800      0t0 462164 socket
node    12534 vwoo   18u  unix 0xffff8801116c9c00      0t0 462326 socket
node    12534 vwoo   19u  unix 0xffff8801116cb100      0t0 462307 socket
node    12534 vwoo   20u  unix 0xffff88004c824000      0t0 463309 socket
node    12534 vwoo   21u  unix 0xffff8800a5423100      0t0 462266 socket
node    12534 vwoo   22u  unix 0xffff88004c827b80      0t0 463350 socket
node    12534 vwoo   23u  unix 0xffff8801128ff800      0t0 462401 socket
node    12534 vwoo   24u  unix 0xffff8800a5422a00      0t0 462271 socket
node    12534 vwoo   25u  unix 0xffff8800a5420380      0t0 462232 socket
node    12534 vwoo   26u  unix 0xffff8800a5422d80      0t0 462230 socket
node    12534 vwoo   27u  unix 0xffff8800a5421880      0t0 462194 socket
node    12534 vwoo   28u  unix 0xffff8800a5422680      0t0 462188 socket
node    12534 vwoo   29u  unix 0xffff8800a5421c00      0t0 462169 socket
node    12534 vwoo   30u  unix 0xffff8801128ffb80      0t0 462380 socket

20 open FDs still.

P.S. here's a JS file if you just want to use that:

var createContainer, docker, i, _i;

docker = new (require('dockerode'))({
  socketPath: '/var/run/docker.sock'
});

createContainer = function(i) {
  console.log("creating container " + i);
  return docker.createContainer({
    Cmd: '/bin/bash',
    Image: 'ubuntu:12.04',
    OpenStdin: true,
    Tty: true
  }, function(err, container) {
    console.log("attaching to container " + i);
    return container.attach({
      stream: true,
      stdin: true,
      stdout: true
    }, function(err, ttyStream) {
      return setTimeout(function() {
        console.log("ending container " + i + " tty stream");
        ttyStream.end();
        return container.remove({
          force: true
        }, function() {
          return console.log("container " + i + " removed");
        });
      }, 5000);
    });
  });
};

for (i = _i = 0; _i <= 20; i = ++_i) {
  setTimeout(createContainer, i * 250, i);
}

@apocas would really love your eyes on this one

@vincentwoo
Copy link
Contributor

@apocas regarding your fix, are we supposed to call end or destroy on our HttpDuplexes?

@apocas
Copy link
Owner

apocas commented Dec 28, 2014

end

I just added destroy to pass this functionality from res and req in case one day we need it :)

@apocas
Copy link
Owner

apocas commented Dec 28, 2014

docker-modem version published: v0.1.20

Update docker-modem to have this fixed :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants