No way to pass stdin to Container.execute

[cjwatson]

I recently added LXD backend support to launchpad-buildd. I was able to use pylxd for almost all of it, but we need to be able to pipe standard input into lxc exec in some cases, and pylxd's Container.execute can't do that. Could this feature be added?

[ajkavanagh]

How would you see that working? Can you give an example of the types of things you are piping into lxc exec?

At the moment the execute command on a container (https://github.com/lxc/pylxd/blob/master/pylxd/models/container.py#L207) takes a list as the command (like subprocess.check_call() and friends). It then uses the lxd RESTful API to run the command (async) and uses a websocket to get the results.

So, how might what you'd like to achieve fit into that model?

[cjwatson]

One example is where we want to add trusted keys to a build. We do this by running the rough equivalent of lxc exec NAME apt-key add -, which takes key material on stdin.

Whether the command is passed as a list or not is immaterial for this purpose, although of course I agree that that's generally the best way to do things. I'm not particularly familiar with websockets, but ideally I'd like to be able to pass something like stdin=fileobj to Container.execute and have it forward bytes from that file object to the stdin websocket returned by POST /1.0/containers/NAME/exec. The API docs seem to indicate that this is supported, since they say "so that users can pass stdin and read stdout".

[ajkavanagh]

Okay, I think I understand. I think we could do something where an object could be passed that has a duck-typed read() method which work similar to io.BufferedReader type class (i.e. read bytes until EOF) which would block if it was still waiting for bytes, etc.) It would run in a separate thread in the ws4py websocket library.

As sys.stdin works like a file object, and thus, (IIRC io.TextIOWrapper) this would work if sys.stdin was passed to it, but equally, piping from files would also work (e.g. the file object from open()).

Is this what you were thinking?

[smoser]

This is pretty important missing feature. Not easily worked around at all and seemingly somewhat basic functionality.

@cjwatson how did you work around this?

[smoser]

Well, just for reference for others, we have worked around this in cloud-init by pushing (files.put) a file to a temporary location in the container, and then using 'sh' to set its stdin to the file, execute the command and then remove the file. platforms/lxd/instance.py:_execute

I came across issue #268 yesterday and filed that. Please keep in mind when fixing this that stdin should not be expected to be utf-8 either.