New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot run ctx.run with pty=True on Alpine Linux (musl) #488

Closed
rajiteh opened this Issue Nov 13, 2017 · 2 comments

Comments

Projects
None yet
3 participants
@rajiteh

rajiteh commented Nov 13, 2017

I'm trying to run invoke in a docker container with the official python:2.7.13-alpine image and running in to OSError: [Errno 5] I/O error whenever invoke tries to execute an external command using ctx.run(cmd, pty=True).

Running the same task on the debian based image of python 2.7.13 doesn't cause the problem.

Steps to reproduce.

create a dummy tasks.py

from invoke import task

@task
def foo(ctx, pty=False):
  ctx.config.run.shell = '/bin/sh'
  ctx.run("/bin/uname -a", pty=pty)

run the docker container (fails)

 docker run -it --rm -v `pwd`/tasks.py:/tasks.py python:2.7.13-alpine /bin/sh -c "pip install invoke && cd / && invoke foo --pty"

Collecting invoke
  Downloading invoke-0.21.0-py2-none-any.whl (154kB)
    100% |████████████████████████████████| 163kB 1.0MB/s
Installing collected packages: invoke
Successfully installed invoke-0.21.0
Linux 8ed453c901c2 4.9.49-moby #1 SMP Wed Sep 27 23:17:17 UTC 2017 x86_64 Linux
Traceback (most recent call last):
  File "/usr/local/bin/invoke", line 11, in <module>
    sys.exit(program.run())
  File "/usr/local/lib/python2.7/site-packages/invoke/program.py", line 293, in run
    self.execute()
  File "/usr/local/lib/python2.7/site-packages/invoke/program.py", line 414, in execute
    executor.execute(*self.tasks)
  File "/usr/local/lib/python2.7/site-packages/invoke/executor.py", line 129, in execute
    result = call.task(*args, **call.kwargs)
  File "/usr/local/lib/python2.7/site-packages/invoke/tasks.py", line 115, in __call__
    result = self.body(*args, **kwargs)
  File "/tasks.py", line 7, in foo
    ctx.run("/bin/uname -a", pty=pty)
  File "/usr/local/lib/python2.7/site-packages/invoke/context.py", line 82, in run
    return self._run(runner, command, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/invoke/context.py", line 89, in _run
    return runner.run(command, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/invoke/runners.py", line 262, in run
    return self._run_body(command, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/invoke/runners.py", line 356, in _run_body
    raise ThreadException(thread_exceptions)
invoke.exceptions.ThreadException:
Saw 1 exceptions within threads (OSError):


Thread args: {'kwargs': {'buffer_': [u'Linux 8ed453c901c2 4.9.49-moby #1 SMP Wed Sep 27 23:17:17 UTC 2017 x86_64 Linux\r\n'],
            'hide': False,
            'output': <open file '<stdout>', mode 'w' at 0x7f069f8ac150>},
 'target': <bound method Local.handle_stdout of <invoke.runners.Local object at 0x7f069c8b66d0>>}

Traceback (most recent call last):

  File "/usr/local/lib/python2.7/site-packages/invoke/util.py", line 202, in run
    super(ExceptionHandlingThread, self).run()

  File "/usr/local/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)

  File "/usr/local/lib/python2.7/site-packages/invoke/runners.py", line 550, in handle_stdout
    reader=self.read_proc_stdout,

  File "/usr/local/lib/python2.7/site-packages/invoke/runners.py", line 515, in _handle_output
    for data in self.read_proc_output(reader):

  File "/usr/local/lib/python2.7/site-packages/invoke/runners.py", line 490, in read_proc_output
    data = reader(self.read_chunk_size)

  File "/usr/local/lib/python2.7/site-packages/invoke/runners.py", line 916, in read_proc_stdout
    data = os.read(self.parent_fd, num_bytes)

OSError: [Errno 5] I/O error

If pty is disabled, it works.

docker run -it --rm -v `pwd`/tasks.py:/tasks.py python:2.7.13-alpine /bin/sh -c "pip install invoke && cd / && invoke foo"

If used the debian based image, it works regardless.

docker run -it --rm -v `pwd`/tasks.py:/tasks.py python:2.7.13 /bin/sh -c "pip install invoke && cd / && invoke foo --pty"

My guess is that musl compiled python returns different exception string than the gcc one and doesn't get trapped here: https://github.com/pyinvoke/invoke/blob/master/invoke/runners.py#L918-L920

@noirbizarre

This comment has been minimized.

noirbizarre commented Jan 16, 2018

Same issue for me.

Sometimes it's possible to set pty=False but sometimes, the call only works with pty=True :/

bitprophet added a commit that referenced this issue Jan 29, 2018

bitprophet added a commit that referenced this issue Jan 29, 2018

@bitprophet

This comment has been minimized.

Member

bitprophet commented Jan 29, 2018

While I haven't actually reproduced the issue on Alpine itself, @rajiteh is right that this seems to be an issue with the unfortunate string test around IO related OSErrors. (Sob. Y U no additional subclasses, Python?)

I've just pushed a test & patch to master and it'll go out next bugfix. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment