Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'run' does not handle quote properly #191

Closed
mijikai opened this issue Nov 9, 2014 · 4 comments
Closed

'run' does not handle quote properly #191

mijikai opened this issue Nov 9, 2014 · 4 comments

Comments

@mijikai
Copy link

@mijikai mijikai commented Nov 9, 2014

Trying to run this:

import invoke

string_in_quote = "      hello\t\t\nworld with spaces"
invoke.run(""" eval 'echo "{}" ' """.format(string_in_quote))

will work as expected but when I run this with pty=True, it throws a parsing error.

This was due to the string passed as it is without the quotes escaped.

@bitprophet
Copy link
Member

@bitprophet bitprophet commented Nov 12, 2014

I don't think it's technically run's own fault...I dug a bit after reproducing the issue:

  • With pty=False, we end up running a straightforward subprocess.Popen(command, shell=True, ...) execution. Clearly this works fine.

  • With pty=True, we use a vendored pexpect module which ends up chopping up the command you give it and handing it to os.execv().

  • Because of how execv works, we have to give pexpect a wrapped version of the command (currently /bin/bash -c "command" though I expect we'll make that configurable sometime), but I don't think this is what's doing it.

  • Instead I suspect something in how pexpect is chopping up the string is breaking it, because when I print out the args it's giving to execve we get this with your input:

    ['/bin/bash', '-c', " eval 'echo ", 'hello', 'world', 'with', "spaces ' "]

    which feels pretty incorrect to me.

  • Indeed, if I do run("echo 'hi' how are 'you'?", pty=True) - which is still a nontrivial quoted multi-word string, but not one quite as wacky as yours - the same variable looks like:

    ['/bin/bash', '-c', "echo 'hi' how are 'you'?"]

    wherein the entirety of the substring given to -c is preserved as a single Python string.

Will poke a bit more and see what's happening exactly. We vendor pexpect but I'm not above patching it if necessary.

Loading

@bitprophet
Copy link
Member

@bitprophet bitprophet commented Nov 12, 2014

Yup, it does its own handrolled parsing here:

def split_command_line(command_line):

Wondering if we can simply replace that with shlex or something.

The good news is that pexpect doesn't run things through that function if we hand it an explicit args - so we can do our own splitting beforehand if we cannot determine a good way to modify that state machine, or if doing so seems worse than just doing pexpect.spawn(command, shlex(command)) or similar.

Loading

@bitprophet
Copy link
Member

@bitprophet bitprophet commented Nov 13, 2014

Sadly, shlex.split yields identical results to pexpect's own homegrown stuff. So whatever logic they're both using, really does not like the nesting going on here, despite sh/bash apparently supporting it. Gotta think a little harder to see what we can realistically do here.

Loading

@bitprophet
Copy link
Member

@bitprophet bitprophet commented Nov 13, 2014

Turns out it was way simpler than that; the entire point here is not to split the command given to bash! So simply changing our pexpect call to pexpect.spawn('/bin/bash', ['-c', command]) makes the example work fine as far as I can tell.

Loading

bitprophet added a commit that referenced this issue Nov 13, 2014
bitprophet added a commit that referenced this issue Nov 13, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants