Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WinError87 when executing local on Connection #2142

Closed
KantiCodes opened this issue Feb 19, 2021 · 26 comments
Closed

WinError87 when executing local on Connection #2142

KantiCodes opened this issue Feb 19, 2021 · 26 comments
Labels
Awaiting feedback Contributor needs to supply extra information Nonstandard platforms Python versions

Comments

@KantiCodes
Copy link

KantiCodes commented Feb 19, 2021

Hi all, I`m new to Invoke and even newer to Fabric!

I have some functionality in Invoke and now I`m trying to make a similar demo using fabric

Following code results in WinError87
image

image
[WinError 87] Parameter is incorrect

I know it's probably caused by me misunderstanding something - but I`d really appreciate a hint :)

Setup:

Python 3.8.5
Fabric 2.6.0

@davidjmemmett
Copy link
Contributor

Hi KantiCodes,

Just to confirm, you have an SSH server running on localhost on port 8050, and the username you're wanting to connect to is 45502?

Which SSH server are you using? If you open an SSH connection using the standard openssh client (ssh 45502@localhost -p8050) and run cd .., does this work as expected?

Cheers,
David

@davidjmemmett davidjmemmett added the Awaiting feedback Contributor needs to supply extra information label Feb 19, 2021
@KantiCodes
Copy link
Author

KantiCodes commented Feb 19, 2021

Hi @davidjmemmett

Thanks so much for taking time to look at my problem :)

No, I do not have an SSH running - my understanding was that until I only run c.local() the connection is not established and it's just as if I was using invoke.

Sorry about that.

To add a little background to what I want to do.

I created invoke module and wrapped it into my own pip installing library: http://docs.pyinvoke.org/en/stable/concepts/library.html#reusing-invoke-s-cli-module-as-a-distinct-binary

Here is a simplified example:

tasker/main.py
image

tasker/tasks.py
image

I can install and access my wrapped "module"

image

Now my goal:

  • For some of the tasks I need to communicate with web server, therefore I would like to use Fabric and keep my current(Invoke's) functionality and add new tasks that.

As far as I understand @bitprophet designed fabric to be Invoke+, therefore I would like to keep my Invoke solution and add being able to connect over SSH to web servers.

To wrap up:
I want to create a fabric module performing tasks both locally and externally(over ssh) and wrap that into a pip package that I can install :D...

I am all new to this topic - let me know if my intentions are clear

@davidjmemmett
Copy link
Contributor

Gotcha - would you be able to provide the full stack trace please? I'd like to see what's happening further up the stack.

@KantiCodes
Copy link
Author

Hey. there you go.
image

@davidjmemmett davidjmemmett removed the Awaiting feedback Contributor needs to supply extra information label Feb 19, 2021
@davidjmemmett
Copy link
Contributor

From what I can see, this appears to be an invoke issue - cc'ing @jefftriplett

I don't have a Microsoft Windows machine at hand to test this with I'm afraid.

@jefftriplett
Copy link

Sadly, I don't have a Windows machine to test either.

I'm a little confused by using local through with a remote connection though. I do see it's documented in the docs by what looks like a pass-through to invoke.

I would try using invoke.run() locally to see if you get the same error. The locals sort-of lead me to this from https://docs.fabfile.org/en/2.5/api/connection.html#fabric.connection.Connection.local to http://docs.pyinvoke.org/en/latest/api/__init__.html#invoke.run

@KantiCodes
Copy link
Author

Thanks for your answer @jefftriplett.

The reason why I would like to use fabric.local instead of the invoke.run is following:

  • As I described my end goal above the tool I`m building is going to use both Invoke and Fabric and it is my understanding that I need to build a Fabric tool that uses invoke (either by sub classing or explicitly) and not the other way around.

Nonetheless I have seen some code examples where people were doing following:
root/tasks.py
image

  • Note that the task is from invoke

root/main.py
image

So now I am a little bit confused which one to choose. I do understand though that using fabric.local() is equivalent to invoke.run() I just thought that's the way to go.

@jefftriplett
Copy link

@KantiCodes sorry for the confusion. I was mostly curious if one worked vs. the other to help us figure out if the issue was really Invoke or Fabric since neither of us has a Windows machine to test it on.

@KantiCodes
Copy link
Author

No probs.

To answer your question: Invoke works properly.

@KantiCodes
Copy link
Author

Btw, could you elaborate on the screen shoot I sent?

What do u think about using Fabric functionality inside invoke@task?

@davidjmemmett
Copy link
Contributor

I'm convinced that the issue you're having is related to the Microsoft Windows executor, rather than anything odd happening between invoke and fabric invoking invoke. I'll spin up a Microsoft Windows test VM and have a play. For now, here's my findings:

tasks.py:

from invoke import task
from fabric import Connection

@task
def test(c):
    c.run('echo test from invoke')

    fab_c = Connection('localhost')
    fab_c.local('echo test from invoke via fabric')

fabfile.py is a symbolic link to tasks.py, meaning they're identical:

$ stat fabfile.py 
  File: fabfile.py -> tasks.py
  Size: 8         	Blocks: 0          IO Block: 4096   symbolic link
Device: 802h/2050d	Inode: 1192219     Links: 1
Access: (0777/lrwxrwxrwx)  Uid: ( 1000/   david)   Gid: ( 1000/   david)
Access: 2021-02-22 20:41:32.409633886 +0000
Modify: 2021-02-22 20:41:30.489574349 +0000
Change: 2021-02-22 20:41:30.489574349 +0000
 Birth: -

invoke output:

$ inv test
test from invoke
test from invoke via fabric

fabric output:

$ fab test
test from invoke
test from invoke via fabric

@davidjmemmett
Copy link
Contributor

Hi @KantiCodes,

I'm afraid I'm unable to reproduce this.

The steps I took:

  1. Create a Microsoft Windows 10 VM using the free images provided.
  2. Installed Python 3.6.6 using the Windows x86-64 executable installer (checked the box to add python to the $PATH.
  3. Launched Microsoft PowerShell
  4. Upgraded pip:
    > python -m pip install --upgrade pip
  5. Installed fabric
    > python -m pip install fabric
  6. Created a tasks.py identical to the one above:
    > notepad.exe tasks.py
  7. Copied tasks.py into fabfile.py (as Microsoft Windows still doesn't properly support symbolic links apparently):
    > cp tasks.py fabfile.py
PS C:\Users\IEUser> inv test
test from invoke
test from fabric
PS C:\Users\IEUser> fab test
test from invoke
test from fabric

Even after replacing the echo calls with cd .., it had no effect.

Can you please provide a small, standalone fabfile.py to reproduce this issue please? If possible, using a clean VM environment to test this would be preferable.

Cheers,
David

@davidjmemmett
Copy link
Contributor

I do apologise, you're using Python 3.8.5; I'll grab that and test!

@davidjmemmett
Copy link
Contributor

Aha, this is definitely related to the Python version. I've managed to reproduce this issue using Python 3.8.5.

@davidjmemmett
Copy link
Contributor

I've also managed to reproduce this issue using Python 3.8.8 (latest 3.8).

@davidjmemmett
Copy link
Contributor

Even in the latest Python 3.9.2 released a couple of days ago.

@davidjmemmett
Copy link
Contributor

I've narrowed the issue itself down to env being dropped when running under fabric. Dumping the arguments to invoke.runners.Local:start shows that env is fully populated when running under invoke, but an empty dictionary when running under fabric.

@davidjmemmett
Copy link
Contributor

It looks like this is a known issue. If I set replace_env to False, it works as expected, however this will need to be set specifically for local calls only and never for remove calls. I wonder why it worked with Python 3.6.6 though... weird.

I'll continue to investigate.

davidjmemmett added a commit to davidjmemmett/fabric that referenced this issue Feb 22, 2021
…es, replacing with specific override to clear environment only for remote.

Issue: fabric#2142
See also: 4358f34
davidjmemmett added a commit to davidjmemmett/fabric that referenced this issue Feb 23, 2021
…es, replacing with specific override to clear environment only for remote.

Issue: fabric#2142
See also: 4358f34
davidjmemmett added a commit to davidjmemmett/fabric that referenced this issue Feb 23, 2021
…es, replacing with specific override to clear environment only for remote.

Issue: fabric#2142
See also: 4358f34
@davidjmemmett
Copy link
Contributor

Hi @KantiCodes,

As a temporary test (please do not rely on this long-term), you could install fabric from the commit I pushed to see if this fixes the issue for you:

python -m pip install git+https://github.com/davidjmemmett/fabric.git@f8cf9ad30efb773f055bf7c174ef98ba225405fc

Cheers,
David

@davidjmemmett
Copy link
Contributor

Disclaimer: given the nature of this commit, there are potential security implications (whereby local environment variables have the potential to be transmitted unknowingly): i.e. it's not yet been reviewed.

@davidjmemmett davidjmemmett added the Awaiting feedback Contributor needs to supply extra information label Feb 23, 2021
@KantiCodes
Copy link
Author

Hi @davidjmemmett,

Considering what you wrote, for now I will just stick with Invoke instead of sub-classing.

@bitprophet
Copy link
Member

Thanks for triaging this David! I may be reading too fast but I don't see if we've found what changed in Python 3.8 to break this - the ultimate change you identified needs to happen regardless, but it's weird if Python 3.6 and 3.7 don't trigger the issue.

@KantiCodes as David and Jeff noted, using "straight Invoke" is fine here, especially if it works around the issue for now. Eventually I do want "just use Fabric because you intend to do both local and remote things" to be a smooth experience - it's not there yet unfortunately. Mixing the two in the same code shouldn't be all that much more annoying for now, hopefully!

I'm talking with David in his PR and I expect to merge it sometime, so let's close this to roll it into there. Thanks for the report!

davidjmemmett added a commit to davidjmemmett/fabric that referenced this issue Mar 11, 2021
…es, replacing with specific override to clear environment only for remote.

Issue: fabric#2142
See also: 4358f34
bitprophet pushed a commit that referenced this issue Mar 19, 2021
…es, replacing with specific override to clear environment only for remote.

Issue: #2142
See also: 4358f34
davidjmemmett added a commit to davidjmemmett/fabric that referenced this issue Mar 19, 2021
…es, replacing with specific override to clear environment only for remote.

Issue: fabric#2142
See also: 4358f34
@fripSide
Copy link

fripSide commented May 14, 2021

I also meet the same problem. In Python 3.8.5 with Fabric 2.6.9, I failed to execute local commend in CMD with either ctx.run or conn.local.
`
from fabric import task, Connection

@task
def test(ctx):
conn = Connection("xxx")
ctx.run("cd D:/")
# conn.local("cd D:/")
`

The error message is,
File "d:\dev\python\python3.8.5\lib\site-packages\invoke\runners.py", line 1271, in start
self.process = Popen(
File "d:\dev\python\python3.8.5\lib\subprocess.py", line 854, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "d:\dev\python\python3.8.5\lib\subprocess.py", line 1308, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
OSError: [WinError 87]

I solve this problem by manually merge the two commits.
davidjmemmett@0ce88d2
https://github.com/davidjmemmett/fabric/blob/0ce88d26ee69f22fda1ebcd4bd579e1652131fd6/fabric/runners.py

I hope that you can release a new version to fix this problem as many people use Python 3.8/3.9 now.

@wookiesh
Copy link

Hello, I'm experiencing the same issue here with pythonpython 3.9.6 and fabric 2.6 (up to date according to poetry ?) but the issue is closed.

Any pointer on how to address it ?

@davidjmemmett
Copy link
Contributor

There hasn't been a release for a while - @bitprophet

@swasher
Copy link

swasher commented Aug 26, 2021

I have this issue too (Windows10, Python 3.9.5, fabric 2.6.0). But if I understood the thread correctly, the problem appear only with local connection. I have with remote too, BUT only in tasks:

from fabric import Connection
c = Connection('sandglass', user='swasher')
c.run('echo test from fabric')

Output:

 λ fab
test from fabric
Usage: fab [--cor....

If I try put this into task, I got error:

from fabric import Connection, task

c = Connection('sandglass', user='swasher')

@task
def proba(c):
    c.run('echo test from fabric')

Output:

(sandglass) λ fab proba
Traceback (most recent call last):
  File "c:\users\alexeyd\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\alexeyd\appdata\local\programs\python\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\alexeyd\.virtualenvs\sandglass-rdO3Dwhs\Scripts\fab.exe\__main__.py", line 7, in <module>
  File "c:\users\alexeyd\.virtualenvs\sandglass-rdo3dwhs\lib\site-packages\invoke\program.py", line 384, in run
    self.execute()
  File "c:\users\alexeyd\.virtualenvs\sandglass-rdo3dwhs\lib\site-packages\invoke\program.py", line 566, in execute
    executor.execute(*self.tasks)
  File "c:\users\alexeyd\.virtualenvs\sandglass-rdo3dwhs\lib\site-packages\invoke\executor.py", line 129, in execute
    result = call.task(*args, **call.kwargs)
  File "c:\users\alexeyd\.virtualenvs\sandglass-rdo3dwhs\lib\site-packages\invoke\tasks.py", line 127, in __call__
    result = self.body(*args, **kwargs)
  File "C:\Users\alexeyd\PycharmProjects\sandglass\fabfile.py", line 8, in proba
    c.run('echo test from invoke')
  File "c:\users\alexeyd\.virtualenvs\sandglass-rdo3dwhs\lib\site-packages\invoke\context.py", line 95, in run
    return self._run(runner, command, **kwargs)
  File "c:\users\alexeyd\.virtualenvs\sandglass-rdo3dwhs\lib\site-packages\invoke\context.py", line 102, in _run
    return runner.run(command, **kwargs)
  File "c:\users\alexeyd\.virtualenvs\sandglass-rdo3dwhs\lib\site-packages\invoke\runners.py", line 380, in run
    return self._run_body(command, **kwargs)
  File "c:\users\alexeyd\.virtualenvs\sandglass-rdo3dwhs\lib\site-packages\invoke\runners.py", line 431, in _run_body
    self.start(command, self.opts["shell"], self.env)
  File "c:\users\alexeyd\.virtualenvs\sandglass-rdo3dwhs\lib\site-packages\invoke\runners.py", line 1291, in start
    self.process = Popen(
  File "c:\users\alexeyd\appdata\local\programs\python\python39\lib\subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "c:\users\alexeyd\appdata\local\programs\python\python39\lib\subprocess.py", line 1420, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
OSError: [WinError 87] Параметр задан неверно

Russian output, OSError: [WinError 87] Параметр задан неверно -> The parameter is incorrect

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Awaiting feedback Contributor needs to supply extra information Nonstandard platforms Python versions
Projects
None yet
Development

No branches or pull requests

7 participants