paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand #2695

arob182 · 2022-03-23T17:12:48Z

ISSUE:

I tried following the thread, #2645, but could not reach a solution. My topology is typical.. Win10 ---> Jumpserver --> Device. When running my /.ssh/config script via powershell I can route through the proxy to end device with no issue. Even without specifying the path in powershell I route through the proxy (ex: ssh user@X.X.X.X brings me through the proxy to the switch). When I attempt to use netmiko I get thrown these errors. Below is my /.ssh/config, python script, and output. I am using Netmiko 3.4.0 and paramiko 2.8.1

Paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand("ssh -F c:/users//.ssh/config -W :22 jumphost

permissionError: [WinError 5] Access is denied

/.ssh/config

host jumphost
IdentityFile ~/.ssh/id_rsa
User test
HostName domain.com

host * !jumphost
User localuser
ProxyCommand ssh -F c:/users//.ssh/config -W %h:%p jumphost

script

from netmiko import ConnectHandler
from getpass import getpass
import os

cisco_9300 = {
'device_type' : 'cisco_xe',
'host' : 'host',
'username' : 'localuser',
'password' : 'password',
'ssh_config_file' : r"c:\users\.ssh\config",
}

net_conenct = ConnectHandler(**cisco_9300)
output = net_conenct.sendcommand("show inv")
print(output)

Python Return Code

`(base) C:\Users\ >python C:\Users\ \Python\test.py
Exception: ProxyCommand("ssh -F c:/users/ /.ssh/config -W :22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket
Traceback (most recent call last):
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\proxy.py", line 107, in recv
r, w, x = select([self.process.stdout], [], [], select_timeout)
OSError: [WinError 10038] An operation was attempted on something that is not a socket

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\transport.py", line 2039, in run
self._check_banner()
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\transport.py", line 2211, in _check_banner
buf = self.packetizer.readline(timeout)
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\packet.py", line 380, in readline
buf += self._read_timeout(timeout)
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\packet.py", line 607, in _read_timeout
x = self.__socket.recv(128)
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\proxy.py", line 119, in recv
raise ProxyCommandFailure(" ".join(self.cmd), e.strerror)
paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand("ssh -F c:/users/ /.ssh/config -W :22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket

Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\ \Anaconda3\lib\threading.py", line 973, in _bootstrap_inner
Traceback (most recent call last):
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\proxy.py", line 107, in recv
self.run()
r, w, x = select([self.process.stdout], [], [], select_timeout)
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\transport.py", line 2166, in run
OSError: [WinError 10038] An operation was attempted on something that is not a socket
self.sock.close()

During handling of the above exception, another exception occurred:

File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\proxy.py", line 122, in close
Traceback (most recent call last):
os.kill(self.process.pid, signal.SIGTERM)
File "C:\Users\ \Python\test.py", line 13, in
PermissionError: [WinError 5] Access is denied
net_conenct = ConnectHandler(**cisco_9300)
File "C:\Users\ \Anaconda3\lib\site-packages\netmiko\ssh_dispatcher.py", line 326, in ConnectHandler
return ConnectionClass(*args, **kwargs)
File "C:\Users\ \Anaconda3\lib\site-packages\netmiko\cisco\cisco_ios.py", line 17, in init
return super().init(*args, **kwargs)
File "C:\Users\ \Anaconda3\lib\site-packages\netmiko\base_connection.py", line 350, in init
self._open()
File "C:\Users\ \Anaconda3\lib\site-packages\netmiko\base_connection.py", line 355, in _open
self.establish_connection()
File "C:\Users\ \Anaconda3\lib\site-packages\netmiko\base_connection.py", line 935, in establish_connection
self.remote_conn_pre.connect(**ssh_connect_params)
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\client.py", line 406, in connect
t.start_client(timeout=timeout)
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\transport.py", line 660, in start_client
raise e
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\transport.py", line 2039, in run
self._check_banner()
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\transport.py", line 2211, in _check_banner
buf = self.packetizer.readline(timeout)
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\packet.py", line 380, in readline
buf += self._read_timeout(timeout)
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\packet.py", line 607, in _read_timeout
x = self.__socket.recv(128)
File "C:\Users\ \Anaconda3\lib\site-packages\paramiko\proxy.py", line 119, in recv
raise ProxyCommandFailure(" ".join(self.cmd), e.strerror)
paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand("ssh -F c:/users/ /.ssh/config -W :22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket

The text was updated successfully, but these errors were encountered:

ktbyers · 2022-03-23T22:46:37Z

Why the double forward slash here?

ProxyCommand ssh -F c:/users//.ssh/config -W %h:%p jumphost

arob182 · 2022-03-23T22:49:25Z

@ktbyers I just deleted my company username from the directory. Sorry should have specified that.

ktbyers · 2022-03-23T22:50:55Z

@arob182 Okay, so you edited this error line here?

paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand("ssh -F c:/users/ /.ssh/config -W :22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket

And that extra space is not really in the output?

Just verifying.

arob182 · 2022-03-23T22:51:00Z

@ktbyers same with

paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand("ssh -F c:/users/ /.ssh/config -W :22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket

after the -W is the IP of the host put here 'host' : 'host',

arob182 · 2022-03-23T22:51:41Z

@arob182 Okay, so you edited this error line here?
paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand("ssh -F c:/users/ /.ssh/config -W :22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket
And that extra space is not really in the output?

Just verifying.

Correct, I can replace with a filler if that makes parsing this easier :)

arob182 · 2022-03-23T22:55:04Z

Here is a new run of the script with it filled < host> as the IP i am trying to reach and < user> for the name.

(base) C:\Users\<user>>python c:\users\<user>\python\test.py
Exception: ProxyCommand("ssh -F c:/users/<user>/.ssh/config -W <host>:22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket
Traceback (most recent call last):
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\proxy.py", line 107, in recv
    r, w, x = select([self.process.stdout], [], [], select_timeout)
OSError: [WinError 10038] An operation was attempted on something that is not a socket

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\transport.py", line 2039, in run
    self._check_banner()
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\transport.py", line 2211, in _check_banner
    buf = self.packetizer.readline(timeout)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\packet.py", line 380, in readline
    buf += self._read_timeout(timeout)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\packet.py", line 607, in _read_timeout
    x = self.__socket.recv(128)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\proxy.py", line 119, in recv
    raise ProxyCommandFailure(" ".join(self.cmd), e.strerror)
paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand("ssh -F c:/users/<user>/.ssh/config -W <host>:22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket

Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\<user>\Anaconda3\lib\threading.py", line 973, in _bootstrap_inner
Traceback (most recent call last):
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\proxy.py", line 107, in recv
    r, w, x = select([self.process.stdout], [], [], select_timeout)
OSError: [WinError 10038] An operation was attempted on something that is not a socket
    self.run()

During handling of the above exception, another exception occurred:

  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\transport.py", line 2166, in run
Traceback (most recent call last):
  File "c:\users\<user>\python\test.py", line 13, in <module>
    self.sock.close()
    net_conenct = ConnectHandler(**cisco_9300)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\proxy.py", line 122, in close
  File "C:\Users\<user>\Anaconda3\lib\site-packages\netmiko\ssh_dispatcher.py", line 326, in ConnectHandler
    os.kill(self.process.pid, signal.SIGTERM)
PermissionError: [WinError 5] Access is denied
    return ConnectionClass(*args, **kwargs)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\netmiko\cisco\cisco_ios.py", line 17, in __init__
    return super().__init__(*args, **kwargs)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\netmiko\base_connection.py", line 350, in __init__
    self._open()
  File "C:\Users\<user>\Anaconda3\lib\site-packages\netmiko\base_connection.py", line 355, in _open
    self.establish_connection()
  File "C:\Users\<user>\Anaconda3\lib\site-packages\netmiko\base_connection.py", line 935, in establish_connection
    self.remote_conn_pre.connect(**ssh_connect_params)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\client.py", line 406, in connect
    t.start_client(timeout=timeout)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\transport.py", line 660, in start_client
    raise e
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\transport.py", line 2039, in run
    self._check_banner()
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\transport.py", line 2211, in _check_banner
    buf = self.packetizer.readline(timeout)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\packet.py", line 380, in readline
    buf += self._read_timeout(timeout)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\packet.py", line 607, in _read_timeout
    x = self.__socket.recv(128)
  File "C:\Users\<user>\Anaconda3\lib\site-packages\paramiko\proxy.py", line 119, in recv
    raise ProxyCommandFailure(" ".join(self.cmd), e.strerror)
paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand("ssh -F c:/users/<user>/.ssh/config -W <host>:22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket

ktbyers · 2022-03-23T23:04:41Z

This error looks like some sort of a Windows permissions error:

permissionError: [WinError 5] Access is denied

That is where I would search and dig some more on.

I assume you have Windows Subsystem for Linux?

arob182 · 2022-03-23T23:20:19Z

@ktbyers Thanks ill dig into it some more. I do have WSL, but I am relatively new to utilizing it. I will also look into how to run the script from there.

ktbyers · 2022-03-23T23:35:33Z

Yeah, that permissionError/WinError is the main thing that looked odd to me.

That and this message also:

An operation was attempted on something that is not a socket

That is where I would probably start.

You might want to also try to just directly use the SSH config level from WSL and openssh (i.e. test wheter it works outside of Netmiko using WSL).

deveshmanish · 2022-06-14T12:16:09Z

Did you manage to get a solution?

shibu503 · 2022-07-17T18:31:31Z

I am receiving same error permissionError: [WinError 5] Access is denied. @arob182 did you manage to get the solution?
Using windows 10, python version is 3.9.1, netmiko 4.1.1. Trying to use ssh proxy with Windows as script server, remote device is behind linux host.

ktbyers · 2022-07-18T15:52:09Z

@shibu503 @deveshmanish Can you try this fix?

https://github.com/paramiko/paramiko/pull/1924/files

And let me know if it works?

shibu503 · 2022-07-18T18:26:51Z

Hi Ktbyers,

Thanks for your prompt response.

I have already tried the above fix to resolve same issue as mentioned in #2855. I was initially receiving below error:

"ProxyCommand("ssh -F ./config -W <dev_ip_address>:22 jumphost") returned nonzero exit status: An operation was attempted on something that is not a socket"

After making changes in paramiko/proxy.py as mentioned in https://github.com/paramiko/paramiko/pull/1924/files, the error was gone however getting permissionError: [WinError 5] Access is denied.
From windows cli, ssh -F C:\Users\ShibuJ.ssh\config <dev_ip_address>, it works fine as I am able to login to device proxying via jump host.

Below is full error:

PS C:\Users\ShibuJ\Documents\python3-venv1\p3-project> & c:/Users/ShibuJ/Documents/python3-venv1/p3-project/Scripts/python.exe c:/Users/ShibuJ/Documents/python3-venv1/p3-project/test.py
{'R8': Host: R8}
Exception in thread Thread-1:
Traceback (most recent call last):
File "threading.py", line 954, in _bootstrap_inner
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\paramiko\transport.py", line 2226, in run
self.sock.close()
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\paramiko\proxy.py", line 127, in close
os.kill(self.process.pid, signal.SIGTERM)
PermissionError: [WinError 5] Access is denied
netmiko_send_config*************************************************************

R8 ** changed : False ********************************************************
vvvv netmiko_send_config ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ERROR
Traceback (most recent call last):
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\nornir\core\task.py", line 99, in start
r = self.task(self, **self.params)
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\nornir_netmiko\tasks\netmiko_send_config.py", line
28, in netmiko_send_config
net_connect = task.host.get_connection(CONNECTION_NAME, task.nornir.config)
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\nornir\core\inventory.py", line 494, in get_connection
self.open_connection(
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\nornir\core\inventory.py", line 546, in open_connection
conn_obj.open(
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\nornir_netmiko\connections\netmiko.py", line 59, in open
connection = ConnectHandler(**parameters)
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\netmiko\ssh_dispatcher.py", line 351, in ConnectHandler
return ConnectionClass(*args, **kwargs)
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\netmiko\base_connection.py", line 434, in init
self._open()
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\netmiko\base_connection.py", line 439, in _open
self.establish_connection()
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\netmiko\base_connection.py", line 1080, in establish_connection
self.remote_conn = self.remote_conn_pre.invoke_shell(
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\paramiko\client.py", line 547, in invoke_shell
chan = self._transport.open_session()
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\paramiko\transport.py", line 920, in open_session
return self.open_channel(
File "C:\Users\ShibuJ\Documents\python3-venv1\p3-project\lib\site-packages\paramiko\transport.py", line 1051, in open_channel
raise e
paramiko.ssh_exception.SSHException: Unable to open channel.

^^^^ END netmiko_send_config ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PS C:\Users\ShibuJ\Documents\python3-venv1\p3-project>

Let me know if anyother logs or output is needed. I am using nornir with netmiko for my code.

Using below version:

netmiko 4.1.1
nornir 3.3.0
nornir-netmiko 0.2.0
nornir-utils 0.2.0
paramiko 2.11.0

Thanks
Shibu

ktbyers · 2022-07-19T16:34:43Z

@shibu503 Have you tried (in your SSH config file)?

ProxyCommand ssh -F C:\Users\ShibuJ.ssh\config -W %h:%p jumphost

Also please post your full SSH config file (after making the above change).

If your SSH config file is large and has other sensitive information in it, then you should make a simplified version of the file that you can test with and share here (also just obscure anything that is confidential).

shibu503 · 2022-07-20T13:23:42Z

Hi Ktbyers,

Below is SSH config file:

# The intermediate server
host 192.168.2.1
  # Use only the key specified in IdentityFile
  IdentitiesOnly yes
  # The SSH key to use to the intermediate server
  IdentityFile ~/.ssh/id_rsa
  User shibuj
  HostName 192.168.2.1

host * !192.168.2.1
  User shibuj
  KexAlgorithms diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
  # -F forces usage of this SSH config file
  ProxyCommand ssh -F C:/Users/ShibuJ/.ssh/config -W %h:%p 192.168.2.1

Also from windows powershell, I can successfully login to remove device via ssh -F C:\Users\ShibuJ.ssh\config x.x.x.x.

Thanks
Shibu John

ktbyers · 2022-07-20T21:32:37Z

And same error using the above (as the previous error)?

shibu503 · 2022-07-21T04:55:35Z

Hi Ktbyers,

Same error, also the I am using user has administrator privileges for the windows pc.

Thanks
Shibu John

youngbobnetwork · 2023-05-05T03:34:47Z

@shibu503 did you ever come to a conclusion on this? A year later I decided to try netmiko again but still hit the wall of I don’t think I can get this to work.

youngbobnetwork mentioned this issue May 5, 2023

in _join_lingering_threads thr.stop_thread() #3195

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand #2695

paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand #2695

arob182 commented Mar 23, 2022 •

edited

ktbyers commented Mar 23, 2022

arob182 commented Mar 23, 2022

ktbyers commented Mar 23, 2022

arob182 commented Mar 23, 2022

arob182 commented Mar 23, 2022

arob182 commented Mar 23, 2022 •

edited

ktbyers commented Mar 23, 2022

arob182 commented Mar 23, 2022

ktbyers commented Mar 23, 2022

deveshmanish commented Jun 14, 2022

shibu503 commented Jul 17, 2022 •

edited

ktbyers commented Jul 18, 2022

shibu503 commented Jul 18, 2022

ktbyers commented Jul 19, 2022

shibu503 commented Jul 20, 2022 •

edited by ktbyers

ktbyers commented Jul 20, 2022

shibu503 commented Jul 21, 2022

youngbobnetwork commented May 5, 2023

paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand #2695

paramiko.ssh_exception.ProxyCommandFailure: ProxyCommand #2695

Comments

arob182 commented Mar 23, 2022 • edited

ktbyers commented Mar 23, 2022

arob182 commented Mar 23, 2022

ktbyers commented Mar 23, 2022

arob182 commented Mar 23, 2022

arob182 commented Mar 23, 2022

arob182 commented Mar 23, 2022 • edited

ktbyers commented Mar 23, 2022

arob182 commented Mar 23, 2022

ktbyers commented Mar 23, 2022

deveshmanish commented Jun 14, 2022

shibu503 commented Jul 17, 2022 • edited

ktbyers commented Jul 18, 2022

shibu503 commented Jul 18, 2022

Below is full error:

Using below version:

ktbyers commented Jul 19, 2022

shibu503 commented Jul 20, 2022 • edited by ktbyers

ktbyers commented Jul 20, 2022

shibu503 commented Jul 21, 2022

youngbobnetwork commented May 5, 2023

arob182 commented Mar 23, 2022 •

edited

arob182 commented Mar 23, 2022 •

edited

shibu503 commented Jul 17, 2022 •

edited

shibu503 commented Jul 20, 2022 •

edited by ktbyers