SOCKS proxy support, like 'ssh -D'

[Hoeze]

Hi, I'd like to create a SOCKS proxy from an accessible SSH server, in order to pass Python Requests over that server.
Is it possible to do this using paramiko?

[bitprophet]

Thought there was an open feature req for SOCKS but all I see is #508 which mentions it in passing. Let's make it this ticket. Thanks for the report!

I also saw some offhand references to using ProxyCommand support to talk to a SOCKS process; I don't use SOCKS myself so not sure how workable that is, but if you haven't already looked into that I'd suggest doing so.

[Cellebyte]

• is their a way to hack it myself?
• and if so, how can I achieve it.
• need this feature purely dew to firewall restrictions.

[maxim-smirnov]

Any updates here?

[Cellebyte]

Look for your needs.

• Do you really need Dynamic Forward?
• I achieve my solution by using something like ssh -L in paramiko

Problem

• ssl certificate verification does not work with Local Forward because all your requests go to localhost.

[maxim-smirnov]

@Cellebyte Yes, I really need socks functionality...

[utkonos]

Support for dynamic port forwarding would be great!

[obikao]

Same

[ksantr]

We need it

[hamadrehman]

we need it

[linkpwd]

i need it .

[anarchy9388]

I need it!

[cybiere]

Need it :)

[pieterbork]

I NEED IT!

[leakedby]

2 years

[pbelx]

I know man..came here looking for that too..

[eugenweissbart]

Same

[mrluaf]

i need it

[HengyueLi]

ssh -D would be good!~

[bdelano]

heh, I thought I was just missing something as I assumed this was part of the package.... another vote to include. ssh -D

[cunningr]

+1

[cybiere]

For those who have a need for this I adapted the socks5 server from Rushter ( https://github.com/rushter ) to forward the connection through a SSH tunnel using Fabric (https://github.com/fabric/fabric) because I'm lazy (and I use Fabric for my project). So with minor adaptation you can use this :
https://gist.github.com/cybiere/abe5caa3a7504bfd733eb2e5eb829fb1

[Dunedan]

As I've the need for a SOCKS proxy provided by paramiko as well, I started looking into this functionality. I have a working POC and would like to get some feedback on how interfacing with this functionality would be desired by other users. Depending on the outcome, this might allow me to upstream the functionality after polishing it (however no promises for that yet).

So here is how using the SOCKS proxy looks like with my current POC:

import paramiko
import requests

ssh_client = paramiko.SSHClient()
…
socks_proxy = ssh_client.open_socks_proxy(bind_address="127.0.0.1", port=1080)

proxies = {
            'http': 'socks5://127.0.0.1:1080',
            'https': 'socks5://127.0.0.1:1080',
        }
session = requests.Session()
session.proxies.update(proxies)
response = session.get("https://example.tld/")

socks_proxy.close()

ssh_client.close()

Calling ssh_client.open_socks_proxy() instanciates a SOCKS server which binds to the given address and port. The resulting socket accepts SOCKS connections, which get then unpacked by the SOCKS server and forwarded over a direct-tcpip channel per request to the desired destination. That's essentially the same as OpenSSH implements it.

To use the SOCKS proxy with requests one just has to configure requests to use the socket as SOCKS proxy. Of course the proxy can be used by any other application as well.

Finally socks_proxy.close() shuts down the proxy again. Calling this is optional as the proxy gets stopped whenever the SSH client gets closed too.

What do you think about this? Is that as you imagine SOCKS functionality in paramiko or do you have something different in mind?

[cunningr]

that would work for me! Much more convenient to access APIs behind a jumphost

…

On Wed, 24 Mar 2021 at 12:43, Daniel Roschka ***@***.***> wrote: As I've the need for a SOCKS proxy provided by paramiko as well, I started looking into this functionality. I have a working POC and would like to get some feedback on how interfacing with this functionality would be desired by other users. Depending on the outcome, this might allow me to upstream the functionality after polishing it (however no promises for that yet). So here is how using the SOCKS proxy looks like with my current POC: import paramiko import requests ssh_client = paramiko.SSHClient() … socks_proxy = ssh_client.open_socks_proxy(bind_address="127.0.0.1", port=1080) proxies = { 'http': 'socks5://127.0.0.1:1080', 'https': 'socks5://127.0.0.1:1080', } session = requests.Session() session.proxies.update(proxies) response = session.get("https://example.tld/") socks_proxy.close() ssh_client.close() Calling ssh_client.open_socks_proxy() instanciates a SOCKS server which binds to the given address and port. The resulting socket accepts SOCKS connections, which get then unpacked by the SOCKS server and forwarded over a direct-tcpip channel per request to the desired destination. That's essentially the same as OpenSSH implements it. To use the SOCKS proxy with requests one just has to configure requests to use the socket as SOCKS proxy. Of course the proxy can be used by any other application as well. Finally socks_proxy.close() shuts down the proxy again. Calling this is optional as the proxy gets stopped whenever the SSH client gets closed too. What do you think about this? Is that as you imagine SOCKS functionality in paramiko or do you have something different in mind? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#955 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHDBWJ3BEBNCA52P6PRZRNDTFHM6PANCNFSM4DKLANVQ> .

[SSgtSpoon]

As I've the need for a SOCKS proxy provided by paramiko as well, I started looking into this functionality. I have a working POC and would like to get some feedback on how interfacing with this functionality would be desired by other users. Depending on the outcome, this might allow me to upstream the functionality after polishing it (however no promises for that yet).

So here is how using the SOCKS proxy looks like with my current POC:

import paramiko
import requests

ssh_client = paramiko.SSHClient()
…
socks_proxy = ssh_client.open_socks_proxy(bind_address="127.0.0.1", port=1080)

proxies = {
            'http': 'socks5://127.0.0.1:1080',
            'https': 'socks5://127.0.0.1:1080',
        }
session = requests.Session()
session.proxies.update(proxies)
response = session.get("https://example.tld/")

socks_proxy.close()

ssh_client.close()

Calling ssh_client.open_socks_proxy() instanciates a SOCKS server which binds to the given address and port. The resulting socket accepts SOCKS connections, which get then unpacked by the SOCKS server and forwarded over a direct-tcpip channel per request to the desired destination. That's essentially the same as OpenSSH implements it.

To use the SOCKS proxy with requests one just has to configure requests to use the socket as SOCKS proxy. Of course the proxy can be used by any other application as well.

Finally socks_proxy.close() shuts down the proxy again. Calling this is optional as the proxy gets stopped whenever the SSH client gets closed too.

What do you think about this? Is that as you imagine SOCKS functionality in paramiko or do you have something different in mind?

I think that would be perfect! Any chance you have a fork of this working that I can access?

[Dunedan]

You can find a fork with working code at https://github.com/yomagroup/paramiko/tree/wip-socks-proxy. Big disclaimer though: This code is not production ready yet and doesn't handle certain cases correctly! Don't use it for anything serious! I'll need some more time to iron out the remaining bugs and to polish it to make it ready for upstreaming. Until I get to that it might take a few weeks, as other topics have priority for me right now. I'll keep you posted about the progress.

[bitprophet]

@Dunedan thanks for taking a stab at this! As noted upthread I've no personal SOCKS experience but when you feel it's ready for review, please do @ me (here or in a new PR, linking back to this ticket/comment) and I'll hopefully find the time to take a gander.

[Dunedan]

@bitprophet Sounds good. I'll do. 👍

If anybody is wondering about the current status: I'm still working on getting the code into an upstreamable state, whenever I find some time. It might still take a few more weeks before I get to a state I'm satisfied with.

[anarchy9388]

@Dunedan Thanks!

[Dunedan]

I've finally found time to put the SOCKS proxy code into a PR, so here it is: #1873
@bitprophet I'm looking forward to your review. 🙂

[FredAmouzgar]

Hi @Dunedan,

I wrote a small python app using this new branch, and it's ok for the most part. It connects to an SSH server and creates a socks proxy. It works fine when I set my OS network proxy setting to manual (Ubuntu 20) using ('localhost', 1080). But when I try to use a PAC file for automatic proxy, it raises the error below. I should mention that the same PAC file is completely functional when I use an 'ssh -D' command. It seems to me that it's unhappy about the struct.unpack on line 143, but I can't spot the issue. Would you please look into this and help me solve it? Many thanks.

Exception happened during processing of request from ('127.0.0.1', 57916)
Traceback (most recent call last):
File "/home/fred/anaconda3/lib/python3.8/socketserver.py", line 650, in process_request_thread
self.finish_request(request, client_address)
File "/home/fred/anaconda3/lib/python3.8/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/home/fred/anaconda3/lib/python3.8/site-packages/paramiko/socks_proxy.py", line 239, in init
super(SOCKS5RequestHandler, self).init(*args, **kwargs)
File "/home/fred/anaconda3/lib/python3.8/socketserver.py", line 720, in init
self.handle()
File "/home/fred/anaconda3/lib/python3.8/site-packages/paramiko/socks_proxy.py", line 257, in handle
version = m.get_char()
File "/home/fred/anaconda3/lib/python3.8/site-packages/paramiko/socks_proxy.py", line 143, in get_char
return byte_ord(struct.unpack("!c", self.get_bytes(1))[0])
struct.error: unpack requires a buffer of 1 bytes

PS: The contents of the PAC file which is accessed via a SimpleHttpServer:
function FindProxyForURL(url, host)
{
return 'SOCKS localhost:1080; DIRECT';
}

Here's my code snippet for firing up the connection:

self.server = paramiko.SSHClient()
self.server.set_missing_host_key_policy(paramiko.AutoAddPolicy())
if using_password:
    self.server.connect(ip, username=user, password=pass, allow_agent=True)
if using_key:
    k = paramiko.RSAKey.from_private_key_file(self.keyfile)
    self.server.connect(ip, username=user, pkey=k, allow_agent=True)
self.server.open_socks_proxy(bind_address="localhost", port=1080)

[FredAmouzgar]

I'm getting another exception that is not affecting the connection, but I thought there might be some value in pointing it out. What do you think is going wrong here?

Secsh channel 29 open FAILED: open failed: Connect failed

Exception happened during processing of request from ('127.0.0.1', 60522)
Traceback (most recent call last):
File "/home/fred/anaconda3/lib/python3.8/socketserver.py", line 650, in process_request_thread
self.finish_request(request, client_address)
File "/home/fred/anaconda3/lib/python3.8/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/home/fred/anaconda3/lib/python3.8/site-packages/paramiko/socks_proxy.py", line 239, in init
super(SOCKS5RequestHandler, self).init(*args, **kwargs)
File "/home/fred/anaconda3/lib/python3.8/socketserver.py", line 720, in init
self.handle()
File "/home/fred/anaconda3/lib/python3.8/site-packages/paramiko/socks_proxy.py", line 318, in handle
channel, af, addr = self._open_channel(af_and_addrs)
File "/home/fred/anaconda3/lib/python3.8/site-packages/paramiko/socks_proxy.py", line 361, in _open_channel
channel = self.server.ssh_transport.open_channel(
File "/home/fred/anaconda3/lib/python3.8/site-packages/paramiko/transport.py", line 1017, in open_channel
raise e
paramiko.ssh_exception.ChannelException: ChannelException(2, 'Connect failed')

[Dunedan]

@FredAmouzgar To debug this, can you please provide the PAC file you're using and a recording of the TCP packets between the browser and the proxy?

[FredAmouzgar]

I guess it'd be easier if I provided my code and you see it for yourself.
Here's its Git repo. It's a simple GUI app. The primary purpose was to access our university server and IP address.

https://github.com/FredAmouzgar/SshTunneller

[Dunedan]

You're using the branch git+https://github.com/yomagroup/paramiko.git@wip-socks-proxy, which is an older branch to test viability of the whole approach. The branch proposed for inclusion in paramiko as referenced in #1873 (yomagroup:add-socks-proxy) includes various improvements related to stability. Can you please try with the add-socks-proxy branch and check whether you get the same problem with it as well?

[FredAmouzgar]

Thanks for your help and your quick reply. I tested this new branch, and it led to the same issues. I should also clarify that I'm not a network guru, and I may have made a mistake in setting up the PAC file through the HttpServer. After all, the SSH proxy is working fine when I manually access it. So, I play around with my code and let you know if I can resolve the issue.