Support HTTP_PROXY when run -U command to find which uploaded packages are outdated

[alexandrul]

pypi-server -U [OPTIONS] [PACKAGES_DIRECTORY] seems to fail due to lack of proxy support. I have tried to set http_proxy but it has no effect. pip is working fine with the proxy.

[edit by @ankostis]: Error was:

proxy support [was: pypi-server -U fails with socket.error: [Errno 10060]]

[schmir]

http://docs.python.org/library/xmlrpclib.html#xmlrpc-client-example shows how to use a custom transport class, which works with a proxy.

do you like to work on a patch?

[alexandrul]

I can try, but I have no prior experience in this area.

[alexandrul]

From b3d9a82e2f7e6202c2d13212fb5b55dc305a9842 Mon Sep 17 00:00:00 2001
From: Eduard-Cristian Stefan <alexandrul.ct@gmail.com>
Date: Wed, 22 Aug 2012 21:56:26 +0300
Subject: [PATCH] Add proxy support (see the docs of urllib.getproxies()).
 Tested with Python 2.7 on Windows.

---
 pypiserver/manage.py | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)

diff --git a/pypiserver/manage.py b/pypiserver/manage.py
index fa30584..2faf229 100644
--- a/pypiserver/manage.py
+++ b/pypiserver/manage.py
@@ -1,11 +1,12 @@

 import sys, os, re
+import httplib, urllib
 from pypiserver import core

 if sys.version_info >= (3, 0):
     from xmlrpc.client import Server
 else:
-    from xmlrpclib import Server
+    from xmlrpclib import Server, Transport

 # --- the following two functions were copied from distribute's pkg_resources module
 component_re = re.compile(r'(\d+ | [a-z]+ | \.| -)', re.VERBOSE)
@@ -62,6 +63,26 @@ def filter_stable_releases(releases):
     return res


+class ProxiedTransport(Transport):
+
+    def set_proxy(self, proxy):
+        self.proxy = proxy
+
+    def make_connection(self, host):
+        self.realhost = host
+        if sys.hexversion < 0x02070000:
+            _http_connection = httplib.HTTP
+        else:
+            _http_connection = httplib.HTTPConnection
+        return _http_connection(self.proxy)
+
+    def send_request(self, connection, handler, request_body):
+        connection.putrequest("POST", 'http://%s%s' % (self.realhost, handler))
+
+    def send_host(self, connection, host):
+        connection.putheader('Host', self.realhost)
+
+
 def find_updates(pkgset, stable_only=True):
     no_releases = set()

@@ -69,7 +90,18 @@ def find_updates(pkgset, stable_only=True):
         sys.stdout.write(s)
         sys.stdout.flush()

-    pypi = Server("http://pypi.python.org/pypi/")
+    http_proxy_url = urllib.getproxies().get("http", "")
+
+    if http_proxy_url:
+        http_proxy_spec = urllib.splithost(urllib.splittype(http_proxy_url)[1])[0]
+
+        p = ProxiedTransport()
+        p.set_proxy(http_proxy_spec)
+
+        pypi = Server("http://pypi.python.org/pypi/", transport=p)
+    else:
+        pypi = Server("http://pypi.python.org/pypi/")
+
     pkgname2latest = {}

     pkgfiles = [pkgfile(x) for x in pkgset.find_packages()]
-- 
1.7.11.msysgit.1

[schmir]

Thanks, I created a feature branch for this issue since it's not ready yet (python 3 support is missing),
see https://github.com/schmir/pypiserver/tree/issue-19-proxy-support

[alexandrul]

Thank you.

[schmir]

I've merged this into master. python 3 support is still missing, i.e. the proxy setting is being ignored on python 3.

[ankostis]

So @schmir the only file involved with this is pypiserver/manage.py, right?
Would you mind describing what problems you bumped into, in python-3?

[schmir]

I don't remember any specific problem. Sorry.

[ankostis]

After searching a couple of days, I'm giving up on this :-(
branch: ankostis/pypiserver@proxy_rpc

Pyhon's XML-RPC support across version is very unstable. This fact, combined with PyPi-server's well-known network problems, makes the time required to ensure a stable and tested implementation disproportionately high compared to the benefits from utilizing an http-proxy when running the -U command.

I note that this check-for-updates-on-all-uploaded-packages command is not necessary for the core service offered by this project, and any dependencies (i.e. the requests library) would make sense if more XML-RPC PyPi-API functionality is planned (which is not).