Support clients behind a proxy

[jpht]

Hi,

maybe I am missing something super easy but... I can't manage to log in when standing behind a proxy. I tried to set up environment variables without success. Googling a bit it seems httplib is not handling proxy that well. Maybe that's the issue.

Thanks for this great lib!

[burnash]

gspread does not handle proxies at the moment. Although I'm not planning to implement it I've received an email several hours ago from a guy who has similar issue and considering to implement proxy handling. So stay tuned.

I haven't used proxies myself with gspread, but from what I can see on Stack Overflow, httplib does support proxies.

The issue in this case is with gspread's HTTPSession.request() and the way it performs a request.

[jpht]

Found out that question on stack overflow. I'll also try to work on it. In the meantime I'll find a workaround :)

[xraywu]

You might want to set both http_proxy and https_proxy system variables. I've been doing so and it works.

[twil]

Environment variables don't work for me. Although they work for requests.

@burnash, what do you think of replacing httplib/http with requests?

[SumNeuron]

any update on this?

[jjdevbiz]

+1

[marnunez]

+1

[tg295]

+1

[lavigne958]

After a quick search I found this article on medium.com => here which mention the classic way of proxy handling for command line tools: use the env var http_proxy (only available on Linux and MacOS).

Proxy is a network feature, and specific to the local environment. Proxy is a system setting, so may be in the library used to send HTTP requests we'll find some proxy handling (I will look into it) but for sure it is something you set in your system, in you terminal and one will make it work very very much faster this way than patching GSpread.

[lavigne958]

I just checked the HTTP decorator used by GSpread: https://github.com/googleapis/google-auth-library-python
And it checks for environment variables set for proxies (like: http_proxy, https_proxy, etc)

It should now work, I don't have a setup with a proxy already setup to test it, if anyone is up for testing this, feel free to comment your result 🙂

[tg295]

Could you point me to the decorator please?

[lavigne958]

yep, you can fin dit using the link above to google-auth-library-python
otherwise here is the source code of the actual decorator: AuthorizedSession
and I double checked and the http_proxy env reading seems to be from the requests package.

[tg295]

So the issue I have is that I do not want connection to the proxy across all http sessions, only those where I am making requests outside my internal network. This is why I am looking for a library-specific solution rather than environment-wide. e.g. there are certain APIs that I do not want to connect to via proxy, and some that I do, this being one of them.

[tg295]

I figured out a solution - I discovered the env variable NO_PROXY which I've used to set my organisations internal domains where I do not want to pass through a proxy. I then have HTTP_PROXY and HTTPS_PROXY set to the proxy address which diverts all other traffic through the proxy. @lavigne958 thanks for your help.

[lavigne958]

pleasure. I'm glad it works.

if you are using linux, remember these env variables works for other software/libraries too (it might work as well on MacOS, can't telle about Windows though).

[amzar96]

You might want to set both http_proxy and https_proxy system variables. I've been doing so and it works.

Hey, I already set both env but the issue is still there 😭

[amzar96]

I figured out a solution - I discovered the env variable NO_PROXY which I've used to set my organisations internal domains where I do not want to pass through a proxy. I then have HTTP_PROXY and HTTPS_PROXY set to the proxy address which diverts all other traffic through the proxy. @lavigne958 thanks for your help.

Hi, how do you set HTTP_PROXY? Is it hardcoded in the script or using export?

[lavigne958]

Hi http_proxy is read from the env, so when you run gspread if you pass the env variable or if you export your env variable it will be read and used (this is handled by the library requests which is used to send HTTP requests to Google API).

so it could look like this: http_proxy=http://proxy.my.university.com:3128 python3 update_spreadsheet.py .
here you script using gspread will use the proxy to reach internet.

About the HTTP_PROXY and HTTPS_PROXY it is just the same variable but written in uppercase 😆
it should alway be lowercase though, this is kind of a typo (AFAIK)

[OoiHS]

Hi, this works for me.

import os
os.environ['HTTP_PROXY'] = '192.168.1.1:8080'
os.environ['HTTPS_PROXY'] = '192.168.1.1:8080'

[poofeg]

How to install a proxy for gspread only:

import requests
from requests.adapters import HTTPAdapter
from google.auth.transport.requests import AuthorizedSession, Request

auth_request_session = requests.Session()
auth_request_session.mount("https://", HTTPAdapter(max_retries=3))
session = AuthorizedSession(
    gspread.client.convert_credentials(credentials),
    auth_request=Request(auth_request_session),
)
if proxy_url:
    session.proxies.update({'https': proxy_url})
    auth_request_session.proxies.update({'https': proxy_url})
gspread = gspread.Client(auth=None, session=session)

[Ousman1301]

Hi! Your code did not work. Session and credentials was not defined. Can you demonstrate this?

[poofeg]

Hi! Your code did not work. Session and credentials was not defined. Can you demonstrate this?

@Ousman1301 Was this question for me? What exactly is not working? credentials were created in the standard way: https://docs.gspread.org/en/latest/oauth2.html

[Ousman1301]

Never mind, I just made a mistake! Thank you soooooooooo much!!