GAE urlfetch

[jpatel531]

The behaviour of urlfetch will be changed from April 2015. This will need to be taken account of

[jpatel531]

"Dear URL Fetch users,

We’re contacting you because you’re listed as the administrator for your organization’s App Engine application, which uses the URL Fetch service. During the phased rollout of an update to this service, we determined that your application may be affected by the change described below.

Currently, the URL Fetch service preserves your original HTTP method (e.g., GET, POST) when it receives and responds to a 302 Moved Temporarily response. Modern user agents typically issue a GET request in response to a 302. After the update, URL Fetch will only issue a GET request after receiving a 302 response, rather than preserving the original method. This may cause requests to be routed differently and/or return 404s or other errors, and will drop the message body from POST requests.

To avoid being affected by this change, please verify that either:
Your app sends non-GET requests (such as POST) directly to the final URL that will handle the request rather than to one that redirects. Some common services, including App Engine, use 302 to redirect HTTP connections to HTTPS and vice versa, so it’s important to check that your protocol is correct as well. You can use the final_url field of the Response object (or a similar object in other APIs) to determine if your fetches are being redirected.
The destination server sends a 303 or 307 HTTP code, each of which specifies the redirect behavior with respect to HTTP methods, rather than a 302.

We will apply the update to your affected application’s URL Fetch usage on April 25, 2015.

If you have any questions, please contact our support team - even if you do not have a support contract.

Sincerely,
Google Cloud Platform team "

[kongakong]

What is the latest status?

[kongakong]

I am not sure if it is going to be a problem...

My target platform is GAE so I have to use python 2.7.x

When I run python setup.py install, I got this error message

  copying build/lib/pusher/util.py -> build/bdist.macosx-10.10-x86_64/egg/pusher
  byte-compiling build/bdist.macosx-10.10-x86_64/egg/pusher/__init__.py to __init__.pyc
  byte-compiling build/bdist.macosx-10.10-x86_64/egg/pusher/aiohttp.py to aiohttp.pyc
    File "build/bdist.macosx-10.10-x86_64/egg/pusher/aiohttp.py", line 25
      response = yield from aiohttp.request(method, url, params=params, data=data, headers=headers)
                          ^
  SyntaxError: invalid syntax

So it looks like 1.0.0 branch may not be compatible with python 2.7?

[zimbatm]

Hi @kongakong, do you mind testing the "run-with-it" branch ? I'm still working on the library to make a 1.0 release, if we can get GAE support in it it would be great.

[zimbatm]

See #45

[kongakong]

@zimbatm Do not have much luck so far.

Let's see if you have any suggestion.

It is my original version (using pusher_client_python):

    if socket_id:
        params['socket_id'] = socket_id

     p[channel_name].trigger('event', json.dumps(data), **params)

With the run-with-it branch I changed the above to

 p.trigger(channel_name, 'event', json.dumps(data), socket_id=socket_id)

And it throws this exception

ERROR    2015-04-17 01:49:59,497 pusher_utils.py:53] ('Connection aborted.', error(13, 'Permission denied'))
Traceback (most recent call last):
  File "/Users/antkong/dev/zeetings/appengine/zeetings/pusher_utils.py", line 51, in _send_message
    p.trigger(channel_name, 'event', json.dumps(data), socket_id=socket_id)
  File "lib/pusher/http.py", line 22, in __call__
    return self.pusher.http.send_request(self.make_request(*args, **kwargs))
  File "lib/pusher/requests.py", line 28, in send_request
    **self.options
  File "lib/requests/sessions.py", line 464, in request
    resp = self.send(prep, **send_kwargs)
  File "lib/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "lib/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(13, 'Permission denied'))

[zimbatm]

Thanks for testing it out @kongakong.

I've read the original Google announcement. We don't use HTTP redirects at all in our API with means that the current stable client library will keep working after the 25th of April.

'Permission denied' is a system-level error which is usually associated with file access. I think Google is blocking socket connections on the system level. It means that the "requests" library (the default adapter) might not be used on GAE and we need another adapter that uses urlfetch. Here's a related discussion which is already a bit old http://stackoverflow.com/questions/9762685/using-the-requests-python-library-in-google-app-engine

Next week I'm working on a new pusher-js release. After that is done I'll be spending some more time on the python library.

[zimbatm]

According to the stackoverflow link new versions of requests and urllib3 might have fixed the problem so that's one thing you could try. Also if you can list your python version and dependencies version it could be useful when I come around to continue work on the library.

[kongakong]

@zimbatm Thanks a lot for the confirmation! It is a good news.

I actually have some follow-up questions regarding transition from pusher_client_python to pusher-http-python. I don't think github issues is a right forum for this kind of discussion. What is the best way to move this discussion forward?

[zimbatm]

I don't mind keeping the discussion on github. To avoid being too out of topic with this issue you can open a new one with your specific topic in mind. Or otherwise contact support and ask for me so they can route the ticket to me.

[jpatel531]

Hey @kongakong ! I've just tested out the new URLFetchBackend (added today) with GAE and it seems to work fine. It's on branch 1.0.0 - if you have time to check it out I'd like to get your thoughts :)

[zimbatm]

Closing for now, please re-open if any issue arises.