upload.py should use OAuth 2.0

[GoogleCodeExporter]

ClientLogin is deprecated; see 
https://developers.google.com/accounts/docs/AuthForInstalledApps

upload.py should switch to using OAuth 2.0 instead.

Original issue reported on code.google.com by mdemp...@google.com on 2 Jun 2012 at 1:13

[GoogleCodeExporter]

I'd be happy to do this.

Original comment by dhermes@google.com on 22 Jan 2013 at 1:39

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Seems easy enough - 
https://github.com/GoogleCloudPlatform/storage-oauth2-tool-python

Original comment by techtonik@gmail.com on 24 Jan 2013 at 10:53

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Since ClientLogin is deprecated, we should do something until somewhere in 2014.

But I find it quite important that we are able to communicate with the server 
without any additional dependencies other than what comes with the Python 
standard library.

It seems quite common that projects which use Rietveld just place a copy of 
upload.py (eventually with some project-specific customizations) in their 
"tools" folder and it just works without any additional setup. IMO it would be 
great if we could keep this working that way.

Original comment by albrecht.andi on 24 Jan 2013 at 11:26

• Added labels: Type-Task
• Removed labels: Type-Defect

[GoogleCodeExporter]

I am +1 for self-sufficient upload.py

Original comment by techtonik@gmail.com on 24 Jan 2013 at 1:19

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Yes, that's my plan. Most of the niceties of google-api-python-client can be 
stripped away.

Original comment by dhermes@google.com on 24 Jan 2013 at 5:39

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

The server side part has begun in:
https://codereview.appspot.com/7326049

Original comment by dhermes@google.com on 15 Feb 2013 at 10:09

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

RE: Andi Albrecht comment about adding a copy of upload.py, it's fairly easy to 
distribute an entire package as an executable by zipping up a folder, so long 
as that folder contains a __main__.py file.

Original comment by dhermes@google.com on 15 Feb 2013 at 10:12

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Isn't this now Fixed?

Original comment by kpreid.switchb.org on 10 May 2013 at 12:45

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Indeed. w00t!

Original comment by dhermes@google.com on 10 May 2013 at 1:47

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by albrecht.andi on 10 May 2013 at 4:10

• Changed state: Fixed
• Added labels: ****
• Removed labels: ****

[cedk]

This is not fixed as Google has stopped ClientLogin but upload.py still use it.
See https://developers.google.com/identity/protocols/AuthForInstalledApps

[cedk]

Indeed it is available with --oauth2 option.
My bad.

[johnousterhout]

I am not able to get the --oauth2 option to work. I also specify --no_oauth2_webbrowser to upload.py, which results in a message of the following form:

Go to the following link in your browser:
https://cs190codereview2.appspot.com/get-access-token

When I do this, I get an error page with the following information:

Error: invalid_client
The OAuth client was not found.
Request Details
scope=https://www.googleapis.com/auth/userinfo.email
response_type=code
access_type=offline
redirect_uri=https://cs190codereview2.appspot.com/oauth2callback
approval_prompt=force
state=https://cs190codereview2.appspot.com/get-access-token:xTUPAJs65ATBnlZTsSJqSjoxNDMyODM0OTYx
client_id=None

I see that there is no client_id in this request, so perhaps that's the problem, but I have created a client ID in the Google Developers Console for this project. I've also specified a product name in the Consent Screen form, as recommended by various StackOverflow discussions. Still no luck, however.

Any suggestions for what I might be doing wrong?

[cedk]

On 28 May 10:54, John Ousterhout wrote:

I am not able to get the --oauth2 option to work. I also specify --no_oauth2_webbrowser to upload.py, which results in a message of the following form:

Go to the following link in your browser:
https://cs190codereview2.appspot.com/get-access-token

When I do this, I get an error page with the following information:

Error: invalid_client
The OAuth client was not found.
Request Details
scope=https://www.googleapis.com/auth/userinfo.email
response_type=code
access_type=offline
redirect_uri=https://cs190codereview2.appspot.com/oauth2callback
approval_prompt=force
state=https://cs190codereview2.appspot.com/get-access-token:xTUPAJs65ATBnlZTsSJqSjoxNDMyODM0OTYx
client_id=None

I see that there is no client_id in this request, so perhaps that's the problem, but I have created a client ID in the Google Developers Console for this project. I've also specified a product name in the Consent Screen form, as recommended by various StackOverflow discussions. Still no luck, however.

Any suggestions for what I might be doing wrong?

I go similar issue. The SecretKey Model was not created the first time I
set client/secret with the form. So I retry many times until I saw the
Model in the datastore viewer.

Cédric Krier - B2CK SPRL
Email/Jabber: cedric.krier@b2ck.com
Tel: +32 472 54 46 59
Website: http://www.b2ck.com/

[skoryky]

We're having the same issue. There wasn't any feedback when setting the client ID and secret, so it's unclear if it took.

[rhinon]

Thanks @cedk !

@johnousterhout
We had the same issue and were able to resolve it by just repeatedly setting the client id and secret (https://yoursubdomain.appspot.com/restricted/set-client-id-and-secret) until a SecretKey entity kind showed up in the Datastore (Storage > Cloud Datastore > Query) in the Google Developers Console.

Not sure what exactly the issue is, but it took me about 10 tries.

[zombieland]

client ID and secret is not showing up in the datastore. Is there a way to manually create an entity in datastore? ... what's the proper syntax to insert a row/entity in datastore ?

[johnousterhout]

Sorry, but I'm a little confused: I created a client id from the Google Developers Console (APIs & Auth -> Credentials). It looks like your suggesting doing it from a page in the code review tool itself? Any constraints on the values for the client id and secret? I've tried this a couple of times, but so far I'm not seeing anything new showing up in (Storage -> Cloud Datastore -> Query).

[rhinon]

I mean take the client id and secret that you generated in the developer console and continually try to submit them at this path: /restricted/set-client-id-and-secret

Until you see the SecretKey entity show up in your datastore.

I noticed that our SecretKey entity had additional_client_ids (one of the attempts I duplicated the client-id in the additional client ids field), so maybe try that too?

[cedk]

On 28 May 18:55, Ryan wrote:

I noticed that the our SecretKey entity had additional_client_ids (one of the attempts I duplicated the client-id in the additional client ids field), so maybe try that too?

I think I did the same.
Maybe the StringListField doesn't validate properly if it is empty.

Cédric Krier - B2CK SPRL
Email/Jabber: cedric.krier@b2ck.com
Tel: +32 472 54 46 59
Website: http://www.b2ck.com/

[johnousterhout]

Thanks for the additional information; I'm running now! I only had to submit the set-client-id-and-secret form once, and no SecretEntity key appeared in my database, but upload.py works. I used the same Client ID in both Client ID fields; I don't know whether that helped. Also, I discovered that I had to create a "Client ID for Web application" for this to work, and I had to specify a Redirect URI of https://myapp.appspot.com/oauth2callback (I initially created a "Client ID for installed application", since that's what upload.py seemed like, but that didn't work.

Anyhow, I'm relieved to have this working (I'm using it for a class project due in 23 hours); thanks for all the help.

[virtualern]

Hi, did any of you have login: required set for the main application in your app.yaml?

- url: /.*
  script: main.application
  login: required

I'm trying to debug an issue and want to know if anyone else has it working.

Thanks,
-Ernie

[lancetarn]

I am struggling with this. Basically, I can generate the auth token, but POST to /upload by upload.py tries four times, then gives up with a 302 response. Details on stackoverflow
At least one other person is feeling this

[mrkschan]

Tried the method on http://stackoverflow.com/a/31866613/433662.

Added the following to app.yaml to let /upload skip user authentication.

+# Upload URLs, use @login_required decorator for user info.
+- url: /upload
+  script: main.application
+
+- url: /\d*/upload_content/.*
+  script: main.application
+
+- url: /\d*/upload_complete/.*
+  script: main.application

Then, added the @login_required decorator to guard the view.

# codereview/views.py
 @deco.require_methods('POST')
+@deco.login_required
 @deco.upload_required
 def upload(request):

Now i can use oauth2 with upload.py.

[virtualern]

@mrkschan: thank you for this!!
We verified that your changes resolve the 404 issues seen by many of our users now that Google is deprecating ClientLogin.

[JKivinen]

Aa, thank you, this worked wonders (though not on my first few attempts, unsure what changed overnight, aside from moving the app.yaml portion to be after the upload.py url definition).

Regardless, this was a lifesaver, so thank you again (as well to tarnation for originally thinking it up).

EDIT: Well, looks like there was at least one more url to be added:

- url: /\d*/upload_patch/.*
  script: main.application

[selurvedu]

I confirm this. Thanks to everyone for the solution. 😄

[byron-hawkins]

Is there a way for users to get a permanent access token, or to refresh the access token without a browser? This is a major obstacle in our workflow. Thanks.

[byron-hawkins]

For anyone looking for an efficient way to refresh the access token:

#!/bin/bash

dst_file="$1"

response=$(elinks -dump 1 https://codereview.appspot.com/get-access-token)
token=$(echo "$response" | tail -1 | awk '{print $1}')

if [ $(echo "$token" | wc -c) -eq 76 ]
then
  echo "Token updated successfully"
  echo "$token" > $dst_file
else
  echo "Failed to update the token. Response: $response"
fi

This uses elinks, which is available in the apt repo for Ubuntu and probably many other *nix. It will require one manual sign-in via elinks https://codereview.appspot.com/get-access-token.