Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

breaking API changes on September 13th #906

Closed
wosym opened this issue Aug 11, 2021 · 24 comments
Closed

breaking API changes on September 13th #906

wosym opened this issue Aug 11, 2021 · 24 comments

Comments

@wosym
Copy link

wosym commented Aug 11, 2021

I received an email from Google today notifying me that my code (that they detected uses the Google Drive API) might require some changes due to a security update they applied.

The email says (among other things):

Items that have a Drive API permission with type=domain or type=anyone, where withLink=true (v2) or allowFileDiscovery=false (v3), will be affected by this security update.

In addition to the item ID, your application may now also need a resource key to access these items. Without a resource key, requests for these items may result in a 404 Not Found error (See below for details). Note that access to items that are directly shared with the user or group are not affected.

Since gspread abstracts the Google API, my guess is that the changes will need to be made in gspread, and not in my own code. Is this assumption correct?

If yes, were you guys aware of this change?

KR
@taewookim
Copy link

was going to post this but since already here, bump

@taewookim
Copy link

One comment i'd like to add. this wording is confusing

`Items that have a Drive API permission with type=domain or type=anyone, where withLink=true (v2) or allowFileDiscovery=false (v3), will be affected by this security update.

In addition to the item ID, your application may now also need a resource key to access these items. Without a resource key, requests for these items may result in a 404 Not Found error (See below for details). Note that access to items that are directly shared with the user or group are not affected.
`

(Screenshot of the email -> https://i.postimg.cc/1zb82Xzn/image.png)

Does this mean this change affects if one of the 3 conditions are met?

  1. type=domain

  2. type=anyone and withLink=true, using v2

  3. allowFileDiscovery=false using v3

And if you're using gspread to SHARE a spreadsheet (instead of user requesting that it be shared), this resource key is irrelevant?

@burnash
Copy link
Owner

burnash commented Aug 12, 2021

Thank you for reporting this. Many thanks for the additional details.

I haven't yet received the email from Google regarding the upcoming changes.

Open for a PR with a fix for this change.

@taewookim
Copy link

Full email with links

Hello Google Drive Developer,

We have identified you as a Developer who has used the Drive API in the last 30 days. We are writing to let you know that on September 13, 2021, Drive will apply a security update that will change the links used to share some files, and may lead to some new file access requests. Access to files won’t change for people who have already viewed or modified these files.

Please update your code as detailed below before September 13, 2021, to avoid failing requests.

What do I need to know?
Items that have a Drive API permission with type=domain or type=anyone, where withLink=true (v2) or allowFileDiscovery=false (v3), will be affected by this security update.

In addition to the item ID, your application may now also need a resource key to access these items. Without a resource key, requests for these items may result in a 404 Not Found error (See below for details). Note that access to items that are directly shared with the user or group are not affected.

Will this change affect me?
If your application uses the Drive API to access files which have been shared with a user through link sharing, your application may be affected by this change.

What do I need to do?
To avoid errors accessing files, you must update your code for accessing files to include the appropriate resource keys. Details on how to do this for each of the affected Drive APIs is included below:

Changes to the Drive API
The resource key of an item is returned on the resourceKey field of the file metadata in the Drive API response.

If the file is a shortcut file, then the resource key for the target of the shortcut can be read from the shortcutDetails.targetResourceKey field of the same resource.
URL type fields such as exportLinks, webContentLink, and webViewLink will include the resourceKey.
Requests to the Drive API can specify one or more resource keys with the X-Goog-Drive-Resource-Keys HTTP request header.
Learn more about this change from the Drive API guide.

Changes to Apps Script
The DriveApp from Apps Script has been updated to return the resource key of a file or folder with the getResourceKey method.

Note: When fetching a file or folder, the resource key can be specified on the getFileByIdAndResourceKey or getFolderByIdAndResourceKey methods.
Changes to Drive UI Integrations
If your application is integrated with the Drive UI to create or open items, it will receive resource keys when your application is invoked from the Drive UI.

The state information for a New URL will contain folderResourceKey, which is the resource key of the folder where the new item should be created.
The state for an Open URL will contain a mapping of file ID to resource key in the resourceKeys field.
Learn more about integrating with the Drive UI on our website.

Which projects may be affected?
Your projects that have used the Drive API in the last 30 days and may be affected by this change are below:

____ DELETED____

Thanks for choosing Drive API.

Sincerely,

The Google Drive Team

@dethfire-zz
Copy link

Yeah, I got this too. Hopefully, a quick fix or I'm in trouble :)

@ChristopherPeterFrank
Copy link

From our testing this only appears to affect older versions on Python 2.7. Newest version 4.0.1 on Python 3.8.2 is not affected.

@mowliv
Copy link

mowliv commented Aug 12, 2021 via email

@wosym
Copy link
Author

wosym commented Aug 13, 2021 •
edited

From our testing this only appears to affect older versions on Python 2.7. Newest version 4.0.1 on Python 3.8.2 is not affected.

@BEVer69 Might I ask how you were able to tell? The way I understand it, the changes will only be breaking from September 13th onward?

@ChristopherPeterFrank
Copy link

ChristopherPeterFrank commented Aug 13, 2021 via email

@dethfire-zz
Copy link

Does 4.0.1 use the resource key?

@airween
Copy link

airween commented Aug 13, 2021

hi @BEVer69

Yesterday we experienced a 404 error using our old version. I attributed this to the new security changes being implemented.

could you solve this problem? We experienced with same issue here - got 404 Page not found since one day.

@mowliv
Copy link

mowliv commented Aug 14, 2021

hi @BEVer69

Yesterday we experienced a 404 error using our old version. I attributed this to the new security changes being implemented.

could you solve this problem? We experienced with same issue here - got 404 Page not found since one day.

I doubt that is what is causing your issue. Google clearly states the change will take place Sept. 13.

@ChristopherPeterFrank
Copy link

ChristopherPeterFrank commented Aug 14, 2021 via email

@CrabbyPete
Copy link

I am getting the 404, but only on my App Engine code. I run it locally and all is fine, when I deploy it on App Engine I get a 404. Here is what I get http://coastalflyrodders.com

@mowliv
Copy link

mowliv commented Aug 16, 2021

Thank you for reporting this. Many thanks for the additional details.

I haven't yet received the email from Google regarding the upcoming changes.

Open for a PR with a fix for this change.

@burnash - Hi Anton - Just checking in about this issue. Is someone lined up to test/fix it? I'd like to have a solution in place by 9/1 so we have breathing room against the deadline. I imagine others would like that, too.

Thanks!
Michael

@CrabbyPete
Copy link

Just updated requirements.txt and updated, it works fine now

@lavigne958
Copy link
Collaborator

Hi everyone,

I received this email too and I am aware that we'll need to make some changes in gspread very soon in order to support that new security fix from Google.

After reading the whole email, the documentation about sharing files I understand the following points:

  • this only affect user accessing file using either a public link or a domain link (like public link but restricted to your domain)
  • this will require both of us to update our code:
    • gspread will need to extract the new resourceKey from the shared file
    • the user will need to provide that key to Gspread when trying to open a file using public link or domain link.
  • this can be tested as soon as today, the resource key seem to be already provided by the response of fetching Metadata of a file. Anyone can start looking into it.
  • this will only start from September 13th, not before.

@lavigne958
Copy link
Collaborator

Hi @mowliv

I understand everyone using gspread need to have a few days ahead of the deadline to test it but I can't confirm the exact it's going to be released.

What I am most afraid of is:
This is a breaking change, but we no way to fully confirm our fix would work before the 13th of September, there is no brown day planed ahead nor test API URL that we can use for this 🙁

I could come up with something, release it, you could take that release try it but before the 13th this resource key would not have any effect anyway so wouldn't know of you provided the right header, the right value etc....

So in my mind what I can do best is:

  • prepare a fix
  • prepare a test scenario
  • wait for the 13th
  • on the 13th, at the first lights run the scenario
  • if it works make a release right away.

If it doesn't.... Fix it 😁😅

Conclusion:
Gspread might be down for a day or two if trying to reach shared files starting from the 13th.

@mowliv
Copy link

mowliv commented Aug 18, 2021

Thanks very much, @lavigne958! I appreciate your in-depth research and clear explanation. Based on that, it seems our project isn't affected because we don't use public links.

Agreed, it is far from ideal that there is no way to test before the 13th. Thanks to you and the team for gspread awesomeness :)

@mowliv
Copy link

mowliv commented Sep 13, 2021

Today is the day this is supposed to roll out. Does anyone know if it's out yet? Things are working for me.

@lavigne958
Copy link
Collaborator

No time for me to work on it recently.

this is my next priority to look at.

@wosym
Copy link
Author

wosym commented Sep 13, 2021

Same here: things seem to be working like they did yesterday. Is there any way for us to check if the changes were already made live, by Google? When do they usually apply changes? at the end of the day, at the start of the day, or some random time?

@lavigne958
Copy link
Collaborator

Hi everyone, I ran some tests and I don't see any issue opening a spreadsheet using a public link, and when I take a private spreadsheet and share it publicly then I don't get the resourceKey that the e-mail mentions :-|

I don't understand if something has changed or not.

If anyone encounter an issue please post a comment here we'll investigate from there.

@lavigne958
Copy link
Collaborator

Hello everyone, it has been over a month now and no one seem to report any issues with the resourceKey that we can't seem to be able to test and trigger.

Closing this issue.

Anyone facing an issue related to this API change, please comment here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@CrabbyPete @airween @burnash @dethfire-zz @mowliv @taewookim @ChristopherPeterFrank @lavigne958 @wosym