-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Auth in Google Colaboratory notebooks #140
Comments
Following up with this issue, I was previously using jobdiogenes's workaround to access Google Drive from R in Colaboratory, which changed In the newest release though, I've had to also add Excluding either of these workarounds gives the non-interactive error:
There's also a separate, related (?) issue when calling drive_auth with cache = TRUE twice. The first time around, drive_auth works and displays the oauth prompt. But if one calls it again, it seems to not be able to accept user input the second time around:
Here's a minimal working version (that also demos the second issue above): |
This makes sense in light of changes in the most recent gargle release. Just from reading, I don't have immediate insight into your second observation. Perhaps a first-hand experience is necessary. |
A quick note that this issue was also solved via the method linked above using the same process of overwriting is_interactive() |
Note that google is completely shutting down out-of-band auth by the end of this month, after it being deprecated for at least a year. https://developers.google.com/identity/protocols/oauth2/resources/oob-migration |
Addressed in #202 |
I just went to re-familiarize myself with the Colab experience and it seems like newly created notebooks can't use R. Do I have that right @craigcitro (or anyone else who knows)? I have some pre-existing notebooks that use R that seem to still be executable. |
It is still possible to create a new Colab notebook with R, using the following link: https://colab.research.google.com/#create=true&language=r I am having authentication issues though, and jobdiogenes's method of overwriting is_interactive() is not working for me. What is your current recommendation? |
I'm not sure I have one. But hopefully I can use your tip to get back into position to experiment there and that will lead to some progress. Thanks! (I'm surprised that the httr patch no longer works, so I would consider trying that again, in case you did something like forget to restart R. Also, make sure you're using current gargle; it recently had a release.) |
It may be version installed on Colab is older since it follows Anaconda package schedules. Would check if you can install latest version. |
Update: I did the httr patch before installing and loading googledrive, and this helped, to a point. When I run However, once I click the link, I get the "Access blocked: Tidyverse API Packages’s request is invalid" 400 error due to the changes in OOB. I confirmed that the gargle version is 1.3.0. It seems like the conventional OOB flow is being triggered rather than the new pseudo-OOB. I'm guessing the issue is that gargle should be using a "web" client for Colab just like on RStudio Server, Posit Cloud, and Posit Workbench, but it's using the "installed" client on Colab? |
I have the same problem. What is the current recommended way to read from google drive in R colab notebooks? |
@jennybc @MarkEdmondson1234 Is there a way to specify the |
You should be able to force the pseudo-oob flow by setting a global option (and many client packages also accept Set the option with code like this, before any gargle usage: options(gargle_oob_default = TRUE) Or you can request it in an explicit auth call, e.g.: googledrive::drive_auth(use_oob = TRUE) |
Trying the following locally in Rstudio:
brings up a browser window with the message:
|
This, or the global option, forces the oob flow but not necessarily the pseudo-oob flow. If I'm reading the code for |
OK yes you folks are right. I will need to develop a way to explicitly request pseudo-oob in this setting. Let me create a PR with some working solution for you to try. Please hold. |
In case anyone here know the answer: Does Colab set an environment variable or provide some other way to detect that code is running in that context? I will also try to discover the answer myself. |
In the meanwhile, I've actually had success using this (maximal) example in Colaboratory, borrowing ideas from jobdiogenes and @leon-seranova: require(devtools)
install_version("httr", version = "1.4.4", repos = "http://cran.us.r-project.org")
install_version("R.utils", version = "2.12.2", repos = "http://cran.us.r-project.org")
install_version("gargle", version = "1.3.0", repos = "http://cran.us.r-project.org")
install_version("googledrive", version = "2.0.0", repos = "http://cran.us.r-project.org")
if (file.exists("/usr/local/lib/python3.8/dist-packages/google/colab/_ipython.py")) {
library(R.utils)
library(httr)
reassignInPackage("is_interactive", pkgName = "httr", function() return(TRUE))
library(gargle)
reassignInPackage("is_rstudio_server", pkgName = "gargle", function() return(TRUE))
} else {
stop("Failed to reassign is_interactive and is_rstudio_server!")
}
library(tidyverse)
library(googledrive)
options(rlang_interactive=TRUE)
drive_auth(use_oob = TRUE, cache = FALSE) |
Thanks @jcccf that's helpful to know, i.e. to confirm exactly where our two blockers are. Perhaps my experimental PR could implement both, to create a working solution for Colab. Not sure I could release such a thing on CRAN (with the httr monkey patch), but let's just get a branch that works, then worry about that. |
The existence of the env var |
Re: detecting Colab: using either Re: package versions: LMK if there's something I can/should update version-wise on the Colab side. 😅 Re: |
I think I will be able to get httr to start using |
I just successfully auth'ed on Colab and listed some Google Drive files. Here's how to install gargle from my branch / draft PR (you could do similar with install.packages("pak")
pak::pak("r-lib/gargle@google-colab")
install.packages("googledrive") Note that I also installed googledrive. With that experimental version of gargle, I can do: library(googledrive)
drive_auth(cache = FALSE)
drive_find(n_max = 5) This also works with I'll be interested to hear if others can replicate my success. |
So if I do the same, but allowing the token to be cached: library(googledrive)
drive_auth(cache = TRUE)
drive_find(n_max = 5) I can restart the runtime, say it's OK to auto-discover an existing token, I can use the cached token to list files again: library(googledrive)
options(gargle_oauth_email = TRUE)
drive_find(n_max = 5) |
This worked for me. I also tried |
The necessary version of httr is now on CRAN. Another thing I want to improve is how cache establishment works on Colab. We can't use
PR where the Lots of things make more sense now that I've (partially) read: |
OK even the interaction around initating the cache or selecting from available user tokens works the same on Colab now as it does, e.g., on a local computer (in the |
@jennybc Matrix products: default locale: attached base packages: loaded via a namespace (and not attached): First of all, I executed the following code that you suggested.
The result is here: ! Using bundled GitHub PAT. Please add your own PAT using gitcreds::gitcreds_set(). ✔ Updated metadata database: 2.71 MB in 6 files. ℹ Updating metadata database ✔ Updating metadata database ... done → Will install 16 packages. → Will install 16 packages. → Will download 15 CRAN packages (6.02 MB). → Will download 15 CRAN packages (6.02 MB). → Will download 1 package with unknown size. → Will download 1 package with unknown size.
ℹ Getting 15 pkgs (6.02 MB) and 1 pkg with unknown size ℹ Getting 15 pkgs (6.02 MB) and 1 pkg with unknown size ✔ Got askpass 1.1 (source) (5.73 kB) ✔ Got askpass 1.1 (source) (5.73 kB) ✔ Got mime 0.12 (source) (12.56 kB) ✔ Got mime 0.12 (source) (12.56 kB) ✔ Got rappdirs 0.3.3 (source) (12.29 kB) ✔ Got rappdirs 0.3.3 (source) (12.29 kB) ✔ Got R6 2.5.1 (source) (63.42 kB) ✔ Got R6 2.5.1 (source) (63.42 kB) ✔ Got cli 3.6.0 (source) (565.15 kB) ✔ Got cli 3.6.0 (source) (565.15 kB) ✔ Got lifecycle 1.0.3 (source) (106.85 kB) ✔ Got lifecycle 1.0.3 (source) (106.85 kB) ✔ Got curl 5.0.0 (source) (682.05 kB) ✔ Got curl 5.0.0 (source) (682.05 kB) ✔ Got sys 3.4.1 (source) (20.13 kB) ✔ Got sys 3.4.1 (source) (20.13 kB) ✔ Got rlang 1.0.6 (source) (742.51 kB) ✔ Got rlang 1.0.6 (source) (742.51 kB) ✔ Got glue 1.6.2 (source) (106.51 kB) ✔ Got glue 1.6.2 (source) (106.51 kB) ✔ Got httr 1.4.5 (source) (160.87 kB) ✔ Got httr 1.4.5 (source) (160.87 kB) ✔ Got withr 2.5.0 (source) (102.09 kB) ✔ Got withr 2.5.0 (source) (102.09 kB) ✔ Got jsonlite 1.8.4 (source) (1.05 MB) ✔ Got jsonlite 1.8.4 (source) (1.05 MB) ✔ Got fs 1.6.1 (source) (1.19 MB) ✔ Got fs 1.6.1 (source) (1.19 MB) ✔ Got gargle 1.3.0.9000 (source) (403.17 kB) ✔ Got gargle 1.3.0.9000 (source) (403.17 kB) ✔ Got openssl 2.0.5 (source) (1.20 MB) ✔ Got openssl 2.0.5 (source) (1.20 MB) ℹ Building cli 3.6.0 ℹ Building cli 3.6.0 ℹ Building curl 5.0.0 ℹ Building curl 5.0.0 ✔ Built curl 5.0.0 (7.8s) ✔ Built curl 5.0.0 (7.8s) ℹ Building fs 1.6.1 ℹ Building fs 1.6.1 ✔ Built cli 3.6.0 (21.2s) ✔ Built cli 3.6.0 (21.2s) ℹ Building glue 1.6.2 ℹ Building glue 1.6.2 ✔ Built glue 1.6.2 (3s) ✔ Built glue 1.6.2 (3s) ℹ Building jsonlite 1.8.4 ℹ Building jsonlite 1.8.4 ✔ Built jsonlite 1.8.4 (9.6s) ✔ Built jsonlite 1.8.4 (9.6s) ℹ Building mime 0.12 ℹ Building mime 0.12 ✔ Built mime 0.12 (2.1s) ✔ Built mime 0.12 (2.1s) ℹ Building R6 2.5.1 ℹ Building R6 2.5.1 ✔ Built R6 2.5.1 (2.8s) ✔ Built R6 2.5.1 (2.8s) ℹ Building rappdirs 0.3.3 ℹ Building rappdirs 0.3.3 ✔ Built rappdirs 0.3.3 (2.8s) ✔ Built rappdirs 0.3.3 (2.8s) ℹ Building rlang 1.0.6 ℹ Building rlang 1.0.6 ✔ Built fs 1.6.1 (54.9s) ✔ Built fs 1.6.1 (54.9s) ℹ Building sys 3.4.1 ℹ Building sys 3.4.1 ✔ Built sys 3.4.1 (2.7s) ✔ Built sys 3.4.1 (2.7s) ℹ Building withr 2.5.0 ℹ Building withr 2.5.0 ✔ Built withr 2.5.0 (4.3s) ✔ Built withr 2.5.0 (4.3s) ✔ Installed cli 3.6.0 (82ms) ✔ Installed cli 3.6.0 (82ms) ✔ Installed curl 5.0.0 (74ms) ✔ Installed curl 5.0.0 (74ms) ✔ Installed fs 1.6.1 (78ms) ✔ Installed fs 1.6.1 (78ms) ✔ Installed glue 1.6.2 (50ms) ✔ Installed glue 1.6.2 (50ms) ✔ Installed jsonlite 1.8.4 (72ms) ✔ Installed jsonlite 1.8.4 (72ms) ✔ Installed mime 0.12 (50ms) ✔ Installed mime 0.12 (50ms) ✔ Installed R6 2.5.1 (40ms) ✔ Installed R6 2.5.1 (40ms) ✔ Installed rappdirs 0.3.3 (44ms) ✔ Installed rappdirs 0.3.3 (44ms) ✔ Installed sys 3.4.1 (41ms) ✔ Installed sys 3.4.1 (41ms) ℹ Building askpass 1.1 ℹ Building askpass 1.1 ✔ Built askpass 1.1 (2s) ✔ Built askpass 1.1 (2s) ✔ Installed askpass 1.1 (43ms) ✔ Installed askpass 1.1 (43ms) ℹ Building openssl 2.0.5 ℹ Building openssl 2.0.5 ✔ Built rlang 1.0.6 (33.4s) ✔ Built rlang 1.0.6 (33.4s) ✔ Installed rlang 1.0.6 (95ms) ✔ Installed rlang 1.0.6 (95ms) ℹ Building lifecycle 1.0.3 ℹ Building lifecycle 1.0.3 ✔ Built lifecycle 1.0.3 (3.3s) ✔ Built lifecycle 1.0.3 (3.3s) ✔ Installed lifecycle 1.0.3 (75ms) ✔ Installed lifecycle 1.0.3 (75ms) ✔ Installed withr 2.5.0 (1.1s) ✔ Installed withr 2.5.0 (1.1s) ✔ Built openssl 2.0.5 (10.3s) ✔ Built openssl 2.0.5 (10.3s) ✔ Installed openssl 2.0.5 (59ms) ✔ Installed openssl 2.0.5 (59ms) ℹ Building httr 1.4.5 ℹ Building httr 1.4.5 ✔ Built httr 1.4.5 (4.5s) ✔ Built httr 1.4.5 (4.5s) ✔ Installed httr 1.4.5 (1.1s) ✔ Installed httr 1.4.5 (1.1s) ℹ Packaging gargle 1.3.0.9000 ℹ Packaging gargle 1.3.0.9000 ✔ Packaged gargle 1.3.0.9000 (577ms) ✔ Packaged gargle 1.3.0.9000 (577ms) ℹ Building gargle 1.3.0.9000 ℹ Building gargle 1.3.0.9000 ✔ Built gargle 1.3.0.9000 (6.1s) ✔ Built gargle 1.3.0.9000 (6.1s) ✔ Installed gargle 1.3.0.9000 (github::r-lib/gargle@bb94a25) (34ms) ✔ Installed gargle 1.3.0.9000 (github::r-lib/gargle@bb94a25) (34ms) ✔ 1 pkg + 15 deps: added 16, dld 16 (NA B) [1m 52.7s] ✔ 1 pkg + 15 deps: added 16, dld 16 (NA B) [1m 52.7s] Installing package into ‘/usr/local/lib/R/site-library’ Next, I executed the following code. The error message is as follows:
However, when I restarted and executed the next code, the authentication went well. |
@jennybc |
Hi, I think I may run into a similar issue within databricks notebook.
|
Please open a new issue if you want to explore the auth situation in a new context, such as Databricks. |
Manually transferring from tidyverse/googledrive#284, since I can't truly do a cross-org transfer.
This is the one about auth in Colaboratory / Jupypter notebooks, where you can't really do normal oob auth, but because
interactive()
returnsFALSE
, the normal oauth dance doesn't work either.See also r-lib/httr#634 for an experiment around that (the
interactive()
issue).The text was updated successfully, but these errors were encountered: