Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with authentication with drive_auth #276

Closed
enricodata opened this issue Sep 27, 2019 · 24 comments
Closed

Problem with authentication with drive_auth #276

enricodata opened this issue Sep 27, 2019 · 24 comments

Comments

@enricodata
Copy link

@enricodata enricodata commented Sep 27, 2019

I have the same issue as in #274
and I posted the same issue here as well:
https://community.rstudio.com/t/problem-with-authentication-with-drive-auth-for-googledrive-library/41013

I have some code that it used to run correctly until one week ago but not when I try to get data from googledrive I get the following error:

createTcpServer: address already in use Error: Can't get Google credentials. Are you running googledrive in a non-interactive session? Consider: * drive_deauth() to prevent the attempt to get credentials. * Call drive_auth() directly with all necessary specifics.

Therefore I run the following again:
drive_auth(use_oob=TRUE)
but when I restart the session or knit a rmd document then I get again the above error.

Then I run the following:
options(gargle_quiet = FALSE)
drive_auth(use_oob=TRUE)
and this is the result:

trying token_fetch()
trying credentials_service_account()
Error: Argument 'txt' must be a JSON string, URL or file.
trying credentials_app_default()
trying credentials_gce()
trying credentials_byo_oauth()
Error: inherits(token, "Token2.0") is not TRUE
trying credentials_user_oauth2()
Gargle2.0 initialize
attempt from: googledrive
adding 'userinfo.email' scope
loading token from the cache
no matching token in the cache

putting token into the cache

I also tried the same in Rstudio with a mac and the result is the same.

Do you have any suggestions how to solve it?

@jennybc
Copy link
Member

@jennybc jennybc commented Sep 29, 2019

createTcpServer: address already in use

This sounds characteristic of a zombie process sitting on a port. Read this and try the remedies described there:

https://gargle.r-lib.org/articles/auth-from-web.html#but-i-didnt-need-oob-yesterday

@enricodata
Copy link
Author

@enricodata enricodata commented Sep 30, 2019

yes, it was indeed the case. Thanks!

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 10, 2019

although I killed the process running on the port 1410 multiple times, I am still getting the same error.
Do you have any other suggestions about it?

The problem now is that when I run:
library(googledrive)
drive_auth(use_oob = TRUE)

the file .httr-oauth is not created anymore

@enricodata enricodata reopened this Oct 10, 2019
@jennybc
Copy link
Member

@jennybc jennybc commented Oct 10, 2019

the file .httr-oauth is not created anymore

Current googledrive no longer defaults to caching in .httr-oauth. It stores its tokens, by default, at the user-level. gargle::gargle_oauth_sitrep() will give you an overview, e.g. cache location and a summary of the tokens found there.

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 10, 2019

when I run gargle::gargle_oauth_sitrep() I get:
No gargle OAuth cache has been established.

then I run this:
gargle:::gargle_default_oauth_cache_path()
and the result is:
.../.R/gargle/gargle-oauth
but in the .R folder there is not a gargle folder.

I tried also with googledrive::drive_auth(use_oob = TRUE) but the credentials are not stored.

I am using rstudio server and this is the sessionInfo():

R version 3.6.0 (2019-04-26)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so

Random number generation:
RNG: Mersenne-Twister
Normal: Inversion
Sample: Rounding

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] gargle_0.4.0 knitr_1.25 lubridate_1.7.4
[4] tidyr_1.0.0 blastula_0.2.2 openxlsx_4.1.0.1
[7] jsonlite_1.6 mongolite_2.1.0 readr_1.3.1
[10] slackr_1.4.2 data.table_1.12.2 stringi_1.4.3
[13] stringr_1.4.0 readxl_1.3.1 googledrive_1.0.0.9000
[16] dplyr_0.8.3

loaded via a namespace (and not attached):
[1] zip_2.0.4 Rcpp_1.0.2 cellranger_1.1.0
[4] pillar_1.4.2 compiler_3.6.0 tools_3.6.0
[7] zeallot_0.1.0 digest_0.6.21 lifecycle_0.1.0
[10] tibble_2.1.3 gtable_0.3.0 pkgconfig_2.0.3
[13] rlang_0.4.0 rstudioapi_0.10 xfun_0.9
[16] withr_2.1.2 httr_1.4.1 fs_1.3.1
[19] htmlwidgets_1.4 vctrs_0.2.0 hms_0.5.1
[22] grid_3.6.0 tidyselect_0.2.5 glue_1.3.1
[25] R6_2.4.0 purrr_0.3.2 ggplot2_3.2.1
[28] magrittr_1.5 backports_1.1.5 scales_1.0.0
[31] htmltools_0.3.6.9004 assertthat_0.2.1 colorspace_1.4-1
[34] lazyeval_0.2.2 munsell_0.5.0 crayon_1.3.4

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 10, 2019

when I run gargle::gargle_oauth_sitrep() I get:
No gargle OAuth cache has been established.

then I run this:
gargle:::gargle_default_oauth_cache_path()
and the result is:
.../.R/gargle/gargle-oauth
but in the .R folder there is not a gargle folder.

I tried also with googledrive::drive_auth(use_oob = TRUE) but the credentials are not stored.

All of this is consistent with never having completed a full auth sequence. The cache will be established the first time there is a token to put in it.

I tried also with googledrive::drive_auth(use_oob = TRUE) but the credentials are not stored.

What exactly happens?

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 10, 2019

so, how do I complete the full auth sequence?
I think that until few weeks ago it was working fine with just googledrive::drive_auth(use_oob = TRUE)

If I try to knit the Rmd file it stays still, as waiting for the authentication from the browser. If I run drive_auth(use_oob=TRUE), then I can run the full script but only manually.
I need to run the Rmd file with a cronjob and not manually.

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 10, 2019

I don't know how to help until you give more detail.

When I do oob auth, here's what I see:

> googledrive::drive_auth(use_oob = TRUE)
The googledrive package is requesting access to your Google account. Select a pre-authorised account or enter '0' to obtain a new token. Press Esc/Ctrl + C to abort.

1: some_user@gmail.com
2: some_other_user@rstudio.com

Selection: 0

Now I'm sent to the browser to specify a Google user in a "Choose your account" page. I choose one.

Then, on the next screen, I choose to "Allow" the Tidyverse API Packages to access my account.

Then I get this screen.

oob

I copy that code, as it says, and go back to R. I paste it.

Enter authorization code: <CODE_PASTED_HERS>

I am now logged in:

> drive_user()
Logged in as:
  *  displayName: Jane Doe
  * emailAddress: jane_doe@example.com

How is your experience different from this?

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 11, 2019

It works only partially.

Like for you after running
library(googledrive)
drive_auth(use_oob=TRUE)

I'm sent to the browser to specify a Google user in a "Choose your account" page. I choose one.

Then, on the next screen, I choose to "Allow" the Tidyverse API Packages to access my account.

And I copy and paste the code into the console in Studio.

Then I am authenticated and I can get files from google drive into rstudio.

However, if after I click on knit to run the whole Rmd file then the authentication is lost.
I need to run the Rmd file with a cronjob so the authentication should be kept

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 11, 2019

In the code that is run as a cron job, you need to have drive_auth(email = “BLAH@gmail.com”), so we know which existing token to use. That prevents the attempt to interact with you for auth in a noninteractive session.

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 11, 2019

do you mean to run once drive_auth(use_oob=TRUE) and after to paste in the code the following?
drive_auth(email = "myemail@myemailprovider.com")

I just tried but I get the same issue, the script does not remember the credentials

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 11, 2019

No.

Once, interactively, you do auth. That stores a token on your computer.

In your code, which might run noninteractively, tell googledrive which already-existing token to use.

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 11, 2019

This is not clear to me yet.
I am using Rstudio server, before to store the credentials I just used drive_auth(use_oob=TRUE) and now that is no longer saving them.

What should I do to store the credentials so that the script find them when I knit a Rmd file?

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 11, 2019

... and now that is no longer saving them.

Please describe your exact evidence for this.

I'm beginning to think this would be better off as a thread in https://community.rstudio.com.

I don't think there's a bug.

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 11, 2019

These are the full steps:

  1. I click knit the Rmd file

  2. I get this error:
    createTcpServer: address already in use
    Error: Can't get Google credentials.
    Are you running googledrive in a non-interactive session? Consider:

  1. I kill the script which is using the port 1410

  2. I click again knit the Rmd file

  3. now the script in the console is blocked at the chunk where I try to download the data from google drive.
    I suppose that it is waiting for a manual authentication.

|........... | 18%
label: download data from google drive

  1. I stop the execution of the Rmd file because it is blocked.

  2. I run drive_auth(use_oob=TRUE)

  3. here I am asked to choose a google account --> I give permission to Tidyverse API Packages --> I copy and paste in the console the authorization code

  4. now I am able to access to google drive but if I try to knit the Rmd file then I am back to the step 5.
    Therefore, I am able to get the authorisation manually but it is not permanent and I cannot use it when I knit the whole script

Is it possible that it is a problem of the package version with Rstudio server?
For googledrive I am using the version 1.0.0.9 and
for Rstudio I am using the version 1.2.1565

I already posted it on rstudio community:
https://community.rstudio.com/t/problem-with-authentication-with-drive-auth-for-googledrive-library/41013

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 13, 2019

Once you successfully authenticate once, interactively, can you show me the output of knitting an .Rmd file with this R code in a chunk? Please replace BLAH@gmail.com with whatever email address you use successfully in the authentication step.

I do not understand why the token you obtain interactively, is not found and used when rendering the .Rmd. Perhaps the gargle_oauth_sitrep() call will be informative.

library(googledrive)

gargle::gargle_oauth_sitrep()

drive_auth(email = "BLAH@gmail.com")

drive_user()

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 14, 2019

I tested the code in a clean environment in Rstudio cloud and the result is the same as before.

First I run drive_auth(use_oob=TRUE)

And then when I knit the Rmd file with the following code:

library(googledrive)
listfiles <- drive_ls()
listfiles <- as.data.frame(listfiles)

I get:
Error: Can't get Google credentials.
Are you running googledrive in a non-interactive session? Consider:

  • drive_deauth() to prevent the attempt to get credentials.
  • Call drive_auth() directly with all necessary specifics.
    Execution halted

The same result if I knit the Rmd file with the following code:
library(googledrive)
gargle::gargle_oauth_sitrep()
drive_auth(email = "myemail@gmail.com")
drive_user()

listfiles <- drive_ls()
listfiles <- as.data.frame(listfiles)

Error: Can't get Google credentials.
Are you running googledrive in a non-interactive session? Consider:

  • drive_deauth() to prevent the attempt to get credentials.
  • Call drive_auth() directly with all necessary specifics.
    Execution halted

Maybe you can test it on your side with the following code in a new environment and check if you are able to knit a Rmd file:
library(googledrive)
listfiles <- drive_ls()
listfiles <- as.data.frame(listfiles)

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 14, 2019

This is the first time you've mentioned RStudio Cloud. That is an important detail.

What does gargle::gargle_oauth_sitrep() show in an interactive session, after you've successfully completed auth with drive_auth(email = "myemail@gmail.com")?

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 14, 2019

RStudio Cloud is very project-based vs. user-based. So it could be a problem with the logic for establishing and finding a token cache. I.e. the cache is not getting established, which would explain why a non-interactive render does not have access to any previously acquired tokens.

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 14, 2019

So you can also try the instructions in this vignette that describe how to specify the cache path yourself, i.e. to embed it in a project.

https://gargle.r-lib.org/articles/non-interactive-auth.html#project-level-oauth-cache

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 14, 2019

my working environment is an Rstudio server, not Rstudio cloud. Therefore I need to make it work on Rstudio server.
I tested it in Rstudio cloud to verify if it was a problem due to the Rstudio server that I am using.

Anyway, on Rstudio cloud, if I run gargle::gargle_oauth_sitrep() I get:
No gargle OAuth cache has been established.

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 14, 2019

I suggest you try the instructions above in which you take control of the cache location.

@jennybc
Copy link
Member

@jennybc jennybc commented Oct 14, 2019

I have done all of this successfully on RStudio Server myself now and have verified with others, internally, that it also works for them.

Here's what getting the initial token looks like for me:

> library(googledrive)
> packageVersion("googledrive")
[1] ‘1.0.0> packageVersion("gargle")
[1] ‘0.4.0> drive_auth(use_oob = TRUE)
Is it OK to cache OAuth access credentials in the folder '/usr/home/jenny/.R/gargle/gargle-oauth' between R sessions?

1: Yes
2: No

Selection: 1
Enter authorization code: abcdefghijklmnopqrstuvwxyz

Then I have a .R file with this code in it:

#' ---
#' output: github_document
#' ---

library(googledrive)

drive_auth(email = "jenny@rstudio.com")

gargle::gargle_oauth_sitrep()

And I can render it non-interactively like so:

> callr::r(function() rmarkdown::render("wtf.R"))
[1] "/usr/home/jenny/wtf.md"

And it produces this Markdown:


wtf.R

jenny
2019-10-14

library(googledrive)

drive_auth(email = "jenny@rstudio.com")

gargle::gargle_oauth_sitrep()
## gargle OAuth cache path:
## /usr/home/jenny/.R/gargle/gargle-oauth
## 
## 1 tokens found
## 
## email             app                scopes   hash...   
## _________________ __________________ ________ __________
## jenny@rstudio.com tidyverse-calliope ...drive cd26ed5...

So ... perhaps something is misconfigured with your RStudio Server or something odd about your personal configuration is getting in the way.

@enricodata
Copy link
Author

@enricodata enricodata commented Oct 15, 2019

I tested it on a new clean ubuntu server (18.10) and when I run the last part:
callr::r(function() rmarkdown::render("test_google_drive.R"))

I got the same error:

Error: callr subprocess failed: Can't get Google credentials.
Are you running googledrive in a non-interactive session? Consider:

  • drive_deauth() to prevent the attempt to get credentials.
  • Call drive_auth() directly with all necessary specifics.

However, I made it work again by running just once:
options(gargle_oauth_cache = ".secrets")
gargle::gargle_oauth_cache()
drive_auth(use_oob = TRUE)
list.files(".secrets/")

and by adding in the Rmd file:
options(
gargle_oauth_cache = ".secrets",
gargle_oauth_email = TRUE
)

I wonder what happened because before it was working totally fine just by running once drive_auth(use_oob = TRUE)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants