gs4_auth not working when deploying shiny apps

[comicalequation]

I have developed a shiny app that works well in my R console. However, when deploying it to shinyio server, I get the following error:

Error in value[[3L]](cond) : Can't get Google credentials.
Are you running googledrive in a non-interactive session? Consider:
  * `drive_deauth()` to prevent the attempt to get credentials.
  * Call `drive_auth()` directly with all necessary specifics.

I have read the recommended troubleshooting options, found at https://gargle.r-lib.org/articles/non-interactive-auth.html

What I did was first run the app in my console in a non-interactive session and get the token with the following code:

token <- gs4_auth(cache = FALSE)
saveRDS(token, file = "googlesheets_token.rds")

Then I removed this code and added the following to my app:

gs4_auth(token = "googlesheets_token.rds")

According to the gargle link above, providing access to the token file directly should work. However, uploading the app.R file along with the googlesheets_token.rds file both to the shinyio server still keeps producing the same error.

I don't seem to be the only one facing this problem, but I haven't found a solution yet that actually works.

[jennybc]

Can I see the app code anywhere?

Have you checked that it's not a path problem? I.e. file.exists("googlesheets_token.rds") returns TRUE.

Also your two snippets are somewhat contradictory.

The first one is about googledrive.
The second one is about googlesheets4.
So that's another area in which cannot tell what's actually going on.

[jennybc]

Also, if you are going to move an OAuth token this way, the here is the section that vignette that applies:

https://gargle.r-lib.org/articles/non-interactive-auth.html#project-level-oauth-cache

That is how you would get a token, locally, interactively, cached to a specific location within your project (Shiny app, in your case). Then deploy it elsewhere.

---

This is NOT how to do it, which looks a bit more like code from the older googlesheets package:

token <- gs4_auth(cache = FALSE)
saveRDS(token, file = "googlesheets_token.rds")

[blumoestit]

Hi, thanks for providing this package!

I also ran into issues with auth on shinyapps.io, after following the instructions at https://gargle.r-lib.org/articles/non-interactive-auth.html#project-level-oauth-cache

I have credentials stored locally in .secrets after running gs4_auth according to the instructions.
.secrets also gets deployed to shinyapps.io and is accessible there:

list.files(".secrets/") on the server logs the credential file (and there is only one!)

But then when trying to have explicit auth via
gs4_auth(cache = ".secrets", email = TRUE)

or implicit auth by calling

gs4_get() directly, I end up with errors like:

Error in value[[3L]](cond) : Can't get Google credentials.
Are you running googlesheets4 in a non-interactive session? Consider:
  * `gs4_deauth()` to prevent the attempt to get credentials.
  * Call `gs4_auth()` directly with all necessary specifics.
See gargle's "Non-interactive auth" vignette for more details:
https://gargle.r-lib.org/articles/non-interactive-auth.html
Calls: local ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>
Execution halted

Calling gs4_deauth() before gs4_auth() doesn't help.
Furthermore, I also tried the approach using:

options(
  gargle_oauth_cache = ".secrets",
  gargle_oauth_email = TRUE
)

which didn't work either.

All that server side code is run on the top level of app.R, not nested in any functions.

Any hints on how to make this work?

[blumoestit]

I was able to resolve this by creating a service account on the Google developer console and sharing the Google sheets with the service account's email.
After that, gs4_auth(path = "<SERVICE_ACCOUNT_JSON">) works like a charm on shinyapps.io and no other gs4 auth calls are needed.

[comicalequation]

I was able to resolve this by creating a service account on the Google developer console and sharing the Google sheets with the service account's email.
After that, gs4_auth(path = "<SERVICE_ACCOUNT_JSON">) works like a charm on shinyapps.io and no other gs4 auth calls are needed.

Is this process free or is it a paid service? Thank you for your response!

ETA: I looked into Google cloud services, and it appears that credit card information is required even for the free service. I am not comfortable with that. R is free, Googlesheets are free, I am looking for a way to integrate them that does not require my billing information.

Thank you for your solution though! I hope it helps the others.

[jennybc]

I don't think you don't have to provide credit card information to get a service account, at least in general. Have you actually tried it @comicalequation? You talk about "Google cloud services" but that's a very broad term and isn't really what we're talking about here.

I still think embedding a regular OAuth token within .secrets should work, but obviously to be convincing with that, I need to make a demo app. This conversation would progress more quickly if anyone shared actual app code that someone else could test or modify.

[comicalequation]

I am still in the process of trying out your earlier suggestions (it's taking me a while to understand the mechanisms going on in the link you provided earlier). I will let you know the results when I'm done, along with a demo app code. Thank you for the suggestions!

I don't think you don't have to provide credit card information to get a service account, at least in general. Have you actually tried it @comicalequation? You talk about "Google cloud services" but that's a very broad term and isn't really what we're talking about here.

I still think embedding a regular OAuth token within .secrets should work, but obviously to be convincing with that, I need to make a demo app. This conversation would progress more quickly if anyone shared actual app code that someone else could test or modify.

[comicalequation]

@jennybc I tried your suggestions. Now the application deploys but every time I click on Submit, it says "Disconnected from Server-Reload" and does not save the response.

Here is a link to my app:
https://samplesurveyscholar.shinyapps.io/expdep/
(I have a free account, so this might be sleeping)

Here is my code:

library(shiny)
library(shinydashboard)
library(googlesheets4)
library(googledrive)
library(DT)
#drive_auth(cache = ".secrets") #for the first time running the app in R to get the OAuth token
drive_auth(cache = ".secrets", email = TRUE, use_oob = TRUE)
fields <- c("resp")
ui <- fluidPage(
    titlePanel("Experimental deployment app"),
  sidebarLayout(
    sidebarPanel(
      helpText("App here"),
      width=6
    ),
    mainPanel(
      numericInput("resp","Enter your age",25,1,100,1),
      actionButton("submit","Submit",class="btn-success"),
      width=4
    )
  )
)
server <- function(input, output, session) {
  observeEvent(input$submit,{
    sheet_append("XXX",as.data.frame(input$resp)) #XXX is my google sheet id/key 
   }) 
}
# Run the application 
shinyApp(ui = ui, server = server)

I have uploaded the .secrets folder to shiny server during deployment. I have checked the google sheet id, it is correct. It works in my RStudio Console and does not need me to authenticate every time after adding the use_oob=TRUE part, but it's failing in Shinyio server yet again.

Here is my application log, in case you need it:

2020-09-02T03:31:57.684200+00:00 shinyapps[2739445]: 
2020-09-02T03:31:57.684199+00:00 shinyapps[2739445]: The following objects are masked from ‘package:googlesheets4’:
2020-09-02T03:31:57.700304+00:00 shinyapps[2739445]: 
2020-09-02T03:31:57.684201+00:00 shinyapps[2739445]: 
2020-09-02T03:31:57.700652+00:00 shinyapps[2739445]: The following objects are masked from ‘package:shiny’:
2020-09-02T03:31:57.839198+00:00 shinyapps[2739445]: 
2020-09-02T03:31:57.700306+00:00 shinyapps[2739445]: Attaching package: ‘DT’
2020-09-02T03:31:57.826774+00:00 shinyapps[2739445]: To suppress this message, modify your code or options to clearly consent to the use of a cached token.
2020-09-02T03:31:57.826776+00:00 shinyapps[2739445]: https://gargle.r-lib.org/articles/non-interactive-auth.html
2020-09-02T03:31:57.827623+00:00 shinyapps[2739445]: The googledrive package is using a cached token for MYEMAIL@gmail.com.
2020-09-02T03:31:57.700653+00:00 shinyapps[2739445]: 
2020-09-02T03:31:57.839199+00:00 shinyapps[2739445]: Listening on http://127.0.0.1:43712
2020-09-02T03:31:57.700654+00:00 shinyapps[2739445]: 
2020-09-02T03:34:04.561741+00:00 shinyapps[2739445]: Warning: Error in : Can't get Google credentials.
2020-09-02T03:34:04.561744+00:00 shinyapps[2739445]: Are you running googlesheets4 in a non-interactive session? Consider:
2020-09-02T03:34:04.561745+00:00 shinyapps[2739445]:   * `gs4_deauth()` to prevent the attempt to get credentials.
2020-09-02T03:34:04.561745+00:00 shinyapps[2739445]:   * Call `gs4_auth()` directly with all necessary specifics.
2020-09-02T03:34:04.567009+00:00 shinyapps[2739445]:   84: observeEventHandler [/srv/connect/apps/expdep/app.R#32]
2020-09-02T03:34:04.567007+00:00 shinyapps[2739445]:   91: gs4_auth
2020-09-02T03:34:04.567036+00:00 shinyapps[2739445]:    7: connect$retry
2020-09-02T03:34:04.567006+00:00 shinyapps[2739445]:   92: stop
2020-09-02T03:34:04.567009+00:00 shinyapps[2739445]:   86: gs4_get
2020-09-02T03:34:04.567007+00:00 shinyapps[2739445]:   90: gs4_token
2020-09-02T03:34:04.567009+00:00 shinyapps[2739445]:   85: sheet_append
2020-09-02T03:34:04.567034+00:00 shinyapps[2739445]:   13: runApp
2020-09-02T03:34:04.567008+00:00 shinyapps[2739445]:   88: request_generate
2020-09-02T03:34:04.567008+00:00 shinyapps[2739445]:   87: gs4_get_impl_
2020-09-02T03:34:04.567036+00:00 shinyapps[2739445]:    6: eval
2020-09-02T03:34:04.561746+00:00 shinyapps[2739445]: See gargle's "Non-interactive auth" vignette for more details:
2020-09-02T03:34:04.561746+00:00 shinyapps[2739445]: https://gargle.r-lib.org/articles/non-interactive-auth.html
2020-09-02T03:34:04.567035+00:00 shinyapps[2739445]:   12: fn
2020-09-02T03:34:04.567037+00:00 shinyapps[2739445]:    5: eval
2020-09-02T03:37:56.783835+00:00 shinyapps[2739445]: Warning: Error in : Can't get Google credentials.
2020-09-02T03:37:56.783838+00:00 shinyapps[2739445]:   * `gs4_deauth()` to prevent the attempt to get credentials.
2020-09-02T03:37:56.783838+00:00 shinyapps[2739445]:   * Call `gs4_auth()` directly with all necessary specifics.
2020-09-02T03:37:56.783839+00:00 shinyapps[2739445]: See gargle's "Non-interactive auth" vignette for more details:
2020-09-02T03:37:56.783837+00:00 shinyapps[2739445]: Are you running googlesheets4 in a non-interactive session? Consider:
2020-09-02T03:37:56.787574+00:00 shinyapps[2739445]:   87: gs4_get_impl_
2020-09-02T03:37:56.787574+00:00 shinyapps[2739445]:   86: gs4_get
2020-09-02T03:37:56.787574+00:00 shinyapps[2739445]:   85: sheet_append
2020-09-02T03:37:56.787575+00:00 shinyapps[2739445]:   84: observeEventHandler [/srv/connect/apps/expdep/app.R#32]
2020-09-02T03:37:56.787575+00:00 shinyapps[2739445]:   13: runApp
2020-09-02T03:37:56.787575+00:00 shinyapps[2739445]:   12: fn
2020-09-02T03:37:56.787576+00:00 shinyapps[2739445]:    7: connect$retry
2020-09-02T03:37:56.787600+00:00 shinyapps[2739445]:    6: eval
2020-09-02T03:37:56.787601+00:00 shinyapps[2739445]:    5: eval
2020-09-02T03:37:56.783839+00:00 shinyapps[2739445]: https://gargle.r-lib.org/articles/non-interactive-auth.html
2020-09-02T03:37:56.787571+00:00 shinyapps[2739445]:   92: stop
2020-09-02T03:37:56.787572+00:00 shinyapps[2739445]:   91: gs4_auth
2020-09-02T03:37:56.787573+00:00 shinyapps[2739445]:   88: request_generate
2020-09-02T03:37:56.787573+00:00 shinyapps[2739445]:   90: gs4_token

I tried replacing drive_auth with gs4_auth and it worked for me! Thank you! This is the final fix:

gs4_auth(cache = ".secrets") #for the first time running the app in R to get the OAuth token
gs4_auth(cache = ".secrets", email = TRUE, use_oob = TRUE)

[phalteman]

I was able to resolve this by creating a service account on the Google developer console and sharing the Google sheets with the service account's email.
After that, gs4_auth(path = "<SERVICE_ACCOUNT_JSON">) works like a charm on shinyapps.io and no other gs4 auth calls are needed.

I've followed all of these same steps with no luck - still getting the same errors you describe in your first post.

To be clear, are you providing the file path to your .json file (ending with something like "project-name-12345678abc1.json")? Or providing the entire json string itself? Or some part of it (e.g., the private_key_id)?

[comicalequation]

I was able to resolve this by creating a service account on the Google developer console and sharing the Google sheets with the service account's email.
After that, gs4_auth(path = "<SERVICE_ACCOUNT_JSON">) works like a charm on shinyapps.io and no other gs4 auth calls are needed.

I've followed all of these same steps with no luck - still getting the same errors you describe in your first post.

To be clear, are you providing the file path to your .json file (ending with something like "project-name-12345678abc1.json")? Or providing the entire json string itself? Or some part of it (e.g., the private_key_id)?

Have you tried my fix?

gs4_auth(cache = ".secrets")

Use this for the first time running the app in R to get the OAuth token. When deploying the app to the server, upload this .secrets folder too. Now replace the code above with the code below and upload the app to the server. Use_oob=TRUE is basically what did the trick for me.

gs4_auth(cache = ".secrets", email = TRUE, use_oob = TRUE)

[phalteman]

gs4_auth(cache = ".secrets")

Use this for the first time running the app in R to get the OAuth token. When deploying the app to the server, upload this .secrets folder too. Now replace the code above with the code below and upload the app to the server. Use_oob=TRUE is basically what did the trick for me.

gs4_auth(cache = ".secrets", email = TRUE, use_oob = TRUE)

This is working, thanks. Since I went to the trouble of creating a service account, I was trying to go that route. But a working app is what counts, so thanks!

[jennybc]

Glad to see progress here!

In terms of the gs4_auth() call in the app code, use_oob = TRUE should not be necessary and, by the principle of minimalism and clarity, it probably should be removed. Out-of-bound auth only matters at initial token acquisition time, which you are doing locally and interactively.

On the Shiny server, you just want to find and refresh an existing token, which is accomplished by cache = ".secrets" (where to look) and email = TRUE (implicit permission to use and refresh any matching token).

[comicalequation]

When I am not using the use_oob=TRUE code in R console, it displays a message asking me to press 1 to use existing token, 0 to acquire new token. Adding that code part suppresses this message in the console. That's why I chose to deploy it to the shiny server with the code. I will try without it too. Thank you!

[jennybc]

I note that we have converged exactly on an approach laid out in Non-interactive auth, specifically in the "Project-level OAuth cache" section recommended for deployed data products:

So I still feel like this is documented. And that the main thing that would help is an example app (tracked in #158) (?).

[comicalequation]

Yes. Your recommendation worked out. Thank you! Issue is resolved!

[JimboMahoney]

I thought I might update this to say that if the scopes are defined in the initial getting of the OAuth token, then they also need to be defined in the server-side code.

e.g.

if (interactive()){
gs4_auth(
  email = "MY_EMAIL",
  path = NULL,
  scopes = c("https://www.googleapis.com/auth/spreadsheets","https://www.googleapis.com/auth/drive"),
  # Get new token if it doesn't exist
  cache = "Local_Directory",
  use_oob = FALSE,
  token = NULL
)
}else{
  gs4_auth(
    email = "MY_EMAIL",
    scopes = c("https://www.googleapis.com/auth/spreadsheets","https://www.googleapis.com/auth/drive"),
    cache = "Server_Directory"
        )
}

Simply having

  gs4_auth(
    email = "MY_EMAIL",
    cache = "Server_Directory"
        )

causes another attempt to get a token, which breaks on the server.

[jennybc]

Yes, token lookup absolutely includes matching on scopes, so if you're using non-default scopes, they must be fully specified.

This is another reason to prefer a service account token over rediscovering a cached oauth token. Quoting from the "non-interactive auth" article:

Arrange for an OAuth token to be re-discovered. This is the least recommended strategy, but it appeals to many users, because it doesn’t require creating a service account. Just remember that the perceived ease of using the token you already have (an OAuth user token) is quickly cancelled out by the greater difficulty of managing such tokens for non-interactive use.

Note that in the particular scope situation above, getting the token with googledrive might make the most sense. In any case, there is no reason get to Sheets scope and Drive scope. Drive scope implies Sheets scope. See https://googlesheets4.tidyverse.org/articles/articles/drive-and-sheets.html.

[ashleyasmus]

Interestingly, I ALSO had to specify use_oob = TRUE when deploying for my app to work - this is the only fix that worked. I followed instructions for "project-level authorization," substituting gs4_auth() for the google drive equivalent.

Use this for the first time running the app in R to get the OAuth token. When deploying the app to the server, upload this .secrets folder too. Now replace the code above with the code below and upload the app to the server. Use_oob=TRUE is basically what did the trick for me.

gs4_auth(cache = ".secrets", email = TRUE, use_oob = TRUE)

[peernisse]

A step that finally worked for me after trying all the things in this thread, and documentation for gargle, googledrive, and googlesheets4, was to go the service account route, yes it is free, and frankly quite useful if you are developing apps. But a step that was required that is not clearly stated anywhere I looked is to go to the Google sheet in question and "Share" it with the email address that is in the JSON key file obtained from the service account. Then it finally worked on my AWS EC2 Ubuntu 18 instance running shiny-server. Finally. This is the last article I read before I got it working: https://robocorp.com/docs/development-guide/google-sheets/interacting-with-google-sheets. I sincerely hope this overlooked step of sharing the JSON key "client_email" address to the Google sheet, giving it Editor permissions, helps someone.