TokenServiceAccount not working - does not get passed into config() with GET

[makanig]

I filed an earlier issue on oauth_service_token.
After fixing that, I found this other problem.
For this one, I don't quite know the best way to fix this one, and as a workaround I used the query list for the GET() to include the service_token.

Here's the issue:

googleToken <- # I got the token ..TokenServiceAccount$new
> req <- GET(surl, config(token=googleToken)) # this seems to be the documented way
Error: is.request(y) is not TRUE
> req <- GET(surl, token=googleToken)  # this does not pass the token into the GET 
> content(req)
...
$error$errors[[1]]$message
[1] "Login Required"

[hadley]

Could you please provide a reproducible exampe?

[MarkEdmondson1234]

I have one here which relates to the two linked issues when working with service tokens, I think the below should result in the same response, but does not.

You need to put in your own Google project secrets, JSON files, tokens etc.:

## example with URL shortner API
SCOPE <- "https://www.googleapis.com/auth/urlshortener"
JSON_SECRET <- "yourJSONsecretfile.json" ## for service auth
PROJECT_WEBAPP_KEY <- "xxxxx" ## for OAuth2 auth
PROJECT_WEBAPP_SECRET <- "dddddd" ## for OAuth2 auth

oauth2_token <- function(){
  endpoint <- httr::oauth_endpoints("google")

  app <- httr::oauth_app("google", 
                         key = PROJECT_WEBAPP_KEY, 
                         secret = PROJECT_WEBAPP_SECRET)

  google_token <- httr::oauth2.0_token(endpoint = endpoint, 
                                       app = app,
                                       scope = SCOPE, 
                                       cache = ".httr-oauth")

}

service_token <- function(secrets){
  endpoint <- httr::oauth_endpoints("google")
  scope <- SCOPE
  secrets <- jsonlite::fromJSON(secrets)

  ## multiple scopes only work if space separated
  scope <- paste(scope, collapse=" ")

  ## client ID etc. not needed as in JSON secrts
  google_token <- httr::oauth_service_token(endpoint, secrets, scope)
}

#### Examples

## OAuth2 - works
o2_token <- oauth2_token() ## go through authentication flow

arg_list <- list(url = "https://www.googleapis.com/urlshortener/v1/url?shortUrl=https://goo.gl/cFVQbk", 
                 config = httr::config(token = o2_token), 
                 encode = "json")
## returns ok
do.call("GET", args = arg_list, envir = asNamespace("httr"))
> Response [https://www.googleapis.com/urlshortener/v1/url?shortUrl=https%3A%2F%2Fgoo.gl%2FcFVQbk]
> Date: 2015-12-17 19:26
> Status: 200
> Content-Type: application/json; charset=UTF-8
> Size: 120 B
> {
> "kind": "urlshortener#url",
> "id": "http://goo.gl/cFVQbk",
> "longUrl": "http://markedmondson.me/",
> "status": "OK"
> }

## ...but service token not working for same call

service_token <- service_token(JSON_SECRET)  ##
arg_list <- list(url = "https://www.googleapis.com/urlshortener/v1/url?shortUrl=https://goo.gl/cFVQbk", 
                 config = httr::config(token = service_token), 
                 encode = "json")

do.call("GET", args = arg_list, envir = asNamespace("httr"))

> Error: is.request(y) is not TRUE

[nathangoulding]

I'm able to reproduce this as well.

> endpoint <- httr::oauth_endpoints("google")
> secrets <- jsonlite::fromJSON("/path/to/service/token.json")
> scope <- "https://www.googleapis.com/auth/bigquery"
> google_token <- httr::oauth_service_token(endpoint, secrets, scope)
> google_token
<Token>
<oauth_endpoint>
 authorize: https://accounts.google.com/o/oauth2/auth
 access:    https://accounts.google.com/o/oauth2/token
 validate:  https://www.googleapis.com/oauth2/v1/tokeninfo
 revoke:    https://accounts.google.com/o/oauth2/revoke
NULL
<credentials> access_token, token_type, expires_in
---
> GET("https://www.googleapis.com/bigquery/v2/projects/projectId/datasets", config(token = google_token))
Error: is.request(y) is not TRUE

FWIW, this is pretty much a carbon copy from the documented way to use oauth_service_token.

[nathangoulding]

This particular issue seems related to line 71 of R/request.R. https://github.com/hadley/httr/blob/master/R/request.R#L71

I forked this repo, removed the check on y, and it's at least that allows for further debugging.

Doing some further digging, and the ultimate source of the problem with the service account appears to be because the prepopulated oauth_endpoints for Google don't return v4 that's used for the service account authorization. See here: https://developers.google.com/identity/protocols/OAuth2ServiceAccount

It looks like it's expecting the JWT to be submitted to https://www.googleapis.com/oauth2/v4/token in order to get back the bearer token, and of course the precanned endpoints from httr don't point there.

Just setting the access endpoint doesn't seem to produce good results either:

> endpoint <- oauth_endpoint(access = "https://www.googleapis.com/oauth2/v4/token")
Error in oauth_endpoint(access = "https://www.googleapis.com/oauth2/v4/token") :
  argument "authorize" is missing, with no default

I'll be the first to admit I'm not intimately familiar with the httr source code, but this seems to be the underlying source of why service accounts aren't working. I expect @hadley may need to weigh in here.

[nathangoulding]

Quick update: it is getting the correct access token back. Still trying to figure out why it's not being used properly.

[nathangoulding]

@MarkEdmondson1234 I believe this is the proper fix: #313

[MarkEdmondson1234]

Great job @nathangoulding :) I'll wait for it to merge then I can hopefully roll it out into my stuff.