Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ga_account_list() "starred" column duplication error #284

Closed
Rifmaz opened this issue Oct 16, 2019 · 22 comments · Fixed by #290
Closed

ga_account_list() "starred" column duplication error #284

Rifmaz opened this issue Oct 16, 2019 · 22 comments · Fixed by #290
Assignees
@MarkEdmondson1234
Labels
bug need-reproducable-example A bug that needs a reproducable example before being fixed

Comments

@Rifmaz
Copy link

Rifmaz commented Oct 16, 2019 •
edited

What goes wrong

On retrieving the views data via ga_account_list(), the function fails with the error

Error : Column name starred must not be duplicated.
Use .name_repair to specify repair.
Call rlang::last_error() to see a backtrace
Error: API Data failed to parse.
Wrote diagnostic object to 'gar_parse_error.rds', use googleAuthR::gar_debug_parse('gar_parse_error.rds') to
debug the data_parse_function.

Steps to reproduce the problem

The problem appeared when I updated the googleAuthR package to the most recent version (the date I posted this bug). When inspecting the traceback, the response was:

>rlang::last_error()

message: Column name starred must not be duplicated.
Use .name_repair to specify repair.
class: rlang_error
backtrace:

  1. googleAnalyticsR::ga_account_list()
  2. googleAuthR::gar_api_page(acc_sum, page_f = get_attr_nextLink)
  3. googleAuthR:::f()
  4. googleAnalyticsR:::data_parse_function(req$content, ...)
  5. x$items
  6. dplyr::transmute(...)
  7. tidyr::unnest(., cols = webProperties)
  8. dplyr::mutate(...)
  9. dplyr::select(., -kind, -id, -name)
  10. tidyr::unnest(., cols = profiles)
  11. tidyr::unpack(data, !!cols, names_sep = names_sep, names_repair = names_repair)
  12. tibble:::as_tibble.list(out, .name_repair = names_repair)
  13. tibble:::lst_to_tibble(x, .rows, .name_repair, col_lengths(x))
  14. tibble:::set_repaired_names(x, .name_repair)
  15. tibble:::repaired_names(names(x), .name_repair = .name_repair)
  16. tibble:::check_unique(new_name)
    Call rlang::last_trace() to see the full backtrace

Expected output

data.frame of the GA views

Actual output

Error - fails to output anything

Session info

R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252
[2] LC_CTYPE=English_United Kingdom.1252
[3] LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United Kingdom.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] googleAnalyticsR_0.7.0

loaded via a namespace (and not attached):
[1] Rcpp_1.0.2 rstudioapi_0.10 knitr_1.24 magrittr_1.5
[5] tidyselect_0.2.5 R6_2.4.0 rlang_0.4.0 httr_1.4.1
[9] dplyr_0.8.3 tools_3.6.1 xfun_0.8 withr_2.1.2
[13] googleAuthR_1.1.1 askpass_1.1 openssl_1.4.1 yaml_2.2.0
[17] assertthat_0.2.1 digest_0.6.20 lifecycle_0.1.0 tibble_2.1.3
[21] gargle_0.4.0 crayon_1.3.4 purrr_0.3.2 tidyr_1.0.0
[25] vctrs_0.2.0 fs_1.3.1 curl_4.0 zeallot_0.1.0
[29] memoise_1.1.0 glue_1.3.1 compiler_3.6.1 pillar_1.4.2
[33] backports_1.1.4 jsonlite_1.6 pkgconfig_2.0.2
@Rifmaz Rifmaz added the bug label Oct 16, 2019
@MarkEdmondson1234
Copy link
Collaborator

Hmm thats unusual, is it possible to use googleAuthR::gar_debug_parse('gar_parse_error.rds') to get the response/request data?

@Rifmaz
Copy link
Author

Rifmaz commented Oct 18, 2019

Hello. For some reason, googleAuthR::gar_debug_parse('gar_parse_error.rds') didn't work but googleAuthR::gar_debug_parsing('gar_parse_error.rds') did.

I can't share the file publicly (it is also fairly long), could you possibly let me know how I can send it to you via a private channel?

@MarkEdmondson1234
Copy link
Collaborator

MarkEdmondson1234 commented Oct 18, 2019 via email

@Rifmaz
Copy link
Author

Rifmaz commented Oct 18, 2019 •
edited

Apologies Mark, in the meantime I've checked if I'm allowed to send this information through and I've been told no as it's too sensitive.

Just so you know, I've removed all the stars from the various views and ga_account_list() now works. Next step I took is to favourite one of the 360 views and one of the standard GA views (the account contains both). This made the error come up again. So I assume there's a different response structure when getting the "starred" values for 360 vs standard GA.

@MarkEdmondson1234
Copy link
Collaborator

MarkEdmondson1234 commented Oct 18, 2019 via email

@mgahan-he
Copy link

I get around this error by installing an older version of the package:

devtools::install_version("googleAnalyticsR", version = "0.5.0", repos = "http://cran.us.r-project.org")

@MarkEdmondson1234
Copy link
Collaborator

I tested it a bit, but I don't think its to do with GA360 as the response is the same, but perhaps if you have many accounts then if the first API page has no starred column, but the second page does, then it creates an uneven bind rows between the two. Is that possible in your case?

@MarkEdmondson1234 MarkEdmondson1234 added the need-reproducable-example A bug that needs a reproducable example before being fixed label Oct 19, 2019
@Rifmaz
Copy link
Author

Rifmaz commented Oct 19, 2019 •
edited

@MarkEdmondson1234 yes, there are a lot of accounts attached to my login and so it is possible to be a pagination issue.

And installing a previous version of the googleAnalyticsR package does solve the problem (I installed version 6). Also, I suspect you've updated of the package in the past few months, as the package I installed a while back was working fine but ga_account_list() broke after I updated it to the most recent version.

@MarkEdmondson1234
Copy link
Collaborator

Yes that function was updated with the latest dplyr/tidyr which is the source of the problem

@MarkEdmondson1234
Copy link
Collaborator

It seems related to this change https://github.com/MarkEdmondson1234/googleAnalyticsR/issues/266

@hidekoji
Copy link
Contributor

I also got the error.

What goes wrong

I was trying to get account information with googleAnalyticsR::ga_account_list()
and it throws below error.

"R error: Error : API Data failed to parse. 
Wrote diagnostic object to 'gar_parse_error.rds', use googleAuthR::gar_debug_parse('gar_parse_error.rds') to
debug the data_parse_function.

And if I look into the gar_parse_error.rds, it seems data_parse_func raises

Column name `starred` must not be duplicated error.

Here are the details.

> options(googleAuthR.verbose=2)
> debugresult$response$data_parse_func(debugresult$response$content)
Error: Column name `starred` must not be duplicated.
Use .name_repair to specify repair.
Call `rlang::last_error()` to see a backtrace
> rlang::last_error()
<error>
message: Column name `starred` must not be duplicated.
Use .name_repair to specify repair.
class:   `rlang_error`
backtrace:
  1. debugresult$response$data_parse_func(debugresult$response$content)
  2. x$items
 10. dplyr::transmute(...)
 11. tidyr::unnest(., cols = webProperties)
 13. dplyr::mutate(...)
 14. dplyr::select(., -kind, -id, -name)
 15. tidyr::unnest(., cols = profiles)
 20. tidyr::unpack(data, !!cols, names_sep = names_sep, names_repair = names_repair)
 21. tibble:::as_tibble.list(out, .name_repair = names_repair)
  1. tibble:::lst_to_tibble(x, .rows, .name_repair, col_lengths(x))
  2. tibble:::set_repaired_names(x, .name_repair)
 10. tibble:::repaired_names(names(x), .name_repair = .name_repair)
 11. tibble:::check_unique(new_name)
Call `rlang::last_trace()` to see the full backtrace

Steps to reproduce the problem

googleAnalyticsR::ga_account_list()

Expected output

Actual output

Before you run your code, please run:

options(googleAuthR.verbose=2) and copy-paste the console output here.

Session Info

Please run sessionInfo() so we can check what versions of packages you have installed

> sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] zoo_1.8-6                 tidyselect_0.2.5          purrr_0.3.2              
 [4] reshape2_1.4.3            gargle_0.4.0              lattice_0.20-38          
 [7] colorspace_1.4-1          vctrs_0.2.0               generics_0.0.2           
[10] yaml_2.2.0                rlang_0.4.0               pillar_1.4.2             
[13] foreign_0.8-71            glue_1.3.1                foreach_1.4.7            
[16] lifecycle_0.1.0           anonymizer_0.2.2          plyr_1.8.4               
[19] stringr_1.4.0             munsell_0.5.0             exploratory_5.4.0.22     
[22] gtable_0.3.0              prediction_0.3.14         zip_2.0.4                
[25] psych_1.8.12              codetools_0.2-16          memoise_1.1.0            
[28] curl_4.2                  parallel_3.6.0            triebeard_0.3.0          
[31] urltools_1.7.3            broom_0.5.2.1             Rcpp_1.0.2               
[34] openssl_1.4.1             backports_1.1.5           scales_1.0.0             
[37] jsonlite_1.6              fs_1.3.1                  googleAuthR_1.1.1      
[40] mnormt_1.5-5              ggplot2_3.2.1             askpass_1.1              
[43] digest_0.6.21             stringi_1.4.3             dplyr_0.8.3              
[46] dtw_1.21-3                grid_3.6.0                googleCloudStorageR_0.5.1
[49] tools_3.6.0               magrittr_1.5              lazyeval_0.2.2           
[52] proxy_0.4-23              tibble_2.1.3              crayon_1.3.4             
[55] tidyr_1.0.0               pkgconfig_2.0.3           zeallot_0.1.0            
[58] data.table_1.12.2         assertthat_0.2.1          httr_1.4.1               
[61] rstudioapi_0.10           iterators_1.0.12          R6_2.4.0                 
[64] nlme_3.1-139              googleAnalyticsR_0.7.0    compiler_3.6.0           
>

@hidekoji
Copy link
Contributor

hidekoji commented Nov 27, 2019 •
edited

If I modify parse_ga_account_summary to exclude starred column with select(-kind, -id, -name, -dplyr::contains("starred")) %>%, it works fine. I'm not sure if this is a correct approach but hope that rings a bell.

#' New parse GA account summary
#' 
#' @param x The account summary items
#' @keywords internal
#' @importFrom dplyr transmute mutate select
#' @importFrom tidyr unnest
#' @importFrom purrr map_if
parse_ga_account_summary <- function(x){

  assert_that(x$kind == "analytics#accountSummaries")
  
  if(x$totalResults == 0){
    myMessage("No results found for username:", x$username, level = 3)
    return(NULL)
  }
  
  # ## hack to get rid of global variables warning
  id <- name <- webProperties <- kind <- profiles <- NULL
  o <- x$items %>%
    transmute(accountId = id,
              accountName = name,
              ## fix bug if webProperties is NULL
              webProperties = purrr::map_if(webProperties, is.null, ~ data.frame())) %>%
    unnest(cols = webProperties) %>% ##unnest webprops
    mutate(webPropertyId = id,
           webPropertyName = name,
           ## fix bug if profiles is NULL
           profiles = purrr::map_if(profiles, is.null, ~ data.frame())) %>%
    select(-kind, -id, -name, -dplyr::contains("starred")) %>%
    unnest(cols = profiles) %>% ## unnest profiles
    mutate(viewId = id,
           viewName = name) %>%
    select(-kind, -id, -name)
  
  attr(o, "nextLink") <- x$nextLink
  
  o
  
}

@MarkEdmondson1234
Copy link
Collaborator

MarkEdmondson1234 commented Nov 27, 2019 via email

@hidekoji hidekoji mentioned this issue Nov 27, 2019
Merged
MarkEdmondson1234 added a commit that referenced this issue Nov 29, 2019
@MarkEdmondson1234
news item for #284
9e28a2c
@j450h1
Copy link
Contributor

j450h1 commented Jan 13, 2020 •
edited

I'm getting this error with an email account that has access to many GA accounts. Looks like the fix that was implemented needs to be updated?

> library(googleAnalyticsR)

> ga_auth(email = '{EMAIL_REDACTED}')
2020-01-13 15:52:56> Default Google Project for googleAnalyticsR is set.  
 This is shared with all googleAnalyticsR users. 
 If making a lot of API calls, please: 
 visit: https://bit.ly/2Evk6hn 
 for instructions on setting your own Google Project 

2020-01-13 15:52:56> Setting client.id from options(googleAuthR.client_id)

> al <- ga_account_list()
Error : Column name `starred` must not be duplicated.
Use .name_repair to specify repair.
Error: API Data failed to parse.  
             Wrote diagnostic object to 'gar_parse_error.rds', use googleAuthR::gar_debug_parse('gar_parse_error.rds') to 
             debug the data_parse_function.

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Catalina 10.15.2

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] googleAnalyticsR_0.7.1

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.3        rstudioapi_0.10   magrittr_1.5      rsthemes_0.0.4   
 [5] tidyselect_0.2.5  R6_2.4.1          rlang_0.4.2       httr_1.4.1       
 [9] dplyr_0.8.3       tools_3.6.1       googleAuthR_1.1.1 fortunes_1.5-4   
[13] assertthat_0.2.1  digest_0.6.23     lifecycle_0.1.0   tibble_2.1.3     
[17] gargle_0.4.0      crayon_1.3.4      purrr_0.3.3       later_1.0.0      
[21] tidyr_1.0.0       vctrs_0.2.1       fs_1.3.1          zeallot_0.1.0    
[25] memoise_1.1.0     glue_1.3.1        compiler_3.6.1    pillar_1.4.3     
[29] backports_1.1.5   jsonlite_1.6      pkgconfig_2.0.3

@MarkEdmondson1234
Copy link
Collaborator

MarkEdmondson1234 commented Jan 14, 2020 via email

@j450h1
Copy link
Contributor

j450h1 commented Jan 14, 2020

@MarkEdmondson1234 I've updated the comment with the sessionInfo(). No, it is currently the CRAN version that is installed. Let me try installing github version.

@j450h1
Copy link
Contributor

j450h1 commented Jan 14, 2020

Still the same error when updating to latest github version.

Now this is the latest sessionInfo():

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Catalina 10.15.2

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

Random number generation:
 RNG:     Mersenne-Twister 
 Normal:  Inversion 
 Sample:  Rounding 
 
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] googleAnalyticsR_0.7.1.9000

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.3        pillar_1.4.3      compiler_3.6.1    later_1.0.0      
 [5] googleAuthR_1.1.1 prettyunits_1.1.0 remotes_2.1.0     tools_3.6.1      
 [9] pkgbuild_1.0.6    zeallot_0.1.0     digest_0.6.23     jsonlite_1.6     
[13] memoise_1.1.0     gargle_0.4.0      tibble_2.1.3      lifecycle_0.1.0  
[17] pkgconfig_2.0.3   rlang_0.4.2       cli_2.0.1         rstudioapi_0.10  
[21] curl_4.3          withr_2.1.2       dplyr_0.8.3       httr_1.4.1       
[25] askpass_1.1       fs_1.3.1          vctrs_0.2.1       rsthemes_0.0.4   
[29] rprojroot_1.3-2   tidyselect_0.2.5  glue_1.3.1        R6_2.4.1         
[33] processx_3.4.1    fansi_0.4.1       tidyr_1.0.0       purrr_0.3.3      
[37] callr_3.4.0       magrittr_1.5      backports_1.1.5   ps_1.3.0         
[41] fortunes_1.5-4    assertthat_0.2.1  openssl_1.4.1     crayon_1.3.4

@MarkEdmondson1234
Copy link
Collaborator

MarkEdmondson1234 commented Jan 14, 2020 via email

@j450h1
Copy link
Contributor

j450h1 commented Jan 14, 2020

Very strange...re-running the exact same code today and it works fine now!

The underlying metadata for the accounts must have changed or access to the "problematic" account(s) was likely removed.

If I see the error re-appear in the future, I will post debug object.

@MarkEdmondson1234
Copy link
Collaborator

MarkEdmondson1234 commented Jan 14, 2020 via email

@adamribaudo
Copy link

adamribaudo commented Apr 18, 2020 •
edited

@MarkEdmondson1234 I'm receiving this error. I have a gar_parse_error.rds I can send over. Should I do so over Slack?

> ga_account_list()

2020-04-18 11:01:31> Request: https://www.googleapis.com/analytics/v3/management/accountSummaries/
Error : Column name `starred` must not be duplicated.
Error: API Data failed to parse.  
             Wrote diagnostic object to 'gar_parse_error.rds', use googleAuthR::gar_debug_parse('gar_parse_error.rds') to 
             debug the data_parse_function.

sessionInfo()

R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18362)

Matrix products: default

Random number generation:
 RNG:     Mersenne-Twister 
 Normal:  Inversion 
 Sample:  Rounding 
 
locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] googleAnalyticsR_0.7.1 googleAuthR_1.1.1      forcats_0.4.0          stringr_1.4.0          dplyr_0.8.5           
 [6] purrr_0.3.4            readr_1.3.1            tidyr_1.0.2            tibble_3.0.0           ggplot2_3.3.0         
[11] tidyverse_1.2.1        arules_1.6-4           Matrix_1.2-17         

loaded via a namespace (and not attached):
 [1] tidyselect_1.0.0 haven_2.1.1      lattice_0.20-38  gargle_0.4.0     colorspace_1.4-1 vctrs_0.2.4      generics_0.0.2  
 [8] rlang_0.4.5      later_1.0.0      pillar_1.4.3     glue_1.4.0       withr_2.1.2      modelr_0.1.5     readxl_1.3.1    
[15] lifecycle_0.2.0  munsell_0.5.0    gtable_0.3.0     cellranger_1.1.0 rvest_0.3.4      memoise_1.1.0    httpuv_1.5.2    
[22] curl_4.3         fansi_0.4.1      broom_0.5.2      Rcpp_1.0.4.6     promises_1.1.0   openssl_1.4.1    scales_1.1.0    
[29] backports_1.1.5  jsonlite_1.6.1   fs_1.4.1         askpass_1.1      hms_0.5.1        digest_0.6.25    stringi_1.4.6   
[36] grid_3.6.1       cli_2.0.2        tools_3.6.1      magrittr_1.5     crayon_1.3.4     pkgconfig_2.0.3  ellipsis_0.3.0  
[43] xml2_1.2.2       lubridate_1.7.4  assertthat_0.2.1 httr_1.4.1       rstudioapi_0.10  R6_2.4.1         nlme_3.1-140    
[50] compiler_3.6.1

@MarkEdmondson1234
Copy link
Collaborator

You need the github version > 0.7.1 - its not on CRAN yet
https://github.com/MarkEdmondson1234/googleAnalyticsR/blob/master/NEWS.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@MarkEdmondson1234 MarkEdmondson1234

Labels
bug need-reproducable-example A bug that needs a reproducable example before being fixed
Projects
None yet
Milestone
No milestone
6 participants
@hidekoji @adamribaudo @MarkEdmondson1234 @j450h1 @Rifmaz @mgahan-he