Use `graphql` API #123

schloerke · 2021-12-28T15:20:48Z

drmowinckels · 2022-01-03T10:36:33Z

Great stuff. Have not looked in detail and tested yet, but have an initial thought by looking at the commits and comments here.

I see currently, we are appending 2 to the new functions, which is fine for now, but perhaps there could be an argument toggle for api_version = c("v3", "gql") in the functions, that would call the relevant version? That could make it easy to switch version when needed now in the beginning while both are supported?

schloerke · 2022-01-03T20:26:42Z

@Athanasiamo I hope to eventually just have the original function names be implemented using the GraphQL queries. I've updated notes above to rename methods like get_events2() to get_events() when ready.

Given meetup.com will drop support for "v3" API sooner than later, I'd like to not add an api_version parameter.

From: https://www.meetup.com/meetup_api/

RESTful API support will stop in winter 2021.

drmowinckels · 2022-01-04T08:32:50Z

That sounds totally reasonable to me, @schloerke ! I hope to get time to do initial tests over the weekend. Great work, and I'm learning a lot by looking at how you're approaching this :)

GregSutcliffe · 2022-02-10T11:49:30Z

How's this looking so far? I was super busy last year, so I missed the ping in #118, but I've just realised the REST API has stopped - I'm getting 401 Unauthorised on my reports since 1st Feb.

I'm willing to pitch in and help, but I'm unsure where to start. Trying to run find_groups2() is currently erroring out for me:

> devtools::load_all()
ℹ Loading meetupr
> find_groups2('Ansible')
Error in gh_process_response(raw) : 
GitHub API error (400): 
Message: 

Errors:
                                                               message
 Variable "$topicCategoryId" of required type "Int!" was not provided.
Type .Last.error.trace to see where the error occurred
Called from: throw(cond)

Any pointers for digging in @schloerke? I'm reasonably familiar with GraphQL (done a fair bit with the GH API), but I haven't yet taken a look at the Meetup query explorer...

benubah · 2022-02-10T15:52:42Z

@GregSutcliffe Try find_groups2('Ansible', topic_category_id = 546)

GregSutcliffe · 2022-02-11T09:58:40Z

@benubah that works, thanks for the pointer. I guess I should go read up on what that parameter does - right now I have 2 calls to find_groups in my code, one for the "Ansible" string, the other for topic "1434182", so that I can catch groups in our field without Ansible in the title. I'm guessing I can't do that anymore, but I'll poke at it.

I can see that get_events2 is working as well (at least from a quick test), and I think I'm only using those two calls, so I can probably get my report back into a working state from here - great work, everyone! I'll be sure to feedback what I find, and if I have time I'll try to help with the remaining pieces ;)

benubah · 2022-02-11T13:56:28Z

@schloerke I tested removing the topicCategoryId variable from the query so that we can just have find_groups2('r-ladies') and it works well and also automatically gives us the value topicCategoryId in the response. With this we don't have to guess what value to pass into topicCategoryId when calling find_groups2().
And we get a little more related records added to what we used to get before.
What do you think?

schloerke · 2022-02-11T15:51:23Z

@benubah I still believe it is useful to have the topic_category_id query parameters. Otherwise it will be very difficult to exclude other topic category ids.

With some testing on the meetup playground, I was able to turn

  $topicCategoryId: Int!
  # into 
  $topicCategoryId: Int = null

... and still have the query work. This would allow for find_groups2(topic_category_id = ) to default to NULL.

benubah · 2022-02-11T18:06:28Z

@schloerke This appears better and works. There seems to be several undocumented stuff around the API. Does this mean we need to provide a list of topicCategoryIds (at least a few) in our documentation for new users to start with. I can't see a list of these ids documented anywhere on the Meetup.com website.

benubah · 2022-02-11T20:20:04Z

I also see that there are many objects and parameters that we need that are hidden in the schema without documentation. Will just take some time to figure out a whole bunch of these hidden things. I'll be taking a look in the next few days.

benubah · 2022-02-14T20:25:49Z

@schloerke Does find_groups2('ladies', topic_category_id = NULL) fail for you. At the Meetup playground, it returns an error, but also returns results.

  Meetup GraphQL API returned errors.
List of 2
 $ :List of 4
  ..$ message   : chr "Unknown time-zone ID: US/None"
  ..$ locations :List of 1
  .. ..$ :List of 2
  .. .. ..$ line  : int 49
  .. .. ..$ column: int 13
  ..$ path      :List of 6
  .. ..$ : chr "keywordSearch"
  .. ..$ : chr "edges"
  .. ..$ : int 0
  .. ..$ : chr "node"
  .. ..$ : chr "result"
  .. ..$ : chr "foundedDate"
  ..$ extensions:List of 2
  .. ..$ code     : chr "INTERNAL_SERVER_ERROR"
  .. ..$ exception:List of 1
  .. .. ..$ message: chr "Unknown time-zone ID: US/None"
 $ :List of 4
  ..$ message   : chr "Unknown time-zone ID: US/None"
  ..$ locations :List of 1
  .. ..$ :List of 2
  .. .. ..$ line  : int 50
  .. .. ..$ column: int 13
  ..$ path      :List of 6
  .. ..$ : chr "keywordSearch"
  .. ..$ : chr "edges"
  .. ..$ : int 0
  .. ..$ : chr "node"
  .. ..$ : chr "result"
  .. ..$ : chr "timezone"
  ..$ extensions:List of 2
  .. ..$ code     : chr "INTERNAL_SERVER_ERROR"
  .. ..$ exception:List of 1```

drmowinckels

I've not yet tested the functions, but have had a peak through the code and made some comments on where I think there could be improvements.

drmowinckels · 2022-02-11T12:33:47Z

.github/workflows/with-auth.yaml

@@ -8,7 +8,7 @@ name: with-auth

 jobs:
  with-auth:
-    runs-on: macOS-latest
+    runs-on: macos-latest


any reason you switched the other workflows to ubuntu, while this remains macos?

drmowinckels · 2022-02-11T12:41:10Z

inst/graphql/find_events.graphql

+    country
+  }
+  << extra_graphql >>
+}


so, if I understand correctly. Most queries to the graphql API will need a similar type of graphql template to retrieve the fields and values?

drmowinckels · 2022-02-17T20:05:10Z

R/graphql.R

+      if (is.null(pb)) {
+        pb <- progress::progress_bar$new(
+          total = total_fn(graphql_res),
+          format = paste0(file, " ", pb_format)


change to paste(file, "pb_format) ?
neater imo

drmowinckels · 2022-02-17T20:06:33Z

R/graphql.R

+        "Authorization" = paste0(
+          .token$auth_token$credentials$token_type,
+          " ",
+          .token$auth_token$credentials$access_token
+        )


change to paste(file, "pb_format) ?
neater imo

drmowinckels · 2022-02-17T20:09:18Z

R/graphql.R

+    country <- get_country(group)
+    group$country_name <-
+      if (length(country) == 0 || nchar(country) == 0) {
+        ""


is it perhaps better to return NA_character rather than empty string?

GregSutcliffe · 2022-02-22T11:24:27Z

Current branch is working for me to generate my reports, thanks to all for the hard work!

Given the Meetup REST API is now dead anyway, does it make sense to rename the methods (s/2//g) and merge what we, and then bring additional PRs for the rest of the calls? A partial library is better than no library at all, I think.

benubah · 2022-02-22T12:16:58Z

Current branch is working for me to generate my reports, thanks to all for the hard work!

@GregSutcliffe were you able to find groups by topic_id?

Given the Meetup REST API is now dead anyway, does it make sense to rename the methods (s/2//g) and merge what we, and > then bring additional PRs for the rest of the calls? A partial library is better than no library at all, I think.

Working on this.

benubah · 2022-03-15T20:04:28Z

Will be continuing the GraphQL updates here: #138

schloerke and others added 9 commits December 28, 2021 09:49

Init graphql code ideas (#120)

0b09f97

Use latest r-lib/actions code for GHA (#121)

b77bcc1

Bump version to 0.2.99.9000 (#122)

25d626a

Cleanup devel warnings (#124)

ea78203

Set up gh for GraphQL calls to meetup.com (#125)

fc7e4c9

Add GraphQL queries for find_groups() (#128)

87c8f55

GraphQL: Flatten all lists into combined names to make a tibble (#130)

82df921

Use location hashing. Implement find_groups2(), get_events2() (#132)

8ac3e97

Delete R/barret.R

1200cc2

ledell requested review from ledell, cvitolo and hfrick January 3, 2022 10:09

ledell added the work in progress label Jan 3, 2022

ledell requested a review from drmowinckels January 3, 2022 10:10

Merge branch 'master' into barret

d44a3c9

GraphQL: Comments and .token parameter (#133)

21247bf

drmowinckels reviewed Feb 17, 2022

View reviewed changes

benubah mentioned this pull request Mar 15, 2022

GraphQL: rewrite remaining functions #138

Merged

6 tasks

ledell merged commit 21247bf into master Apr 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `graphql` API #123

Use `graphql` API #123

schloerke commented Dec 28, 2021 •

edited by benubah

Loading

drmowinckels commented Jan 3, 2022

schloerke commented Jan 3, 2022

drmowinckels commented Jan 4, 2022

GregSutcliffe commented Feb 10, 2022

benubah commented Feb 10, 2022

GregSutcliffe commented Feb 11, 2022

benubah commented Feb 11, 2022 •

edited

Loading

schloerke commented Feb 11, 2022

benubah commented Feb 11, 2022

benubah commented Feb 11, 2022 •

edited

Loading

benubah commented Feb 14, 2022

drmowinckels left a comment

drmowinckels Feb 11, 2022

drmowinckels Feb 11, 2022

drmowinckels Feb 17, 2022

drmowinckels Feb 17, 2022

drmowinckels Feb 17, 2022

GregSutcliffe commented Feb 22, 2022

benubah commented Feb 22, 2022

benubah commented Mar 15, 2022

Use graphql API #123

Use graphql API #123

Conversation

schloerke commented Dec 28, 2021 • edited by benubah Loading

drmowinckels commented Jan 3, 2022

schloerke commented Jan 3, 2022

drmowinckels commented Jan 4, 2022

GregSutcliffe commented Feb 10, 2022

benubah commented Feb 10, 2022

GregSutcliffe commented Feb 11, 2022

benubah commented Feb 11, 2022 • edited Loading

schloerke commented Feb 11, 2022

benubah commented Feb 11, 2022

benubah commented Feb 11, 2022 • edited Loading

benubah commented Feb 14, 2022

drmowinckels left a comment

Choose a reason for hiding this comment

drmowinckels Feb 11, 2022

Choose a reason for hiding this comment

drmowinckels Feb 11, 2022

Choose a reason for hiding this comment

drmowinckels Feb 17, 2022

Choose a reason for hiding this comment

drmowinckels Feb 17, 2022

Choose a reason for hiding this comment

drmowinckels Feb 17, 2022

Choose a reason for hiding this comment

GregSutcliffe commented Feb 22, 2022

benubah commented Feb 22, 2022

benubah commented Mar 15, 2022

Use `graphql` API #123

Use `graphql` API #123

schloerke commented Dec 28, 2021 •

edited by benubah

Loading

benubah commented Feb 11, 2022 •

edited

Loading

benubah commented Feb 11, 2022 •

edited

Loading