Add MeetUp import automation 🔄 #27

sicktastic · 2017-01-21T09:38:21Z

Because:

Meetup Event importer #24

This commit:

Add meetup.rb script
HTTParty meetup api
Parse data with Jekyll
Option to archive files

Because: + #24 This commit: + Add meetup.rb script + HTTParty meetup api + Parse data with Jekyll + Option to archive files

sicktastic · 2017-01-21T09:57:58Z

Past events: https://api.meetup.com/2/events?key=#{meetup_api_key}&group_urlname=#{meetup_group_url}&sign=true&status=past

joshuawscott · 2017-01-21T16:40:59Z

meetup.rb

+response = HTTParty.get(base_uri)
+body = JSON.parse(response.body)
+
+mu = body["results"][0]


Perhaps we should iterate over the results? Maybe someday we will actually plan multiple events in the future :)

joshuawscott · 2017-01-21T16:41:31Z

Need gems added to Gemfile:

HTTParty
colorize
dotenv

As a note to @philtr
.env file should look like:

MEETUP_API_KEY='abcdef1234567890'

replaced with actual API key from https://secure.meetup.com/meetup_api/key/

One other comment, but otherwise, LGTM

philtr

Hey @antwonlee, thank you so much for doing this! I really appreciate the effort put in here to make this process as simple as possible.

This is a fantastic start; however, I have quite a few concerns with this pull request that I would like to address before merging. For most of them, I left comments scattered throughout the code. Here are a few general concerns that I think would make this script really stellar:

Jekyll Command – I mentioned this in Meetup Event importer #24, but I think it would be real slick if we could make this a jekyll command. Doesn't have to be its own repo at this point, take a look at these files on my personal website to see how a command could be implemented as a gem that could eventually be extracted out:
- jekyll-deploy inside _gems/ folder
- Gemfile points to _gems/ path for the deploy gem
Multiple Events - Like @joshuawscott said, I think it should iterate over all the events returned by the Meetup API, and it should move old events to the _archived_events folder (but not currently upcoming events). Also, it looks like the way it's set up, if there are any events in the _upcoming_events folder already, it won't make it to the logic to create a new event. I don't think that should be what decides what to do.

I'm happy to jump in and help or pair with you on any of this stuff. (Sorry for throwing up on your PR!)

philtr · 2017-01-21T18:18:40Z

meetup.rb

+meetup_group_url = "ny-tech"
+# meetup_group_url = "RubyFTW-Fort-Worth-Ruby-User-Group"
+base_uri = "https://api.meetup.com/2/events?key=#{meetup_api_key}&group_urlname=#{meetup_group_url}&sign=true"
+response = HTTParty.get(base_uri)


If this is all we're using HTTParty for, you could drop the dependency and do this instead:

response = Net::HTTP.get(URI.parse(base_uri))

philtr · 2017-01-21T18:24:21Z

meetup.rb

+Dotenv.load
+meetup_api_key = ENV["MEETUP_API_KEY"]
+
+meetup_group_url = "ny-tech"


This should probably be an environment variable as well.

philtr · 2017-01-21T18:24:40Z

meetup.rb

+    puts "No files were archived.".light_blue + " Do you want to create a new post?\n" + @yes_or_no
+    validate_to_create_new_post
+  else
+    puts "Wrong input... Make sure to type 'Y' or 'N'  😎".red


philtr · 2017-01-21T18:26:55Z

meetup.rb

+@yes_or_no = "[Y]".green + "Yes" + " [N]".red + "No"
+
+def validate_to_archive
+  answer_to_archive = gets.chomp


Do you think it might be possible check to see if the date is in the past instead of asking for user input? Would save a bunch of time, especially if for some reason I haven't run the script in a while and I need to archive a bunch of old meetups.

What date would you prefer to compare it with? Technically, they should all be in the past if the files exist there. Let me know what you are thinking.

🤔 Well, hopefully there is at least one in the future, right? If I run the script and import an event, then run the script again, there will be one in the future. We could just compare w/ the date in the file name so we don't have to parse the frontmatter.

philtr · 2017-01-21T18:30:15Z

meetup.rb

+    puts "Wrong input... Make sure to type 'Y' or 'N'  😎".red
+  end
+end
+def validate_to_create_new_post


This and the function above it are eerily similar. I wonder if you could extract it to something like this instead:

def confirm? answer = gets.chomp case answer when "Y", "y" then true when "N", "n" then false else puts "Wrong input... Make sure to type 'Y' or 'N' 😎".red end end # ... if @directory_file_count >= 1 puts "..." archive_post if confirm? else puts "..." create_post if confirm? end

Way better. Much cleaner for sho.

philtr · 2017-01-21T18:45:14Z

meetup.rb

+
+def create_post
+  content = "---\nlayout: post\ntitle: #{@title}\nlong_date: #{@long_date}\nlink: #{@link}\ntime: #{@time}\nlocation: #{@location}\nlocation_url: #{@location_url}\nrsvp_url: #{@rsvp_url}\n---\n#{@description}"
+  new_event = File.new("#{Dir.pwd}/#{@directory}/#{@file_name_date}-#{@title.downcase!.gsub! /\s+/, '-'}.md", "w")


This will only put dashes in place of spaces, so if you have "Rails & Docker" as your event name, the file name is going to be 2017-09-09-rails-&-docker.md which might cause problems. Maybe try something like this instead?

@title = "Rails & Docker: Extravaganza!" @title.downcase!.gsub!(/[^[:alnum]]/,"-").squeeze!("-").chomp!("-") # => "rails-docker-extravaganza"

Also, do you think we should be doing some kind of check to make sure we don't try to import the same Meetup twice? I suspect this script would either error or overwrite the existing file (which may have been previously modified to look nicer on the website).

Agree, more validations the better.

philtr · 2017-01-21T18:52:20Z

meetup.rb

+  answer_to_archive = gets.chomp
+  if answer_to_archive === "Y" || answer_to_archive === "y"
+    puts "Moving files...".light_blue
+    FileUtils.mv Dir.glob("#{@directory}/*"), @archived_directory


This moves all the files in _upcoming_events to _archived_events, right? Might cause problems if there is still a future event in _upcoming_events.

Yes, as of now all the files go to _archived_events. Even if you iterate through each files in the future, you would probably still need to manually decided since the data is created into Jekyll rather than just parsing the live data from MeetUp. For example, I can think of filtering through status of the event but that will be challenging since you are basically copying and pasting data into Jekyll file oppose to live data. Any thoughts?

sicktastic · 2017-01-21T20:03:13Z

@philtr These are great inputs! I am hoping that we have 10 or more MeetUps planned for this script to work. 😄 I am not that familiar with Jekyll as much but running a jekyll something something command seems more clean for sure since you are already working in that ecosystem.

It was a start. I just took a stab at it last night when I was watching my 👶

It will be fun to do some pairing at the MeetUp with other members to get the script you deserve!

I think coding together and code reviews are best way to learn from each other.

sicktastic · 2017-01-28T17:31:27Z

@philtr Hey, it was a busy work week. I am looking into this again with your preference of making it a Jekyll command. Do you have any resources where I can read about to see how it can be implemented rather than your sample code? I thought perhaps you already did some since you have the experience. I hardly use Jekyll other than making a simple static pages so don't know much in depth.

I think I actually want this script for myself. 😄

philtr · 2017-01-29T03:24:13Z

Hey no worries, there is no rush on this at all 😄

I don't know of any resources except the official documentation, which is not very helpful. I had to stumble through it myself until it finally clicked. I think maybe I have a topic for my first blog post in over 2 years 😆

Like I said, I'd be happy to pair with you on getting it started, but I think we might have difficulties finding a time to do it because 👦 👶 👶 .

philtr · 2017-01-29T03:58:41Z

Oh yes, and I forgot, here is the official library that Jekyll uses under the hood for their commands: https://github.com/jekyll/mercenary

sicktastic · 2017-01-30T00:58:48Z

This is true. 👶 No joke. This script is actually fun little project whenever I have sometime at night when I am watching the 👶 😄

philtr · 2017-08-17T21:36:53Z

@antwonlee Just a heads up that I am closing this PR since #28 added JavaScript Meetup integration, which is honestly probably better, since we won't have to push an update to the website every time we add a new event. I really appreciate the effort you put in here and want to thank you for the time you spent on this!

sicktastic · 2017-08-18T01:28:59Z

@philtr Sicktastic~ Glad you found a solution!

Add meetup import automation 🔄

8705246

Because: + #24 This commit: + Add meetup.rb script + HTTParty meetup api + Parse data with Jekyll + Option to archive files

joshuawscott reviewed Jan 21, 2017

View reviewed changes

philtr requested changes Jan 21, 2017

View reviewed changes

philtr closed this Aug 17, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add MeetUp import automation 🔄 #27

Add MeetUp import automation 🔄 #27

sicktastic commented Jan 21, 2017

sicktastic commented Jan 21, 2017

joshuawscott Jan 21, 2017

joshuawscott commented Jan 21, 2017 •

edited

philtr left a comment •

edited

philtr Jan 21, 2017

sicktastic Jan 21, 2017

philtr Jan 21, 2017

sicktastic Jan 21, 2017

philtr Jan 21, 2017

sicktastic Jan 21, 2017

philtr Jan 21, 2017

sicktastic Jan 21, 2017

philtr Jan 22, 2017

philtr Jan 21, 2017

sicktastic Jan 21, 2017

philtr Jan 21, 2017

philtr Jan 21, 2017

sicktastic Jan 21, 2017

philtr Jan 21, 2017

sicktastic Jan 21, 2017

sicktastic commented Jan 21, 2017

sicktastic commented Jan 28, 2017

philtr commented Jan 29, 2017

philtr commented Jan 29, 2017

sicktastic commented Jan 30, 2017

philtr commented Aug 17, 2017

sicktastic commented Aug 18, 2017

Add MeetUp import automation 🔄 #27

Add MeetUp import automation 🔄 #27

Conversation

sicktastic commented Jan 21, 2017

sicktastic commented Jan 21, 2017

Choose a reason for hiding this comment

joshuawscott commented Jan 21, 2017 • edited

philtr left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sicktastic commented Jan 21, 2017

sicktastic commented Jan 28, 2017

philtr commented Jan 29, 2017

philtr commented Jan 29, 2017

sicktastic commented Jan 30, 2017

philtr commented Aug 17, 2017

sicktastic commented Aug 18, 2017

joshuawscott commented Jan 21, 2017 •

edited

philtr left a comment •

edited