Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Download and include images in posterous migrator. #4

Closed
wants to merge 1 commit into from

3 participants

@parkr
Owner

ACHTUNG!! This is being merged into a non-master branch.

In jekyll/jekyll#509, @markhepburn suggested that we import images from Posterous, and supplied the code. Thought it was a cool idea.

I haven't tested this (hadn't heard of Posterous before a couple weeks ago) but I think it worked for Mark.

Here's his original message:

I've added image importing to the posterous migrator; the other media types could be added similarly but I thought I'd push this out there first. (I'm not actually expecting this to go in as-is -- I'm new to both ruby and jekyll for a start! It also includes a few arbitrary decisions however)

It does the following:

  • downloads the "full" image from the media section of each post;
  • saves it in a directory imgs/;
  • removes the [[posterous-content:...]] block at the end of the content, and replaces it with an ordered list of images, including the base_path

(I figured a list should be easy to turn into a gallery, etc if people wanted to add their own js)

Is this useful?

@markhepburn markhepburn Download and include images in posterous migrator.
Signed-off-by: Parker Moore <parkrmoore@gmail.com>
9997aeb
@parkr parkr referenced this pull request in jekyll/jekyll
Closed

Added image importing to posterous migrator #509

@parkr
Owner

Having some way to turn this off when you call the importer would be way cool.

@markhepburn

Hey, thanks for taking a look at this. Like I said, it was just to get the ball rolling as much as anything else.

What do you think the best interface for disabling it would be? And opt-in or opt-out I suppose is the other question; maybe opt-in is preferable, since it will take more time and bandwidth. Perhaps a flag --include-images?

I think my posterous account is still active if you'd like me to play with it.

@parkr
Owner

Hey! Thanks for getting back to me so quickly. I think the best interface would be to pass in an options has or something into the migrator that sets :include_images => true or something. We can encompass other options that way, too.

@parkr
Owner

By default, though, I don't think I would want to download all those images.

@markhepburn

Sorry for the delay in getting back to you -- had to wait for the weekend :)

Thanks for the "keyword args" tip, like I said I haven't really done any ruby. That looks much nicer though. I've implemented that functionality anyway, and it works fine on my old account. Do you want me to issue another pull request? I've pushed the changes to my fork, although I haven't rebased against the current state of affairs.

Thanks for your work on this, it'll be good to see jekyll getting some love again!

@parkr
Owner

If you have any improvements that you think should be included, maybe you can submit a pull-request against this branch? Then I can check those and merge them into this branch, then merge this branch into our project :)

@markhepburn

Huh, newbie question -- I was working against the entire mojombo/jekyll, not this extraction. Do you know if there's an easy way to pull my commits + history onto a clone of just jekyll-import? Short of using git-subtree, I suppose? Maybe it's easiest just via a single patch, since it's not exactly a big change :)

@mojombo
Owner

Merged.

@mojombo mojombo closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 31, 2013
  1. @markhepburn @parkr

    Download and include images in posterous migrator.

    markhepburn authored parkr committed
    Signed-off-by: Parker Moore <parkrmoore@gmail.com>
This page is out of date. Refresh to see the latest.
Showing with 46 additions and 2 deletions.
  1. +46 −2 lib/jekyll/importers/posterous.rb
View
48 lib/jekyll/importers/posterous.rb
@@ -27,7 +27,35 @@ def self.fetch(uri_str, limit = 10)
end
end
- def self.process(email, pass, api_token, blog = 'primary')
+ def self.fetch_images(directory, imgs)
+ def self.fetch_one(url, limit = 10)
+ raise ArgumentError, 'HTTP redirect too deep' if limit == 0
+ response = Net::HTTP.get_response(URI.parse(url))
+ case response
+ when Net::HTTPSuccess then response.body
+ when Net::HTTPRedirection then self.fetch_one(response['location'], limit - 1)
+ else
+ response.error!
+ end
+ end
+
+ FileUtils.mkdir_p directory
+ urls = Array.new
+ imgs.each do |img|
+ fullurl = img["full"]["url"]
+ uri = URI.parse(fullurl)
+ imgname = uri.path.split("/")[-1]
+ imgdata = self.fetch_one(fullurl)
+ open(directory + "/" + imgname, "wb") do |file|
+ file.write imgdata
+ end
+ urls.push(directory + "/" + imgname)
+ end
+
+ return urls
+ end
+
+ def self.process(email, pass, api_token, blog = 'primary', base_path = '/')
@email, @pass, @api_token = email, pass, api_token
FileUtils.mkdir_p "_posts"
@@ -41,7 +69,23 @@ def self.process(email, pass, api_token, blog = 'primary')
date = Date.parse(post["display_date"])
content = post["body_html"]
published = !post["is_private"]
- name = "%02d-%02d-%02d-%s.html" % [date.year, date.month, date.day, slug]
+ basename = "%02d-%02d-%02d-%s" % [date.year, date.month, date.day, slug]
+ name = basename + '.html'
+
+ # Images:
+ post_imgs = post["media"]["images"]
+ if post_imgs.any?
+ img_dir = "imgs/%s" % basename
+ img_urls = self.fetch_images(img_dir, post_imgs)
+
+ img_urls.map! do |url|
+ '<li><img src="' + base_path + url + '"></li>'
+ end
+ imgcontent = "<ol>\n" + img_urls.join("\n") + "</ol>\n"
+
+ # filter out "posterous-content", replacing with imgs:
+ content = content.sub(/\<p\>\[\[posterous-content:[^\]]+\]\]\<\/\p\>/, imgcontent)
+ end
# Get the relevant fields as a hash, delete empty fields and convert
# to YAML for the header
Something went wrong with that request. Please try again.