Skip to content


Alagu posterous importer (conflicts resolved, includes rate limiting) #477

wants to merge 4 commits into from

4 participants


I merged #402 onto master at a point after #472 and resolved the conflicts. The merge is at 5d6e87b. In testing the importer I ran into some rate limiting issues from Posterous and updated the code to handle those. (sleep for retry_after + 1 seconds)

When I tried to rebase master onto #402 git informed me that the branches had diverged. I would have needed to --force the pushto GitHub and I'm not sure what would have happend to the PR at that point. I am not skilled enough at git to know how I could have done this any differently so I created a new branch and PR instead.

If it would be better to --force the existing PR please let me know the best way for me to do that. :) Otherwise I am hoping this PR merges cleanly.


Is there a way for me to use this script now, so I don't have to wait for it to be merged?

If that's possible, any advice on how to do it would be appreciated.

@alagu alagu referenced this pull request

Posterous importer update #651


I have pulled changes and updated in pull request 651

Jekyll member

Updated in #651.

@parkr parkr closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 16, 2011
  1. @alagu

    Change to new posterous API

    alagu committed
Commits on Nov 26, 2011
  1. @simensen

    YAML front matter fix, cleaned up slug generation, added tags and sto…

    simensen committed
    …re some posterous information from the original post.
Commits on Jan 23, 2012
  1. @simensen

    Merge remote-tracking branch 'origin/alagu-master' into alagu-postero…

    simensen committed
    Conflict created due to #472.
  2. @simensen

    Handle Posterous rate limiting

    simensen committed
Showing with 27 additions and 7 deletions.
  1. +27 −7 lib/jekyll/migrators/posterous.rb
34 lib/jekyll/migrators/posterous.rb
@@ -1,11 +1,14 @@
require 'rubygems'
require 'jekyll'
require 'fileutils'
-require 'net/http'
+require 'net/https'
+require 'open-uri'
require 'uri'
require "json"
-# ruby -r './lib/jekyll/migrators/posterous.rb' -e 'Jekyll::Posterous.process(email, pass, api_key, blog)'
+# ruby -r './lib/jekyll/migrators/posterous.rb' -e 'Jekyll::Posterous.process(email, pass, api_token, blog, tags_key)'
+# You can find your api token in posterous api page - . Click on any of the 'view token' links to see your token.
+# blog is optional, by default it is the primary one
module Jekyll
module Posterous
@@ -14,6 +17,9 @@ def self.fetch(uri_str, limit = 10)
raise ArgumentError, 'Stuck in a redirect loop. Please double check your email and password' if limit == 0
response = nil
+ puts uri_str
+ puts '-------'
Net::HTTP.start('') do |http|
req =
req.basic_auth @email, @pass
@@ -23,36 +29,50 @@ def self.fetch(uri_str, limit = 10)
case response
when Net::HTTPSuccess then response
when Net::HTTPRedirection then fetch(response['location'], limit - 1)
+ when Net::HTTPForbidden then
+ retry_after = response.to_hash['retry-after'][0]
+ puts "We have been told to try again after #{retry_after} seconds"
+ sleep(retry_after.to_i + 1)
+ fetch(uri_str, limit - 1)
else response.error!
- def self.process(email, pass, api_token, blog = 'primary')
+ def self.process(email, pass, api_token, blog = 'primary', tags_key = 'categories')
@email, @pass, @api_token = email, pass, api_token
FileUtils.mkdir_p "_posts"
- posts = JSON.parse(self.fetch("/api/v2/users/me/sites/#{blog}/posts?api_token=#{@api_token}").body)
+ posts = JSON.parse(self.fetch("/api/2/sites/#{blog}/posts?api_token=#{@api_token}").body)
page = 1
while posts.any?
posts.each do |post|
title = post["title"]
- slug = title.gsub(/[^[:alnum:]]+/, '-').downcase
+ slug = title.gsub(/[^[:alnum:]]+/, '-').gsub(/^-+|-+$/, '').downcase
+ posterous_slug = post["slug"]
date = Date.parse(post["display_date"])
content = post["body_html"]
published = !post["is_private"]
name = "%02d-%02d-%02d-%s.html" % [date.year, date.month,, slug]
+ tags = []
+ post["tags"].each do |tag|
+ tags.push(tag["name"])
+ end
# Get the relevant fields as a hash, delete empty fields and convert
# to YAML for the header
data = {
'layout' => 'post',
'title' => title.to_s,
- 'published' => published
+ 'published' => published,
+ tags_key => tags,
+ 'posterous_url' => post["full_url"],
+ 'posterous_slug' => posterous_slug
}.delete_if { |k,v| v.nil? || v == ''}.to_yaml
# Write out the data and content to file"_posts/#{name}", "w") do |f|
+ puts name
f.puts data
f.puts "---"
f.puts content
@@ -60,7 +80,7 @@ def self.process(email, pass, api_token, blog = 'primary')
page += 1
- posts = JSON.parse(self.fetch("/api/v2/users/me/sites/#{blog}/posts?api_token=#{@api_token}&page=#{page}").body)
+ posts = JSON.parse(self.fetch("/api/2/sites/#{blog}/posts?api_token=#{@api_token}&page=#{page}").body)
Something went wrong with that request. Please try again.