Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Super DRY Sitemaps for Rails and Sinatra Apps
Ruby
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
lib
rails
test
.gitignore
MIT-LICENSE
README.markdown
Rakefile
init.rb

README.markdown

Crawlable

Super DRY Sitemaps for Rails and Sinatra Apps (works on Heroku!)

Usage

Install

sudo gem install crawlable

Sitemap

Sitemaps are defined using a simple DSL, and even have a customizable XML Stylesheet (XSLT) so that if someone went to your sitemap.xml page in the browser, they would see a pretty table (see my sitemap and view source, it's just an xml file!).

Sitemap "http://www.example.com" do
  link posts_path, :priority => 0.7, :changes => 'daily'

  Post.all.each do |post|
    link posts_path(post), :updated_at => post.updated_at do
      image images_path(post.featured_image)
    end
  end
end

Result

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="/stylesheets/sitemap.xsl"?>
<urlset xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>http://viatropos.com/blog/how-to-make-any-client-side-cross-domain-request</loc>
    <lastmod>2010-08-08T01:13:40+00:00</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.5</priority>
  </url>
  <url>
    <loc>http://viatropos.com/blog/10-tips-to-reverse-engineer-any-web-service</loc>
    <lastmod>2010-08-08T01:13:40+00:00</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.5</priority>
  </url>
  <url>
    <loc>http://viatropos.com/blog/html5-blog-post-architecture</loc>
    <lastmod>2010-08-08T01:13:40+00:00</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.5</priority>
  </url>
  ...
</urlset>

Feed

Feeds are a key part of your site. Spiders and people can subscribe to it and be notified whenever you create new content. RSS and Atom are the two prominent feed formats today. There is actually quite a lot to the feed specs, and lots of little optimizations you can do to make sure you maximize your return on feed-investment. Here's a test for you:

  • What feed elements can have HTML?
  • Can feeds have images?
  • How do you put "tags" into a feed?
  • How many different xml elements does a feed node have?

Crawlable gives you an easy interface to create feeds. And feeds are all nicely formatted, just like they appear here. Here is a sample feed declaration using the Crawlable DSL:

Feed do
  posts do
    title         "Viatropos' Feed"
    author        "Lance Pollard"
    description   "A Blog About Rails"
    url           "http://viatropos.com"
    alternate_url "http://viatropos.com/blog/"
    category_url  "http://viatropos.com/tags/"
    icon          "http://viatropos.com/favicon.ico"
    logo          "http://viatropos.com/images/logo.png"

    site.posts.each do |post|
      entry post_url(post),
        :title        => post.title,
        :description  => post.description,
        :created_at   => post.created_at,
        :updated_at   => post.updated_at,
        :author       => "Lance Pollard",
        :categories   => post.tags,
        :content      => post.body
    end
  end
end.write("feed", :rss, :atom)

RSS Output

<?xml version="1.0"?>
<rss xmlns:slash="http://purl.org/rss/1.0/modules/slash/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" version="2.0">
  <channel>
    <title>Viatropos' Feed</title>
    <link>http://viatropos.com</link>
    <description>Creativity and Emergence</description>
    <lastBuildDate>Sat, 07 Aug 2010 19:13:41 -0600</lastBuildDate>
    <image>
      <url>/images/icons/viatropos-site-icon.png</url>
      <title>Viatropos' Feed</title>
      <link>http://viatropos.com</link>
      <description>A Blog About Rails</description>
    </image>
    <item>
      <title>How to Build an HTML5 Layout</title>
      <link>http://viatropos.com/blog/html5-blog-post-architecture</link>
      <description><![CDATA[There is not much information out there yet describing how to create a fully-featured layout using the entire set of new HTML5 tags. This post shows how to structure a rich Blog Post using most of the ...]]></description>
      <content:encoded>There is not much information out there yet describing how to create a fully-featured layout using the entire set of new HTML5 tags. This post shows how to structure a rich Blog Post using most of the ...</content:encoded>
      <guid>http://viatropos.com/blog/html5-blog-post-architecture</guid>
      <author>Lance Pollard</author>
      <category><![CDATA[html5]]></category>
      <category><![CDATA[html]]></category>
      <category><![CDATA[blog]]></category>
      <category><![CDATA[dom]]></category>
    </item>
    ...
  </channel>
</rss>

Atom Output

<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns:g="http://base.google.com/ns/1.0" xmlns:thr="http://purl.org/syndication/thread/1.0" xmlns:re="http://purl.org/atompub/rank/1.0" xmlns="http://www.w3.org/2005/Atom" xml:base="http://viatropos.com" xml:lang="en-us">
  <id>http://viatropos.com/blog/</id>
  <uri>http://viatropos.com</uri>
  <title type="xhtml">
    <div xmlns="http://www.w3.org/1999/xhtml">Viatropos' Feed</div>
  </title>
  <subtitle type="xhtml">
    <div xmlns="http://www.w3.org/1999/xhtml">A Blog About Rails</div>
  </subtitle>
  <updated>2010-08-07T19:13:41Z</updated>
  <author>
    <name>Lance Pollard</name>
  </author>
  <link href="http://viatropos.com" rel="self" type="application/atom+xml"/>
  <link href="http://viatropos.com/blog/" rel="alternate" type="text/html"/>
  <icon>/images/icons/viatropos-favicon.png</icon>
  <logo>/images/icons/viatropos-site-icon.png</logo>
  <entry>
    <id>http://viatropos.com/blog/html5-blog-post-architecture</id>
    <title type="xhtml">
      <div xmlns="http://www.w3.org/1999/xhtml">How to Build an HTML5 Layout</div>
    </title>
    <link href="http://viatropos.com/blog/html5-blog-post-architecture" rel="alternate"/>
    <published>2010-08-04T15:51:00Z</published>
    <updated>2010-08-04T15:51:00Z</updated>
    <author>
      <name>Lance Pollard</name>
    </author>
    <category scheme="http://viatropos.com/tags/" label="Html5" term="html5"/>
    <category scheme="http://viatropos.com/tags/" label="Html" term="html"/>
    <category scheme="http://viatropos.com/tags/" label="Blog" term="blog"/>
    <category scheme="http://viatropos.com/tags/" label="Dom" term="dom"/>
    <summary type="xhtml">
      <div xmlns="http://www.w3.org/1999/xhtml">There is not much information out there yet describing how to create a fully-featured layout using the entire set of new HTML5 tags. This post shows how to structure a rich Blog Post using most of the ...</div>
    </summary>
    <content type="xhtml">
      <div xmlns="http://www.w3.org/1999/xhtml">
        Your post
      </div>
    </content>
  </entry>
</feed>

Integration

Ruby on Rails

To setup sitemaps in a Rails app:

  1. Create a file in either lib/sitemap.rb or config/sitemap.rb.
  2. Build the sitemap DSL like above.
  3. Create an observer for after_create on a Post or whatever you want in your sitemap.
  4. Call Crawlable::Sitmap.write("sitemap.xml")

Crawlable::Sitmap is just the foundation, creating the structure for generating a Sitemap. You're free to regenerate it wherever/whenever you want.

Sinatra or just Plain Ruby

I just write the sitemap DSL straight in app.rb, and it gets regenerated everytime you restart your app. Easy. There are no dependencies other than ActiveSupport.

Features

  • Works on Heroku
  • Pings Google, Bing, Yahoo!, and Ask whenever anything changes

Alternatives

Resources

Something went wrong with that request. Please try again.