Environment specific robots.txt for your Rails Apps
Ruby JavaScript Shell
Switch branches/tags
Nothing to show
Pull request Compare This branch is 42 commits behind LaunchAcademy:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
app
bin
config
lib
script
spec
.gitignore
.rspec
.rvmrc
.travis.yml
.yardopts
Gemfile
Guardfile
LICENSE
MIT-LICENSE
README.md
Rakefile
roboto.gemspec

README.md

Roboto

Build Status

Roboto is a Rails Engine that gives you the ability to specify enviornment specific robots in your Rails 3.1+ application.

Don't let crawlers access your staging environment. This is bad for SEO.

Installing

First, remove the default, generate robots.txt in your Rails App

  #> rm public/robots.txt

Next, add roboto to your gemfile:

  gem 'roboto'

Then, add robot to your routes (config/routes.rb):

  Rails.application.routes.draw do
    mount_roboto
  end

You can now specify environment specific robots.txt files in config/robots.

It's recommended for staging that you do disallow crawlers from accessing your site. Once you've created a separate Rails environment for staging, define a config/robots/staging.txt file like so:

  #place this in config/robots/staging.txt
  User-Agent: *
  Disallow: /

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request