This repository has been archived by the owner. It is now read-only.
Crawls a Rails project looking for dead links, unused routes or other problems
Ruby JavaScript
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
lib
priv/test-app
spec
README.markdown

README.markdown

Invoking rails-crawler

rails-crawler takes two arguments: the path to the Rails application and the site-relative start URL:

ruby crawl.rb /path/to/app /

If your start URL is a static page, you need to specify it explicitly:

ruby crawl.rb /path/to/app /index.html

How it works

rails-crawler uses the same interface as Rails integration tests to query URLs and retrieve their responses. Thus, it does not actually use the network at all.

Beginning with a GET request to the start URL, the site is walked in a depth-first fashion, following anchors and form actions, until all reachable paths have been processed. Each path is matched to its corresponding route (except static resources or non-existent pages which are handled separately); routes which are never hit are marked as "unused".

Currently, the application is loaded in the production environment, but this will probably change in the future.

Current limitations

rails-crawler currently only crawls through anonymously-accessible paths. If they require the user to log in, they will be marked as unused. Forms are submitted with no post variables, and so most will usually fail with HTTP error 422.

TODO list

  • The ability to authenticate with sites
  • A way to submit valid form data
  • Look in the config/ directory for a configuration file

License

rails-crawler is released under the GPL.