You will need to install PhantomJS. We use the network requst abort functionality that was added in version 1.9, so lower versions will not work. You can check the installed version of PhantomJS by running
Add this to your Gemfile:
That's it, we're done!
require 'indexable' use Indexable::Middleware
There is an example Rack application in the
(Optional) Adding the following meta tag to the head section of your pages will opt you into the AJAX crawling scheme, which the middleware uses in addition to checking the user agent in order to determine whether a request is coming from a crawler.
<meta name="fragment" content="!">