Skip to content
Find file
Fetching contributors…
Cannot retrieve contributors at this time
27 lines (26 sloc) 640 Bytes
0.10.0
- Make sure the fetch robots.txt operation happens with a user agent (via rb2k)
0.9.0
- Fix http://github.com/fizx/robots/issues#issue/1
- Tests don't rely on network.
0.8.0
- Add multiple values from robots.txt (via joost)
0.7.3
- Move to jeweler, gemcutter
0.7.2
- Add Ruby 1.9 compatibility
0.5-0.7.1
- Lost the changelog information :/
0.4.0
- Fixed other_values bug
- added crawl-delay support
0.3.2
- fixed breaking on reddit.com
0.3.1
- fixed bug in disallows handling
- partially mocked out open-uri
0.3.0
- added loggable dependency
0.2.0
- IF robot.txt 404s, assume allowed.
- Added CHANGELOG
Jump to Line
Something went wrong with that request. Please try again.