You can clone with
Just catch the blithering exception...
Use non-unicode regexps for when $KCODE="u"
Allow gem to be built.
Allow robots.txt with no useragent
Update URL chars whitelist
add more to readme
Add new files
Add test for Getter
Split relevant bits of module into Getter
Rewrite and polish parser to a happy degree, next to do the requestor
Add some more detailed (and currently failing tests)
Fix indentation to use spaces not tabs
Import changes from Rapportive
like Google support the useragent (over that user-agent) for indentif…
…y the robot ID
accepts aslo the relative path for the XML sitemap, technically is no…