Sitemaps parser#42

Merged

jnioche merged 12 commits intomasterfrom

Jan 7, 2015

Commits on Dec 18, 2014

Commits on Dec 19, 2014

SiteMapParser acks even if an exception is caught
jnioche
committed

Commits on Jan 5, 2015

Fetcher and Parser use the status stream to report errors instead of failing + handle redirections
jnioche
committed
Fetcher : use the metadata from the protocol response + pass the exception message into metadata
jnioche
committed
fetcher and parser declare status stream + use printerbolt to display tuples from status stream
jnioche
committed
Bug fix : finishFetchItem and ack only needed in finally section
jnioche
committed
renamed REDIR status into REDIRECTION
jnioche
committed
SitemapParserBolt sends a tuple to the status stream for the main URL + adds a status field when sending to the status stream
jnioche
committed
updated the tests for SitemapParserBolt + showed how to use it in Topologgy (even if the demo spout doesn't currently generate a relevant URL and metadata)
jnioche
committed

Commits on Jan 6, 2015

merged CrawlTopology
jnioche
committed

Commits on Jan 7, 2015

Implemented Gui's comments to sitemap parser
jnioche
committed