Multiple crawls with the same crawler

[jdemaeyer]

While we don't support using the same crawler for multiple spiders, it is currently possible to run the same crawler (with the same spider) twice, possibly with different spider arguments. I.e., this works:

crawler = Crawler(myspidercls)
crawler.crawl(my_sparg='blah')
crawler.crawl(my_sparg='other_blah')

This is done in some of our tests (1, 2, 3). Are these tests legacy code or is support for multiple crawls part of the public API (see also this discussion)?

[nramirezuy]

@jdemaeyer pretty sure is legacy code. We should remove those tests to avoid confusion.

[wRAR]

I think we can deprecate running crawl() for the second time after the first one finishes (they can't be run concurrently already).

As for tests I'm sure it's just a shortcut, not something written explicitly.