Tutorial needs an update

[johnmee]

I'm just trying to run through the tutorial and am thinking quite a few things have changed since it was last revised...

For example:

• it talks about creating a directory "dmoz/spiders" which contrasts with the comment in "spiders/init.py" suggesting we run "scrapy genspider ...".
• the code in the tutorial subclasses "BaseSpider", yet the genspider creates almost identical but subclasses "CrawlSpider"
• doing as instructed (creating dmoz/spiders/dmoz_spider.py and running scrapy crawl...) fails with "Spider not found: dmoz" but following the comments in the codebase works ok.

That area seems to be the only bit that has changed significantly.

[barraponto]

It seems scrapy genspider defaults to the crawler template, yet I'd rather use -t basic...

[pablohoffman]

Scrapy tutorial doesn't cover genspider command. That's more of an advanced command for when you're dealing with project with lots of spiders and have predefined templates for some of them.

I don't understand the last point. Are you saying the code in the tutorial is wrong? Which part? Did you know there is a fully functional code in the scrapy/dirbot github project?

[johnmee]

You're right. The tutorial doesn't cover the genspider command. When I followed the tutorial and got to the "Our first spider" section it conflicted with the comment in spiders/init "To create first spider ... use scrapy genspider...". This caused me to lose confidence in the tute. Would it be better to have the tutorial go straight to showing how to use genspider? or at least mention that you're demonstrating first principles and genspider is for later.

With the last point, from memory, I didn't appreciate at the time that the commands available vary with respect to the current working directory. It made more sense after I came to that, so perhaps put more emphasis on the text "go to the project’s top level directory" and why that's relevant.

No big drama. Just keen to help because the tutorial is great and Scrapy even better!

[pablohoffman]

@johnmee thanks for your suggestions, I've pretty much implemented all of them (see referenced commits)