* start with scrapy.Spider, then mention spider arguments, then describe generic spiders; * change wording regarding start_urls/start_requests; * show an example of start_requests vs start_urls; * show an example of dicts as items; * as defining Item is an optional step now, docs for Items are moved below Spider docs.
Please check - all docs except for overview & tutorial should be updated.
What do you think about adding FEED_EXPORT_FIELDS option, to allow defining a list of fields to export? Without Item classes CSV exporter can't figure out the header robustly (currently fields of a first item are used). @nramirezuy also mentioned this feature here.
It was a hack, and dicts-as-items cover most use cases. Dicts don't allow to attach metadata to fields, but e.g. adding "_meta" key and removing it in a custom serializer is no worse than creating classes dynamically.
Maybe instead of checking for dict/BaseItem explicitly we can start checking for MutableMapping, but it is more risky. I think that starting with more strict requirements on spider output is better - be can make them less strict in future.