For those who are tired to use scrapy's brackets and quotes. In one line of code this library will increase the readability of your code, make it more stable to human error. And will introduce IDEs code completion features to your scrapy items. You will be able to use scrapy items as normal python objects using dot syntax.
The regular bulky way to do it:
regular_scrapy_item['some_field'] = 42
print(regular_scrapy_item.get('some_field'))
The simple dot-items way:
scrapy_dot_item.some_field = 42
print(scrapy_dot_item.some_field)
pip install scrapy-dot-items
You can add functionality with the dot_item decorator to a single class
from scrapy_dot_items import dot_item
@dot_item
class RealEstateItem(scrapy.Item):
price = scrapy.Field()
dot_scrapy_item = RealEstateItem()
dot_scrapy_item.price = 1000
print(dot_scrapy_item.price) # prints 1000
You can apply this functionality globally
from scrapy_dot_items import dot_items_globally
dot_items_globally() # now every scrapy.Item class will have this functionality
class RealEstateItem(scrapy.Item):
price = scrapy.Field()
dot_scrapy_item = RealEstateItem()
dot_scrapy_item.price = 1000
print(dot_scrapy_item.price) # prints 1000
scrapy-dot-items are backwards compatible. You can install it into your old project and mix regular and dot style.
dot_scrapy_item['price'] = 2000 # you can still set and get the items the regular way too
print(dot_scrapy_item.get('price')) # prints 2000
print(dot_scrapy_item.price) # prints 2000
This project uses pipenv instead of pip. Setup:
- intall pipenv with 'pip install pipenv'
- create a virtual environment using pipenv with 'pipenv shell' in the root directory
- install the dependencies with 'pipenv install'
Attention: Please install all packages with 'pipenv install <package_name>'. Do NOT use 'pip install ...'. The same for unistalling 'pipenv uninstall <package_name>'