scrapy-feedstreaming

Scrapy live Streaming data. scrapy.extensions.feedexport.FeedExporter fork to export item during scraping. See [https://medium.com/@alex_ber/scrapy-streaming-data-cdf97434dc15]

See CHANGELOG.md for detail description.

Getting Help

QuickStart

python3 -m pip install -U scrapy-feedstreaming

Installing from Github

python3 -m pip install -U https://github.com/alex-ber/scrapy-feedstreaming/archive/master.zip

Optionally installing tests requirements.

python3 -m pip install -U https://github.com/alex-ber/scrapy-feedstreaming/archive/master.zip#egg=alex-ber-utils[tests]

Or explicitly:

wget https://github.com/alex-ber/scrapy-feedstreaming/archive/master.zip -O master.zip; unzip master.zip; rm master.zip

And then installing from source (see below).

Installing from source

python3 -m pip install -r req.txt # only installs "required" (relaxed)

python3 -m pip install . # only installs "required"

python3 -m pip install .[tests] # installs dependencies for tests

Alternatively you install install from requirements file:

python3 -m pip install -r requirements.txt # only installs "required"

python3 -m pip install -r requirements-tests.txt # installs dependencies for tests

From the directory with setup.py

python3 setup.py test #run all tests

or

pytest

Installing new version

See https://docs.python.org/3.1/distutils/uploading.html

python3 setup.py sdist upload

Requirements

scrapy-feedstreaming requires the following modules.

• Python 3.6+