This is a demo project to show the use of Scrapy and Splash for infinite scroll. Specifically, it gets all accordion related tracks from the last week on SoundCloud.
- Docker
- Docker Compose
Run:
docker-compose up
This will launch both Splash and the spider. The resulting scrapes will be stored in the scrapes folder and new items will be persisted to a SQLite database tracks.db.
By default all items will be scraped. To specify a maximum number, copy the provided .env.example to .env and edit the TARGET_COUNT value within.