Flickr Scrapy Bot is a web scraping bot build upon Scrapy, an application framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival.
This bot collects images licensed under CC using Flickr API.
Flickr Scrapy Bot automates collecting of images on creativechain.net.
You must already have:
- python (2.7 or 3.3 or above)
- Flickr API KEY
You need to install scrapy:
$ pip install scrapy
It is recommended to set a virtual environment.
Clone the repo:
$ git clone https://github.com/gcamerli/flickr_scrapy_bot.git
Change path:
$ cd flickr_scrapy_bot
Run spider, setting up the Flickr API key:
$ FLICKR_KEY=******** scrapy crawl flickr_cc
Collected images are saved into images dir, which will be created if not exists.
All you need to know about Scrapy it is possible to find here.
@orangain: sushibot
This work is licensed under the terms of GNU General Public License v3.0
Donations are accepted at:
- BTC: 1DNWtR4wJbFE7vjcfQvuj4iE7FURYBURtr
- CREA: CRgURSBHqM5FzQhy2iuGKPAHycTUwzr3Ei