Skip to content
Simple Scrapy project to crawl bol.com
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bol_crawler
.gitignore
LICENSE
readme.md
setup.py

readme.md

bolcom_crawler

This is a really simple crawler that makes use of Scrapy to crawl bol.com.

Usage

The Crawler instance has two functions you can use, crawl_products and crawl_category. See an example below.

from bol_crawler.crawler import Crawler
crawler = Crawler()

# to crawl products
products = crawler.crawl_products(
    [
        'https://www.bol.com/nl/p/lg-34gl750-b-ultragear-gaming-monitor/9200000115819731',
    ]
)

# to crawl a category
products = crawler.crawl_category(
    [
        'https://www.bol.com/nl/l/gaming-toetsenborden/N/18214/', 0  # the 0 value is how often you want to go to the next page. 0 is just crawling the first page
    ]
)
You can’t perform that action at this time.