Home

GreekParliament-Records-Scraper-Downloader

Python crawler-downloader for greek parliaments conference records.

    BeautifulSoup, Requests, sqlite3, pandas

First create a database table on sqlite, the file 'vouliDBschema' describes the schema that is used.

    python scraper.py -p <number> or --pagecrawl <number>

    python scraper.py -f <number> or --startfrom <number>

    python scraper.py -a or --all

The data are extracted and stored to sqlite db. The db file can be used as input to the downloader script.

    sqlite3, urllib2, requests

    python recordsDownload.py -d or --download  '/UserFiles/32323-2312382-232852-2929/es2828582.doc'

    python recordsDownload.py -f or --fromID <id>

    python recordsDownload.py -a or --all