Skip to content
Python API to scrape data from senscritique.com.
Python
Branch: master
Clone or download

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples
senscritiquescraper
tests
.coveragerc
.gitignore
.travis.yml
LICENSE
Pipfile
Pipfile.lock
README.md
codecov.yml
setup.py

README.md

senscritiquescraper

Build Status Codacy Badge

API to extract data from senscritique.com.

Some examples of its usage are included in the examples folder.

Examples of what can be extracted :

The "list_work" urls are difficult to find, here are the one for each genre where you can access all the subgenres :

Requirements

  • requests
  • beautifulsoup4
  • lxml
  • pandas

Installation in a virtualenv (recommended)

pipenv install '-e .'

Examples Usage

Show the help and the available options.

python scr_get_topchart.py -h

Some scripts using the API are provided in the examples folder.

scr_get_topchart

usage: scr_get_topchart [-h] [--debug] [-u URL] [main_argument]

Senscritique scraper for a top list/chart.

positional arguments:
  main_argument      URL to parse

optional arguments:
  -h, --help         show this help message and exit
  --debug            Display debugging information
  -u URL, --url URL  URL to parse (same as without argument)

scr_get_collection

python scr_get_collection.py -h
usage: scr_get_collection [-h] [--debug] [-u USER] [main_argument]

Senscritique scraper for an user collection

positional arguments:
  main_argument         Name of the user

optional arguments:
  -h, --help            show this help message and exit
  --debug               Display debugging information
  -u USER, --user USER  Name of the user (same as without argument)

scr_get_survey

python scr_get_survey.py -h
usage: scr_get_survey.py [-h] [--debug] [-u URL] [main_argument]

Senscritique scraper for a survey.

positional arguments:
  main_argument      URL to parse

optional arguments:
  -h, --help         show this help message and exit
  --debug            Display debugging information
  -u URL, --url URL  URL to parse (same as without argument)

scr_get_list_work

python scr_get_list_work.py -h
usage: scr_get_list_work.py [-h] [--debug] [-u URL] [main_argument]

Senscritique scraper for a list_work.

positional arguments:
  main_argument      URL to parse.

optional arguments:
  -h, --help         show this help message and exit
  --debug            Display debugging information
  -u URL, --url URL  URL to parse (same as without argument)

scr_get_work_details

python scr_get_work_details.py -h
usage: scr_get_work_details.py [-h] [--debug] [-f FILE] [main_argument]

This script will export in a csv file one or several works from senscritique.
The -f option will use the 'URL' field of a csv file.

positional arguments:
  main_argument         File to parse.

optional arguments:
  -h, --help            show this help message and exit
  --debug               Display debugging information
  -f FILE, --file FILE  File to parse.
You can’t perform that action at this time.