Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
157 lines (113 sloc) 4.1 KB

murf

Metacritic Movie user review fetcher 🎦

NPM version NPM download count Greenkeeper badge

Scraping GIF

Description

This Node.js package fetches Metacritic Movie user reviews by scraping pages using cheerio and node-fetch. Notable features such as:

  • Fetch whole user reviews or by defined value
  • Compiled from Typescript to CommonJS and ES modules
  • Typescript declaration file available

Note that this package doesn't use official packages or API endpoints provided by Metacritic.

Usage

Adding to your project

  • Using npm

    npm install murf
  • Using yarn

    yarn add murf

Search titles using searchByQuery

  • Script

    // using require (commonjs)
    const { searchByQuery } = require('murf')
    
    // using import (es module)
    import { searchByQuery } from 'murf'
    
    searchByQuery('venom').then(results => {
      console.log(JSON.stringify(results, null, 2))
    })
  • Output (truncated, fetched on February 13, 2019)

    [
      {
        "name": "Venom",
        "year": 2018,
        "slug": "venom-2018",
        "url": "...",
        "image": "...",
        "score": 35,
        "scoreWord": "unfavorable"
      },
      {
        "name": "Venom",
        "year": 2005,
        "slug": "venom",
        "url": "...",
        "image": "...",
        "score": 25,
        "scoreWord": "unfavorable"
      }
    ]

Fetch reviews using scrapeReviews

  • Script

    // using require (commonjs)
    const { scrapeReviews } = require('murf')
    
    // using import (es module)
    import { scrapeReviews } from 'murf'
    
    scrapeReviews('venom-2018').then(results => {
      console.log(JSON.stringify(results, null, 2))
    })
  • Output (truncated, fetched on February 13, 2019)

    [
      {
        "reviewer": "moviemitch96",
        "date": "Oct  5, 2018",
        "score": 6,
        "review": "..."
      },
      {
        "reviewer": "rusty_toaster",
        "date": "Oct  5, 2018",
        "score": 7,
        "review": "..."
      },
      ...
    ]

Lambda Deployment

Powered with Zeit Now, we created a lambda deployment that you can consume to search and scrape on murf-lambda.now.sh using these queries:

  • q to search titles (e.g. spider verse)
  • s to scrape reviews using title slugs obtained from search (e.g. venom-2018)
  • c to define how many to scrape (works only with s)

Example queries:

You can view the source code for the deployment on the demo directory or on the deployment source page.

API

Haven't done this part. Do submit a pull request if you want to contribute.

Related

License

MIT

You can’t perform that action at this time.