Skip to content

Web scrapers used by Kanojo for database seeding and periodical updates of new titles

License

Notifications You must be signed in to change notification settings

kanojo-db/scrapers

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Kanojo Scrapers

This repository contains web scrapers used for partially seeding and for periodical updates of the Kanojo database.

Contents

Currently, scrapers for the following are provided:

  • DMM Actresses
    A scraper for actress information on actress.dmm.co.jp
  • JAVLibrary
    A scraper for movie information on javlibrary.com
  • MGStage
    A sitemap scraper for movie information on mgstage.com
  • Minna no AV
    A scraper for actress information on minnano-av.com
  • u15dvdinfo
    A sitemap scraper for movie information on u15dvdinfo.com

Requirements

This repository requires a somewhat recent version of Python 3.

Direct dependencies for the scrapers are provided in requirements.txt.

Planned Scrapers

The following scrapers are planned/needed:

  • 10musume.com (Movie information)
  • 1pondo.tv (Movie information)
  • 7mmtv.tv (Movie information)
  • adult.contents.fc2.com (Movie information)
  • aventertainments.com (Movie and model information)
  • caribbeancom.com (Movie information)
  • caribbeancompr.com (Movie information)
  • dmm.co.jp (Movie information)
  • eic-book.com (Movie information)
  • faleno.jp (Movie and model information)
  • fc2hub.com (Movie information)
  • heyzo.com (Movie information)
  • idolerotic.net (Movie and model information)
  • javbus.com (Movie information)
  • javfc2.net (Movie information)
  • javhoo.com (Movie information)
  • moodyz.com (Movie and model information)
  • muramura.tv (Movie information)
  • pacopacomama.com (Movie information)
  • sod.co.jp and subsidiaries (Movie and model information)
  • supjav.com (Movie information)
  • tktube.com (Movie information)
  • tokyo-hot.com (Movie information)

About

Web scrapers used by Kanojo for database seeding and periodical updates of new titles

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages