Skip to content

Latest commit

Β 

History

54 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

MyDramaList Unofficial API

A serverless FastAPI-based web scraper for MyDramaList.com, designed for deployment on Vercel.

Deploy with Vercel

Note: This project is inspired from @tbdsux/kuryana. Special thanks to @tbdsux!


πŸš€ Features

  • 13 Endpoints β€” Search, details, cast, episodes (list / single / enriched-all), reviews, recommendations, people, seasonal, lists, user watchlists, airing calendar, plus health check
  • Interactive Dashboards β€” Includes interactive API docs UI (/static/index.html) and a live drama calendar (/static/calendar.html)
  • Episode Deep Scraping β€” Visits each /episode/{n} page to extract description, cover image, rating, and season
  • Concurrent Fetching β€” episodes/all batches requests (4 at a time) with anti-ban delays
  • Serverless Ready β€” Optimized for Vercel deployment
  • Caching Mechanisms β€” In-memory caching for airing calendar to prevent anti-ban limits (1-hour TTL)
  • Rate Limiting β€” Built-in 1 s delay per endpoint call; 0.5 s pause between episode batches
  • Error Handling β€” Consistent JSON error responses with proper HTTP status codes
  • Modular Design β€” Separate scraper.py for all scraping logic

πŸ“‹ API Endpoints

πŸ” Search

Method Endpoint Description
GET /api/search/q/{query} Search dramas by title. Returns up to 20 results.

🎬 Drama

Method Endpoint Description
GET /api/id/{slug} Full drama details β€” title, synopsis, genres, cast, rating, etc.
GET /api/id/{slug}/cast Cast & crew grouped by role
GET /api/id/{slug}/reviews User reviews (up to 10)
GET /api/id/{slug}/recs Drama recommendations with reasons and votes

πŸ“Ί Episodes

Method Endpoint Description
GET /api/id/{slug}/episodes Episode list β€” number, title, air date
GET /api/id/{slug}/episodes/{n} Single episode β€” title, description, cover image, air date, rating, season
GET /api/id/{slug}/episodes/all All episodes enriched β€” concurrently fetches every episode page for full details

πŸ‘€ People & Lists

Method Endpoint Description
GET /api/people/{people_id} Person details β€” biography, filmography, personal info
GET /api/seasonal/{year}/{quarter} Top dramas for a season (quarter: 1=Winter 2=Spring 3=Summer 4=Fall)
GET /api/list/{id} Items in a user-created public list
GET /api/dramalist/{user_id} A user's public watchlist

πŸ“… Calendar

Method Endpoint Description
GET /api/calendar Get currently airing dramas grouped by day of the week (Monday–Sunday)

βš™οΈ Utility

Method Endpoint Description
GET /api/health Health check

Slug format: {id}-{drama-name} β€” e.g., 58651-run-on, 746993-my-demon


πŸ“Š Response Examples

GET /api/id/{slug}/episodes/{n} β€” Single Episode

{
  "episode_number": "1",
  "url": "https://mydramalist.com/58651-run-on/episode/1",
  "title": "Run On Episode 1",
  "image": "https://i.mydramalist.com/pRvkV_3m.jpg",
  "description": "Ki Seon Gyeom notices bruises on Kim Woo Shik's body. Trying to get back into her professor's good graces, Oh Mi Joo takes on an interpreting gig. (Source: Netflix)",
  "air_date": "December 16, 2020",
  "rating": "8.5/10",
  "season": "1"
}

GET /api/id/{slug}/episodes/all β€” All Episodes Enriched

{
  "episodes": [
    {
      "episode_number": "1",
      "title": "Run On Episode 1",
      "air_date": "Dec 16, 2020",
      "description": "Ki Seon Gyeom notices bruises on Kim Woo Shik's body...",
      "image": "https://i.mydramalist.com/pRvkV_3m.jpg",
      "rating": "8.5/10",
      "season": "1"
    }
  ],
  "total": 16
}

GET /api/id/{slug}/episodes β€” Episode List

{
  "episodes": [
    { "episode_number": "1", "title": "Run On Episode 1", "air_date": "Dec 16, 2020" },
    { "episode_number": "2", "title": "Run On Episode 2", "air_date": "Dec 17, 2020" }
  ],
  "total": 16
}

GET /api/search/q/{query}

{
  "results": [
    {
      "title": "Squid Game",
      "slug": "40257-round-six",
      "year": "2021",
      "image": "https://i.mydramalist.com/X6vkX_4s.jpg",
      "rating": "8.4",
      "url": "https://mydramalist.com/40257-round-six"
    }
  ],
  "total": 20
}

GET /api/id/{slug}/recs

{
  "recommendations": [
    {
      "title": "My Secret Romance",
      "year": "2017",
      "slug": "21465-my-secret-romance",
      "url": "https://mydramalist.com/21465-my-secret-romance",
      "image": "https://i.mydramalist.com/...",
      "rating": "7.3",
      "reasons": ["Both have office romance", "Similar chemistry between leads"],
      "recommended_by": "username",
      "votes": "42"
    }
  ],
  "total": 25,
  "pages_fetched": 1
}

GET /api/calendar β€” Airing Calendar

{
  "days": {
    "Monday": [
      {
        "title": "Scent of the Wind",
        "slug": "785294-kaze-kaoru",
        "url": "https://mydramalist.com/785294-kaze-kaoru",
        "image": "https://i.mydramalist.com/l0bepx_4t.jpg",
        "rating": "",
        "episode": "Episode 66",
        "air_time": "04:30 AM",
        "network": "Asia/Tokyo"
      }
    ],
    "Tuesday": [],
    "Wednesday": [],
    "Thursday": [],
    "Friday": [],
    "Saturday": [],
    "Sunday": []
  },
  "total": 413,
  "url": "https://mydramalist.com/calendar"
}

Error Response

{
  "code": 404,
  "error": true,
  "description": "404 Not Found"
}

πŸ› οΈ Tech Stack

Tool Purpose
Python 3.12 Primary language
FastAPI Web framework + auto /docs
BeautifulSoup4 HTML parsing
curl_cffi Anti-bot HTTP requests (browser impersonation)
Uvicorn ASGI server

πŸ“ Project Structure

project_root/
β”œβ”€β”€ main.py              # FastAPI routes
β”œβ”€β”€ scraper.py           # All scraping logic
β”œβ”€β”€ requirements.txt     # Dependencies
β”œβ”€β”€ vercel.json          # Vercel serverless config
β”œβ”€β”€ static/
β”‚   β”œβ”€β”€ calendar.html    # Airing calendar dashboard UI
β”‚   └── index.html       # Interactive API docs UI
└── README.md

πŸ”§ Local Development

  1. Clone and setup:

    git clone https://github.com/B1PL0B/MyDramaList-Unofficial-API.git
    cd MyDramaList-Unofficial-API
  2. Install dependencies:

    pip install -r requirements.txt
  3. Run development server:

    python -m uvicorn main:app --reload --port 9000
  4. Access:


πŸš€ Vercel Deployment

# 1. Install Vercel CLI
npm i -g vercel

# 2. Deploy
vercel --prod

Or use the one-click button at the top of this README.


⚠️ Important Notes

Episode Endpoints

  • /episodes/{n} β€” makes 1 extra HTTP request per call (the episode detail page)
  • /episodes/all β€” makes N extra requests (one per episode), batched 4 at a time with 0.5 s delays. Expect ~5–15 s for a 16-episode drama.

Vercel Limits (Free Tier)

Limit Value
Max execution time 10 s
Function size 15 MB
Cold starts Possible on first request

⚠️ /episodes/all may time out on Vercel's free tier for long dramas. Consider deploying your own instance or using individual /episodes/{n} calls instead.

Rate Limiting

  • 1 s delay on every endpoint entry
  • 0.5 s pause between episode batch groups

Calendar Caching

  • To optimize execution times and avoid aggressive scraping, /api/calendar caching is built-in.
  • Data is cached in memory for 1 hour (3600 seconds).
  • Responses will contain headers X-Cache: HIT (with X-Cache-Age in seconds) or X-Cache: MISS to indicate status.

πŸ” Error Handling

Code Meaning
400 Invalid parameters or private resource
404 Resource not found
500 Server-side scraping error

πŸ“ License

Educational use only. Please respect MyDramaList.com's terms of service.

🀝 Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Test thoroughly
  5. Submit a pull request

About

A serverless FastAPI-based web scraper for MyDramaList.com, designed for deployment on Vercel.

Topics

Resources

Stars

11 stars

Watchers

1 watching

Forks

Used by

Contributors

Languages