Extractors

Extractors

🌐 English · Deutsch

Extractors resolve a video host's embed URL into a direct stream link (usually HLS/MP4) that yt-dlp/FFmpeg can download.

Host status

Enabled (SUPPORTED_PROVIDERS in config.py):

Host	Module	Notes
VOE	voe.py	Multi-stage deobfuscation (ROT13, junk patterns, Base64), captcha support
Vidmoly	vidmoly.py	Requires referer https://vidmoly.biz
Vidoza	vidoza.py	—
VeeV	veev.py	Custom headers (referer/origin veev.to), availability check via /api/veev/check

Present but disabled (commented out in SUPPORTED_PROVIDERS): Doodstream, Filemoon, LoadX, Luluvdo, Streamtape, Vidara.

hanime (special case — browser-based, no third-party hoster)

hanime.tv doesn't use an external video hoster; it serves its own AES-128 encrypted HLS. Since its Astro rewrite the site also signs every /api/v8 request (handshake via auth.hanime.tv + a per-request signature) and the player only loads the stream after the poster is clicked, so plain HTTP can't reach the data. The provider therefore splits its work:

• Browsing / search — the unsigned search.htv-services.com host, plain HTTP (no browser). This is why start-page cards load instantly.
• Detail + stream — a headless browser (patchright, same dependency as VeeV) in models/hanime_tv/browser.py. fetch_video(slug) reads metadata from the page's ld+json + DOM (title, poster, tags, censored, franchise episodes) and, when a stream is needed, clicks the play overlay and intercepts the …highwinds-cdn.com/….m3u8 request. Results are cached for 5 min with a per-slug lock so opening a title spawns only one browser.

The extractor extractors/provider/hanime.py exposes get_direct_link_from_hanime (returns the m3u8 via the scraper) and download_from_hanime.

The stream is AES-128 encrypted, so yt-dlp cannot download it directly — the segments have to be decrypted first. The download is therefore done by a single FFmpeg pass, which fetches the segments and decrypts AES-128 natively (-allowed_extensions ALL, since the segments are disguised as .html; crypto in the protocol whitelist; and a Referer so the sign.bin key can be fetched). The known manifest duration gives a real progress bar.

Auto-discovery

extractors/__init__.py imports all modules from extractors/provider/ and registers every function whose name starts with get_direct_link_from_ or get_preview_image_link_from_ in the provider_functions dict:

provider_functions["get_direct_link_from_voe"](embed_url)

At app startup, _get_working_providers() (web/app.py) probes each entry from SUPPORTED_PROVIDERS with an empty URL string: if the extractor raises NotImplementedError it counts as not implemented; any other exception means "implemented" (it correctly rejected the empty URL). Only functional providers appear in the UI (WORKING_PROVIDERS).

Adding a new extractor

1. Create the file: src/mediaforge/extractors/provider/<host>.py
2. Implement the function (mind the naming convention — <host> lowercase):

import logging
logger = logging.getLogger(__name__)

try:
    from ...config import GLOBAL_SESSION, is_source_unavailable
except ImportError:
    from mediaforge.config import GLOBAL_SESSION, is_source_unavailable


def get_direct_link_from_<host>(embed_url: str) -> str:
    if not embed_url:
        raise ValueError("Empty URL")
    resp = GLOBAL_SESSION.get(embed_url)
    if is_source_unavailable(resp.text, resp.status_code):
        raise ValueError("Source removed")
    # ... parse HTML/JS, extract the stream URL ...
    return direct_link

3. Enable the provider: add its name to SUPPORTED_PROVIDERS (config.py) — spelling must match func_name = f"get_direct_link_from_{p.lower()}".
4. Define headers: if the host needs special headers, add them to PROVIDER_HEADERS_D (download) and possibly PROVIDER_HEADERS_W (watch) in config.py — these are passed to yt-dlp/FFmpeg.
5. Optional: get_preview_image_link_from_<host>() for preview images.

Helpers from config.py

Function	Purpose
GLOBAL_SESSION	Thread-safe niquests session with realistic headers and DoH DNS
is_source_unavailable(html, status)	Detects removed videos (404/410/451 + HTML markers) without an extra request
check_redirect_available(url)	Follows the streaming site's redirect and checks whether the host still has the video (curl_cffi with Chrome impersonation against Cloudflare)
resolve_redirect_url(url)	Returns the final host URL behind a redirect

Captcha integration

When an extractor detects a captcha page (is_captcha_page from playwright/captcha.py), it can call solve_captcha(...). If the call runs inside the queue worker, the queue_id is set via thread-local — which makes the solving interactive in the WebUI (live screenshot + click forwarding). Chromium runs with stealth patches, network ad blocking and protection against ad overlays.