[XVideos] Support profiles, searches, channels and favorites

[rum416]

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

Imported from ytdl-org/youtube-dl#30774

The XVideos extractor supported single videos presented via various URL formats. The site also offers various playlist-like pages:

actual playlists: /favorite/ID/SLUG
related videos/playlist: video URL + #_related-...
channel activity: /...channels/ID
channel videos: channel URL + #_tabVideos
channel favourites (playlist): channel URL + #_tabFavorites
search results: ...?k=(ID).

This PR tries to support extracting from those pages.

Added /profiles tab support by me.

Template

Before submitting a pull request make sure you have:

• [x ] At least skimmed through contributing guidelines including yt-dlp coding conventions
• [x ] Searched the bugtracker for similar pull requests
• [x ] Checked the code with flake8 and ran relevant tests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

• I am the original author of this code and I am willing to release it under Unlicense
• [ x] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

• [ x] Fix or improvement to an extractor (Make sure to add/update tests)
• New extractor (Piracy websites will not be accepted)
• Core bug fix/improvement
• New feature (It is strongly recommended to open an issue first)

[rum416]

Example of downloading xvideos profile videos tab:

F:\downloader\xvideos>yt-dlp https://www.xvideos.com/profiles/cordon31#_tabVideos -f best -o "%(playlist_id)s\%(title)s [%(id)s].%(ext)s" --verbose
[debug] Command-line config: ['https://www.xvideos.com/profiles/cordon31#_tabVideos', '-f', 'best', '-o', '%(playlist_id)s\\%(title)s [%(id)s].%(ext)s', '--verbose']
[debug] User config "D:\OS Files\Users\X\AppData\Roaming\yt-dlp\config": ['--continue', '--format-sort', 'vbr,abr', '--http-chunk-size', '1M', '--buffer-size', '16K', '--sub-lang', 'english', '--no-embed-thumbnail', '--embed-metadata', '--abort-on-unavailable-fragment', '-o', '%(uploader)s\\%(title)s [%(id)s].%(ext)s', '-f', 'bestvideo+bestaudio/Source/hls-original/0/4320p/2160p/1440p/1080p/720p/540p/480p/240p', '--ffmpeg-location', 'D:\\OS Files\\Program Files\\ffmpeg\\bin']
WARNING: "-f best" selects the best pre-merged format which is often not the best option.
         To let yt-dlp download and merge the best available formats, simply do not pass any format selection.
         If you know what you are doing and want only the best pre-merged format, use "-f b" instead to suppress this warning
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.10.04 [4e0511f27] API
[debug] Lazy loading extractors is disabled
[debug] Python 3.10.4 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] Checking exe version: "D:\OS Files\Program Files\ffmpeg\bin\ffmpeg" -bsfs
[debug] Checking exe version: "D:\OS Files\Program Files\ffmpeg\bin\ffprobe" -bsfs
[debug] exe versions: ffmpeg 5.0.1-essentials_build-www.gyan.dev (setts), ffprobe 5.0.1-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.14.1, brotli-1.0.9, certifi-2021.10.08, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.2
[debug] Proxy map: {}
[debug] Loaded 1695 extractors
[debug] [XVideosChannel] Extracting URL: https://www.xvideos.com/profiles/cordon31#_tabVideos
[XVideosChannel] cordon31: Downloading webpage
[XVideosChannel] cordon31: Downloading webpage
[download] Downloading playlist: cordon31
[XVideosChannel] Playlist cordon31: Downloading 1 videos of 1
[download] Downloading video 1 of 1
[debug] [XVideos] Extracting URL: https://www.xvideos.com/video35527333
[XVideos] 35527333: Downloading webpage
[XVideos] 35527333: Downloading m3u8 information
[XVideos] 35527333: Checking hls-360p video format URL
[XVideos] 35527333: Checking hls-250p video format URL
[debug] Sort order given by user: vbr, abr
[debug] Formats sorted by: hasvid, ie_pref, vbr, abr, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, asr, proto, vext, aext, hasaud, source, id
[info] 35527333: Downloading 1 format(s): hls-360p
[debug] Invoking hlsnative downloader on "https://hls-hw.xvideos-cdn.com/videos/hls/a2/9a/73/a29a73105c0a9eb435dfd713e0e6dff0/hls-360p.m3u8?e=1665056036&l=0&h=e8bbb3228f5ea21e70096cd0f55e9987"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 211
[download] Destination: cordon31\Title or code of this please？ [35527333].mp4
[debug] File locking is not supported. Proceeding without locking
[download]   4.3% of ~ 154.23MiB at    1.80MiB/s ETA 01:20 (frag 9/211)

[pukkandan]

If you want to get this PR merged here directly, you must read through and obey our contributing guidelines (including removing pre-3.6 compat code). Otherwise, wait for it to be merged in ytdl, and I will port it here

[rum416]

I have read the contributing guidelines & yt-dlp-coding-conventions but I'm very new in coding. So I guess I'll just wait.

[pukkandan]

Would you be able/willing to make the necessary changes if I write a review?

[rum416]

Yeah, of course. Sure.

[pukkandan]

To start off, add tests - atleast one for each URL pattern

[rum416]

Did you mean for channel, profile, and pornstar urls? And paste the verbose logs here?

[pukkandan]

No, I meant add test cases to the extractors

[rum416]

You mean like this one?

    _TESTS = [{
        'url': 'https://www.xvideos.com/pornstar-channels/sienna-west',
        'playlist_mincount': 5,
    }, ]

[master-leonardo]

@rum416
Could you add the option to download from xvideos.red as well?
I don't think .red changes anything besides the address

[reverse123]

You can use this userscript for the playlist page. It will record all video urls onto the current page into the localstorage variable "my_vid_urls". You have to clear this variable if it exists and then open every page of the playlist you want to download. Do not click too fast, wait for the page to load completely, otherwise some pages will not have time to record. Then copy variable "my_vid_urls" from Chrome Devtools to new file list.txt, and pass to yt-dlp like this: "yt-dlp -a list.txt"
It can be easily modified for any lists with pagination.

// @name         url recorder for xvideos playlists
// @namespace    http://tampermonkey.net/
// @version      0.1
// @description  try to take over the world!
// @author       You
// @match        https://www.xvideos.com/favorite/*
// @icon         https://www.google.com/s2/favicons?sz=64&domain=xvideos.com
// @grant        none
// ==/UserScript==

(function() {
    'use strict';

    $('#content div.thumb-under > p.title > a').each(function() {
        let was = localStorage.getItem("my_vid_urls");
        //console.log(this.href);
        localStorage.setItem("my_vid_urls", was+"\n"+this.href);
    });
    let arr = localStorage.getItem("my_vid_urls").split("\n");

    var index = arr.indexOf('null');
    if (index !== -1) {
        arr.splice(index, 1);
    }

    let uniqueArr = [...new Set(arr)];
    localStorage.setItem("my_vid_urls", uniqueArr.join("\n"));
    console.log(uniqueArr);
})();

[dirkf]

Could you add the option to download from xvideos.red as well?

@master-leonardo, I'd be happy to do that in the to-be-updated yt-dl PR but how would it be tested?