[extractor/Bilibili] Fixed bilibili bangumi extractors

[GD-Slime]

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

ADD DESCRIPTION HERE

Fixes #6701, Fixes #7400
This bug was caused by changes to the source website's webpage, which resulted in the inability to locate playback information within the webpage. The issue has been resolved by utilizing the API from the source site.
The API reference is derived from: https://github.com/SocialSisterYi/bilibili-API-collect .

Template

Before submitting a pull request make sure you have:

• At least skimmed through contributing guidelines including yt-dlp coding conventions
• Searched the bugtracker for similar pull requests
• Checked the code with flake8 and ran relevant tests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

• I am the original author of this code and I am willing to release it under Unlicense
• I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

• Fix or improvement to an extractor (Make sure to add/update tests)
• New extractor (Piracy websites will not be accepted)
• Core bug fix/improvement
• New feature (It is strongly recommended to open an issue first)

Copilot Summary

🤖 Generated by Copilot at 2d34de7

Summary

✨♻️🐛

Refactor the Bilibili Bangumi extractor to use new API endpoints and support more URL types. Add new classes for extracting playlists of episodes from media and season URLs. Rename BiliBiliBangumiIE to BiliBiliBangumiEpisodeIE. Update the extractor imports in _extractors.py.

Bilibili Bangumi
New classes and API calls
Autumn of refactor

Walkthrough

• Refactor Bilibili Bangumi extractor to support different types of URLs and use new API endpoints (link, link, link, link)
• Import new classes BiliBiliBangumiEpisodeIE and BiliBiliBangumiSeasonIE from bilibili.py module and add them to _extractors.py (link)
• Rename BiliBiliBangumiIE to BiliBiliBangumiEpisodeIE and inherit from BilibiliBaseIE instead of InfoExtractor (link)
• Change _VALID_URL pattern of BiliBiliBangumiEpisodeIE to match only episode URLs that start with ep followed by a number (link)
• Update _TESTS list of BiliBiliBangumiEpisodeIE to include a new test case for an episode URL and change expected id and title fields (link)
• Modify _real_extract method of BiliBiliBangumiEpisodeIE to use new API endpoints for getting video and season information and extract more metadata fields (link)
• Add episode_id, series_id, series, and subtitles fields to the returned dictionary of BiliBiliBangumiEpisodeIE (link)
• Change title field of BiliBiliBangumiEpisodeIE to include both episode title and long title (link)
• Handle region and premium restrictions and raise appropriate errors in BiliBiliBangumiEpisodeIE (link)
• Modify BiliBiliBangumiMediaIE to inherit from InfoExtractor instead of BilibiliSpaceBaseIE and use BiliBiliBangumiEpisodeIE for generating playlist result (link)
• Add geo_verification_headers to API requests and use traverse_obj helper function in BiliBiliBangumiMediaIE (link)

[catscarlet]

Tested with a bangumi site free video.

View Code

root@9f080ce2de5d:~/test# python3 pyinst.py
Building yt-dlp v2023.03.04 for linux x86_64 with options ['--onefile']
Remember to update the version using  "devscripts/update-version.py"
WARNING: Building without lazy_extractors. Run  "devscripts/make_lazy_extractors.py"  to build lazy extractors
Destination: dist/yt-dlp_linux

Running PyInstaller with ['--name=yt-dlp_linux', '--icon=devscripts/logo.ico', '--upx-exclude=vcruntime140.dll', '--noconfirm', '--additional-hooks-dir=yt_dlp/__pyinstaller', '--onefile', 'yt_dlp/__main__.py']
61 INFO: PyInstaller: 5.12.0
61 INFO: Python: 3.9.9
62 INFO: Platform: Linux-5.4.0-152-generic-x86_64-with-glibc2.36
63 INFO: wrote /root/test/yt-dlp_linux.spec
64 INFO: UPX is not available.
64 INFO: Extending PYTHONPATH with paths
['/root/test']
171 INFO: checking Analysis
171 INFO: Building Analysis because Analysis-00.toc is non existent
171 INFO: Initializing module dependency graph...
172 INFO: Caching module graph hooks...
178 INFO: Analyzing base_library.zip ...
682 INFO: Loading module hook 'hook-encodings.py' from '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks'...
1472 INFO: Loading module hook 'hook-pickle.py' from '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks'...
2035 INFO: Loading module hook 'hook-heapq.py' from '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks'...
2450 INFO: Caching module dependency graph...
2530 INFO: running Analysis Analysis-00.toc
2541 INFO: Analyzing /root/test/yt_dlp/__main__.py
2568 INFO: Loading module hook 'hook-yt_dlp.py' from '/root/test/yt_dlp/__pyinstaller'...
Adding imports: ['yt_dlp.compat._legacy', 'Cryptodome', 'websockets', 'websockets.__main__', 'websockets.auth', 'websockets.client', 'websockets.connection', 'websockets.datastructures', 'websockets.exceptions', 'websockets.extensions', 'websockets.extensions.base', 'websockets.extensions.permessage_deflate', 'websockets.frames', 'websockets.headers', 'websockets.http', 'websockets.http11', 'websockets.imports', 'websockets.legacy', 'websockets.legacy.async_timeout', 'websockets.legacy.auth', 'websockets.legacy.client', 'websockets.legacy.compatibility', 'websockets.legacy.framing', 'websockets.legacy.handshake', 'websockets.legacy.http', 'websockets.legacy.protocol', 'websockets.legacy.server', 'websockets.protocol', 'websockets.server', 'websockets.speedups', 'websockets.streams', 'websockets.sync', 'websockets.sync.client', 'websockets.sync.compatibility', 'websockets.sync.connection', 'websockets.sync.messages', 'websockets.sync.server', 'websockets.sync.utils', 'websockets.typing', 'websockets.uri', 'websockets.utils', 'websockets.version', 'mutagen', 'brotli', 'certifi']
2744 INFO: Loading module hook 'hook-xml.py' from '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks'...
2758 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks'...
2833 INFO: Loading module hook 'hook-certifi.py' from '/usr/local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
3165 INFO: Loading module hook 'hook-sqlite3.py' from '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks'...
3290 INFO: Loading module hook 'hook-websockets.py' from '/usr/local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
3423 INFO: Loading module hook 'hook-Cryptodome.py' from '/usr/local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
3654 INFO: Loading module hook 'hook-platform.py' from '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks'...
4200 INFO: Loading module hook 'hook-multiprocessing.util.py' from '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks'...
8492 INFO: Processing module hooks...
8809 INFO: Looking for ctypes DLLs
8880 INFO: Analyzing run-time hooks ...
8888 INFO: Including run-time hook '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks/rthooks/pyi_rth_inspect.py'
8889 INFO: Including run-time hook '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py'
8890 INFO: Including run-time hook '/usr/local/lib/python3.9/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
8911 INFO: Looking for dynamic libraries
10118 INFO: Looking for eggs
10120 INFO: Using Python library /usr/local/lib/libpython3.9.so.1.0
10130 INFO: Warnings written to /root/test/build/yt-dlp_linux/warn-yt-dlp_linux.txt
10197 INFO: Graph cross-reference written to /root/test/build/yt-dlp_linux/xref-yt-dlp_linux.html
10223 INFO: checking PYZ
10223 INFO: Building PYZ because PYZ-00.toc is non existent
10223 INFO: Building PYZ (ZlibArchive) /root/test/build/yt-dlp_linux/PYZ-00.pyz
10834 INFO: Building PYZ (ZlibArchive) /root/test/build/yt-dlp_linux/PYZ-00.pyz completed successfully.
10850 WARNING: Ignoring icon; supported only on Windows and macOS!
10851 INFO: checking PKG
10852 INFO: Building PKG because PKG-00.toc is non existent
10852 INFO: Building PKG (CArchive) yt-dlp_linux.pkg
18929 INFO: Building PKG (CArchive) yt-dlp_linux.pkg completed successfully.
18931 INFO: Bootloader /usr/local/lib/python3.9/site-packages/PyInstaller/bootloader/Linux-64bit-intel/run
18931 INFO: checking EXE
18931 INFO: Building EXE because EXE-00.toc is non existent
18931 INFO: Building EXE from EXE-00.toc
18931 INFO: Copying bootloader EXE to /root/test/dist/yt-dlp_linux
18932 INFO: Appending PKG archive to custom ELF section in EXE
19038 INFO: Building EXE from EXE-00.toc completed successfully.


root@9f080ce2de5d:~/test/dist# ./yt-dlp_linux https://www.bilibili.com/bangumi/play/ep469151      
WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this.
[BiliBiliBangumiEpisode] Extracting URL: https://www.bilibili.com/bangumi/play/ep469151
[BiliBiliBangumiEpisode] 469151: Downloading webpage
[BiliBiliBangumiEpisode] 469151: Extracting episode
[BiliBiliBangumiEpisode] Format(s) 720P  are missing; you have to login or become premium member to download them. Use --cookies-from-browser or --cookies for the authentication. See  https://github.com/yt-dlp/yt-dlp/wiki/FAQ#how-do-i-pass-cookies-to-yt-dlp  for how to manually pass cookies
[BiliBiliBangumiEpisode] 469151: Get episode details
[info] 469151: Downloading 1 format(s): 8+2
WARNING: You have requested merging of multiple formats but ffmpeg is not installed. The formats won't be merged
[download] Destination: 42-[469151].f8.mp4
[download] 100% of   37.86MiB in 00:00:04 at 9.24MiB/s
[download] Destination: 42-[469151].f2.m4a
[download] 100% of   18.16MiB in 00:00:01 at 10.52MiB/s
root@9f080ce2de5d:~/test/dist# 

Seems it works.

[bashonly]

@GD-Slime @jackyyf @catscarlet @Thalia500
if/when you get a chance, test my changes and let me know

[Thalia500]

Well I mean now it can work well when I give it a right URL.https://www.bilibili.com/bangumi/play/ep754144 is ok but https://www.bilibili.com/bangumi/play/ep754144?from_spmid=666.25.episode.0 is not even though the comtent is the same . But I think I find a way to solve it.

…

------------------ 原始邮件 ------------------ 发件人: "yt-dlp/yt-dlp" ***@***.***>; 发送时间: 2023年6月27日(星期二) 凌晨2:59 ***@***.***>; ***@***.******@***.***>; 主题: Re: [yt-dlp/yt-dlp] [extractor/Bilibili] Fixed bilibili bangumi extractors (PR #7337) @GD-Slime @jackyyf @catscarlet @Thalia500 if/when you get a chance, test my changes and let me know — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[bashonly]

@EnternalLR2
Don't manually modify your pip installation. It looks like parts of the patch were missed.
Either follow the instructions here: #7400 (comment)
Or use one of these builds: https://github.com/bashonly/yt-dlp/releases/tag/2023.06.29.164819